Support a collection of parallel training components

LiBai provides multiple parallelisms such as Data Parallelism, Tensor Parallelism, and Pipeline Parallelism. It's also extensible for other new parallelisms.

Varied training techniques

LiBai provides many out-of-the-box training techniques such as Distributed Training, Mixed Precision Training, Activation Checkpointing, Recomputation, Gradient Accumulation, and Zero Redundancy Optimizer(ZeRO).

Support for both CV and NLP tasks

LiBai has predefined data process for both CV and NLP datasets such as CIFAR, ImageNet, and BERT Dataset.

Easy to use

LiBai's components are designed to be modular for easier usage as follows:

• LazyConfig system for more flexible syntax and no predefined structures
• Friendly trainer and engine
• Used as a library to support building research projects on it. See projects/ for some projects that are built based on LiBai

High Efficiency