DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
https://code.morphllm.com/deepspeedai/DeepSpeed.git billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero