Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead

Phi mái chèo đại mô hình khai phá bộ kiện, cung cấp đại ngôn ngữ mô hình, vượt mô thái đại mô hình, sinh vật tính toán đại mô hình chờ lĩnh vực toàn lưu trình khai phá công cụ liên.

LiBai( Lý Bạch ): A Toolbox for Large-Scale Distributed Parallel Training

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.

Distributed Keras Engine, Make Keras faster with only one line of code.

Ternary Gradients to Reduce Communication in Distributed Deep Learning (TensorFlow)

Orkhon: ML Inference Framework and Server Runtime

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

Multi-GPU training for Keras

Distributed training (multi-node) of a Transformer model

SC23 Deep Learning at Scale Tutorial Material

WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.

☕Implement of Parallel Matrix Multiplication Methods Using FOX Algorithm on Peking University's High-performance Computing System

This repository provides hands-on labs on PyTorch-based Distributed Training and SageMaker Distributed Training. It is written to make it easy for beginners to get started, and guides you through step-by-step modifications to the code based on the most basic BERT use cases.

Understanding the effects of data parallelism and sparsity on neural network training

Fast and easy distributed model training examples.

A mostly POSIX-compliant utility that scans a given interval for vampire numbers.