Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

https://learning-at-home.github.io/

Collaborative inference of latent diffusion via hivemind

Training a model similar to OpenAI DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)

Memory-efficient transformer. Work in progress.

This is an early in-development version of training CLIP models with hivemind.