nebulgym: open-source training accelerator release

nebulgym open-source

Easy-to-use library to accelerate AI training leveraging state-of-the-art optimization techniques ??♀?

nebulgym greatly reduces the training time of AI models without requiring any modification to the training setup. nebulgym optimizes the full training computing stack, from efficient data loading, to faster forward and backward passes, to earlier convergence.

No matter what model, framework, or training recipe you use, with?nebulgym?you speed up training by simply adding?nebulgym?class decorators to your code. These short lines will make sure that you use your hardware's computing power to the fullest and achieve the shortest possible training time.

Your code + @nebulgym_class_decorators = superfast training ??♀?

???Superfast. The library speeds up training and thus the time it takes you to test your model, reduces computing costs and energy consumption.

???Easy-to-use. Just add?nebulgym?class decorators to your code and continue programming on your favorite training framework.?nebulgym?will let you achieve awesome training times.

???Training setup agnostic.?nebulgym?can be coupled with any model, trainer, or other training technique to achieve a compound effect on training performance.

???Framework agnostic.?The library aims to support all frameworks (PyTorch, TensorFlow, Hugging Face, Jax, etc.) to allow any developer to continue working with the configuration they are used to. Currently,?nebulgym?supports PyTorch and we are working on expanding?nebulgym?capabilities.

???Deep learning model agnostic.?nebulgym?supports all the most popular architectures such as transformers, LSTMs, CNNs, and FCNs.

???Hardware agnostic. The library aims to support any artificial intelligence hardware on the market, from general-purpose GPUs and CPUs to hardware accelerators such as FPGAs and ASICs. Currently,?nebulgym?has been tested on many CPUs and GPUs.

Supported tech & roadmap

nebulgym has just been launched, and it is already capable of cutting training time in half. At the same time, it is expected that it may crash or fail in untested use cases. Moreover, the project is in its early stages and there is a lot of room for improvement for nebulgym to become a new paradigm for artificial intelligence training.

The library aims to support every framework, every model, every hardware, and make the most of your hardware and software capabilities to train your model in a fraction of the time required now. In addition, nebulgym will always be extremely easy to use to empower any developer to build powerful AI applications.

nebulgym already embeds many great technologies, and in the documentation you can find a list of the features already implemented and those that will be implemented soon. Any ideas about what could be implemented next? Would you like to contribute to this fantastic library? We welcome any ideas, questions, issues, and pull requests! For more info join the community

Do you like the library??Leave a ? on GitHub

Happy training! ???