Open Source Solution Replicates ChatGPT Training Process

ChatGPT is the biggest buzz in AI today!

ChatGPT demonstrates remarkable capabilities so there is a high interest to replicate it. Colossal-AI just open-sourced a solution that replicates the ChatGPT training process.

One of the most important implementation details of ChatGPT is RLHF (Reinforcement Learning with Human Feedback).

RLHF essentially involves a reinforcement learning framework that allows LLMs to fit and capture human preferences. That’s the magic sauce behind ChatGPT.?

While ChatGPT is excellent for many tasks, it is closed source so there is a high demand for an open-source ChatGPT equivalent.

The open-source library, Colossal-AI, made a recent release that allows replication of the ChatGPT training process at significantly low costs and that reduces hardware restrictions.

Here is what Colossal-AI’s ChatGPT equivalent implementation process offers:

• A mini demo training process requiring only 1.62B of GPU memory
• 7.73 times faster single-machine training?
• Easy and efficient fine-tuning
• Support for different models and sizes

Why does it matter?

The introduction of reinforcement learning means that there will be more model calls as there are more components to optimize (policy, reward, etc.).

In addition, the hardware requirements (e.g, GPUs) make it challenging to reproduce a ChatGPT-like system.

Colossal-AI greatly reduces the GPU memory overhead of ChatGPT training which can significantly reduce the cost of ChatGPT-style applications.

“It only requires half the hardware resources to start 175 billion parameter model training (from 64 cards to 32 cards)”.

With this new release, speedup improves by 7.73 times for single-server training and 1.42 times faster for single-GPU inference.?

"Colossal-AI also boosts the capacity of a single GPU by 10.3 times to 8 billion parameters."

That’s impressive!??

Those improvements mean that for ChatGPT training based on a small model of 120 million parameters, a minimum of 1.62GB of GPU memory is required.

Here is the best part:

Colossal-AI provides out-of-the-box ChatGPT training code and support for mainstream pre-trained models like GPT and OPT.

Here is how it might look in code:

For a usage example, check out this script showcasing simple usage on the Colossal-AI repo: https://github.com/hpcaitech/ColossalAI/tree/main/applications/ChatGPT

---

Useful links:

Full blog post: https://www.hpc-ai.tech/blog/colossal-ai-chatgpt

Colossal-AI repo: https://github.com/hpcaitech/ColossalAI

Twitter: https://twitter.com/HPCAITech

LinkedIn: HPC-AI Tech

Join the Slack group to engage with the Colossal-AI community: https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w