课程: Generative AI: Working with Large Language Models

今天就学习课程吧！

今天就开通帐号，24,700 门业界名师课程任您挑！

Scaling laws

Scaling laws

课程: Generative AI: Working with Large Language Models

开始一个月试用 购买团队方案

Scaling laws

“

- [Instructor] Up to this point, we've looked at a couple of models, but now is a good time to try and understand why we have such large parameter models. Around the time of the release of GPT-3, the OpenAI team released some results around what they called the scaling laws for large models. They suggested that the performance of large models was a function of the model parameters, the size of the data set, and the total amount of compute available for training. They performed several experiments on language models. Let's take a look at some of the results. On the Y axis is the test loss. The test loss will converge for each of the models. So the lower the test loss, the better performing the model. Across the x axis is the number of parameters of the model. You can increase the sizes of these models by making them wider or increasing the number of layers. So as we go across, we're going with models with a hundred thousand to…

内容