课程: Generative AI: Working with Large Language Models

今天就学习课程吧！

今天就开通帐号，24,700 门业界名师课程任您挑！

Inference time vs. pre-training

Inference time vs. pre-training

课程: Generative AI: Working with Large Language Models

开始一个月试用 购买团队方案

Inference time vs. pre-training

“

- [Instructor] Over this course, we've looked at scaling laws and lessons from the chinchilla models. Let's bring this all together as we look at the current trends for large language models. When training of large language models first kicked off, most of the focus was on improving the models, and then the scaling laws were the guiding principle, and the scaling laws suggested that you would get the biggest improvement by increasing the size of the models. So you do this by increasing the model's architecture, so that's the number of layers in a model, the number of attention heads, and so on. But that was only one dimension because as model providers created large models, the associated training cost became an important consideration. So the second important dimension became training cost, because being able to train a model effectively given a budget was important. And the learnings from the chinchilla paper earlier in the course suggested that large language models were being…

内容