课程: Introduction to Large Language Models

今天就学习课程吧！

今天就开通帐号，24,700 门业界名师课程任您挑！

How are large language models trained? Pre-training

How are large language models trained? Pre-training

课程: Introduction to Large Language Models

开始一个月试用 购买团队方案

How are large language models trained? Pre-training

“

- [Instructor] We've seen an example of a large language model at work and the results are pretty impressive. But how do you go about training a large language model? And that's what we're going to look at in this video. Initially, the language model has random weights, and at this point, the model has no knowledge of language. Now, if you were to prompt it, it would just return gibberish. But if you train the model and pass it a large corpus of data, it adjusts these weights as part of the training process. And this pre-training stage is very resource heavy. So you need lots of data, and this includes a variety of different types of data like books and articles and websites. Let me give you an example. LLaMA was a group of language models that were released in 2023 by Meta. And this is a data mixture of Meta used for pre-training. So Common Crawl and C4 are web scripts of the internet that have been cleaned and filtered.…

内容