课程: Generative AI: Working with Large Language Models

今天就学习课程吧！

今天就开通帐号，24,700 门业界名师课程任您挑！

Transformers: History

Transformers: History

课程: Generative AI: Working with Large Language Models

开始一个月试用 购买团队方案

Transformers: History

“

- [Instructor] The models based on the original transformer paper from 2017 have evolved over the years. One of the challenges with training large language models in 2017 was that you needed labeled data. This required a lot of time and effort. The ULMFiT model proposed by Jeremy Howard and Sebastian Ruda provided a framework where you didn't need labeled data. And this meant large corpus of text, such as Wikipedia, could now be used to train models. In June of 2018, GPT or Generative Pre-Trained Model, which is developed by Open AI, was the first pre-trained transformer model. Now, this was used for fine tuning on various NLP tasks and obtained state-of-the-art results. And a couple of months later, researchers at Google came up with BERT or Bidirectional Encoder Representations from Transformers. We saw a couple of examples of BERT being used in production at Google. In February, 2019, Open AI released a bigger and…

内容