课程: Hands-On AI: Build a Generative Language Model from Scratch

Solution: Markov chains - Python教程

课程: Hands-On AI: Build a Generative Language Model from Scratch

开始一个月试用

Solution: Markov chains

“

(upbeat music) - [Instructor] So the goal of this challenge was to complete this generate method of our model so that after we've trained our model we can give it a prompt and receive a completion for it. And the key to doing this is here in this for loop where we're going to constantly redefine the current token or the state of our model. So on line 31, you see that I grabbed the last token as current but here in the loop I'm going to say that the current is random choice of options. And each time I'm going to grab new options using my current token. So next what I have to do is I have to add this to my output. So I'll append my output. And for this, I like to use a format string that way it's a clean way to add spacing. And now we're just about ready to test it out. Now keep in mind the quality of the completion is going to vary. Sometimes it'll sound a lot like an English sentence, sometimes less so and this is part of the fun with this kind of model. So let's go ahead and test my code. He was later by directly accessing the great grandson of Helsinki in, let's see, computer. So this is semi English sounding and there are ways of optimizing this using Markov chains. But eventually there are different models for getting more realistic sounding language. Let's try another one. He was born in late Torvalds so hard to the university. Let's try another one. He was later by a penguin nicknamed Tux which uses the. So you see it's English sounding and it almost sounds like it could make sense. Now when you only look at the last token, you're going to have this situation where it's going to sound like language, sometimes it'll make less sense. Sometimes you can look back at more than one token, and there are definitely different models that can look farther back. In the next video, we're actually going to use the fact that our model looks at one token at a time to have a little bit of fun.

内容