课程: Hands-On AI: Build a Generative Language Model from Scratch
Building your model - Python教程
课程: Hands-On AI: Build a Generative Language Model from Scratch
Building your model
- [Instructor] So we're just about ready to train the model that I've set up for you, and if you're following along in the exercise files, we're in 01, 01_3_begin, where right at the top, I bring in the random module, as well as punctuation from string, and that default dictionary from collections. Now, the kind of model that we're building is considered a type of what's called a Markov chain. And that's why on line seven, you'll notice that I define a class of Markov chain. Then, I initialize the class and give our model a default dictionary that produces lists as its graph, and that's going to hold our raffle possibilities for each token. Now, more advanced models have sophisticated ways of tokenizing. Our model is basically going to get rid of all punctuation in numbers and split things up, removing any spaces and new lines. Then, there's that train method, and this one, we're going to implement together in this video. So what am I going to do? The first thing I'm going to do is I'm going to loop through my tokens. So for token, and that's not going to be enough information, so I should probably use enumerate. And if I do, I need some sort of counter, which is going to be i. This is going to allow me to know what the next token is. And the first thing I want to know is if I'm in the last token of the list, so that I don't try to look up the next token. So if... And I'll say that the length of tokens minus one equals i, I'll go ahead and break the loop. And this is the case where I'm at the last token. Otherwise, I'll say that self.graph, and I'm going to look up what's at the key of my token. And if there's nothing, it'll give me an empty list, which is fine. And to that, I'm going to append my next token, which is going to be at tokens, and that's going to be i plus one. And if I didn't do this check right here, then I would be at an index out of range in my last token. So running this is going to build out that chart that's going to allow me to run my raffles in my generate method. Now, for the generate method, I'm going to take a prompt and the length of the generation. The first thing I'll say is, what's the current word that I'm looking at? What's the last word of the prompt? Then I'm going to say that output begins by being the prompt itself. Then I'm going to do a for loop, the amount of times that the desired length of the generation is. I'm going to grab some options, and if there are no options, I'm going to continue. Now, in the coming challenge, I want you to think of how you can produce the right output. How can you run that raffle, and what do you do with the results?