Pocket LLM or how to automate your repetitive tasks with Ollama
Intro
In this post, I will tell you how to use the Ollama library for your routine tasks – let's talk a little about how to perceive LLM from the point of view of the layman, install the library and run the model locally on a PC (without SMS and registration), as well as create our own assistant who will solve some task.
Despite the fact that quite a lot has been written about LLM and how to use them, another generalizing article, if it helps at least one person to start using these models for their own purposes, is already worth the time spent on it (and yes, this text was written by me personally and not by artificial intelligence).
What does the average person not from the world of IT or mathematics know about LLM? He will probably say that there is a ChatGPT who knows a lot and can answer the questions posed. In general, it's not bad – but what if we try to develop this idea – what are the questions? Let's make a list of a couple of banal chat requests:
Despite the fact that the questions look quite complete, we can divide them into a general statement of the problem – what is called an instruction, and into context – i.e. variable parameters. In this case, our requests can be presented in the following form:
? Instructions {Show interesting places that you can visit over the weekend} in the context of {Berlin}
? Instructions {Write a 10-sentence long post on a given topic} context {Blogging on LinkedIn}
? Instructions {Build a route between two points with 5 stops in historical places} context {Barcelona, Munich}
As we can see, each request requires certain instructions, which are rather inconvenient to pass each time when describing our task (especially if we repeat them many times), so we can use assistants – specially configured services that know their task, and take only the context as variables – that is, the specific conditions of the task.
Setting up the environment and the first assistant
To automate the solution of our tasks, we will need to install the Ollama application, this can be done via the link: https://ollama.com/download
Download and run the application following the instructions – if everything went without errors, you will see Ollama in the list of running programs.
Next, go to the code editor, for example Spyder, and import the necessary extensions.
Now we can move on to writing instructions – that is, a description of the task for our assistant. Let's create the get_response function, which will accept a string variable as input, and write the instruction directly in the body of the function.
领英推荐
We specified two messages in the msgs variable – the first with the "system" role, in which we specified the instruction, and the second with the "user" role, we simply pass the original query string. We pack these messages into the parameters of the ML assistant call and pass the assistant's response to the returned parameter of the function – that's all at this step.
Now we can call the function and write the response to a variable.
We received a text with a list of places for the transferred location – Berlin. Let's change the location, for example, to Madrid.
Super – we just called our function from a new location and got the necessary answer, while we did not have to rewrite all the instructions and re-explain exactly what we need.
Implementing an assistant in the workflow of an application
Despite the fact that the assistant returns information to us in a human–readable form, we may need to standardize responses to a format that other parts of the application can work with. As we can see, in addition to the list of places in our case, the assistant gives introductory words and additional remarks that make it difficult to further work with the data.
The most convenient format for communication between individual parts of applications is the json format, so let's try to configure the assistant so that we can transfer its response to the following application modules without manual processing.
To do this, create a json schema in the function body in which we want to see our response.
We explicitly specified the response format by describing the scheme and passing it as an addition to the instructions and got this result
Now, we can parse the response in the usual way by running a loop through the places elements in our json file and transfer each of the proposed places isolated from others to our next functions – this may be a page on the site with telegram bot tours or any other.
Conclusion
As you can see, in less than 10 minutes we have set up our personalized assistant who can solve our routine tasks. Of course, after this stage, further ones follow – setting up answers for specific tasks, experimenting and fine-tuning models, and others, but the stage of immersion itself to the extent that a person initially unfamiliar with the use of ML assistants can independently use them for their tasks seems as simple as possible.