My chat

LLM models become a new "internet" nowadays. Jumping on the train of big models is too late, it has to be either a big budget or a strong open-source community. In that sense, the focus is shifting to the users of the models. It's time to build interesting services based on already existing platforms and that's what happening already now.

I was curious to try to set up my own chat app based on some open-source LLM. The main question was: how expensive that can be and what quality I might expect to achieve for the budget of one person ( you may think small startup)?

Well, the result you can check here:

https://chat.relaximus.com

Note: be patient sometimes it takes 20-30s to wake up for the chat.

Comparison

Chat based on the open-sourced engine Ollama for onboarding different LLMs. It turned out, that it is a very convenient way of using any kind of LLM.

Example of code question to my chat:

Same question for OpenAI ChatGPT:

One more example of a simple text-based question requiring geographical context:

And the same question to ChatGPT:

To summarize

Although it might seem like nothing special, the difference is huge. Having the approximately same quality for simple questions, I have full control over what where, and how is being deployed and running. What API to expose, what data to add, and even preconfigure the model in a slightly different way.

I am not restricted by the pricing model of Open AI or other "big players", I can scale the way I want. This is a huge advantage and all of that became possible because of the rapid growth of the community and ecosystem around popular LLMs.

So, the next time I need to introduce these capabilities to the project I'll have much more options than it seems on the first view.

P.S. I am going to write the next part describing a bit more architecture of this chat deployment. I made quite a few interesting findings which I want to share with you. Stay tuned!