The Big Problem with Open Source LLMs

Meta's LLAMA2 and the various open source models from Mistral have become quite capable. The recently released Mistral Large is supposed to be better than Open AI's GPT-4 the current leader. LLAMA3 is imminent and will likely challenge GPT-4.

One very important reason to pursue the open source LLMs is the confidentiality needs of most Fortune 2000 companies. We can run these on private hosted infra and not make any API calls to any public hosted LLMs like GPT4/3.5/Claude etc

However the biggest problem with running these models is the GPU infra required for it to give reasonably fast responses [I am talking about the full-blown models and not about the quantized models which can be run even on your phone]. This high power GPU Infra is not easy to put together on shoestring budgets especially when you don't yet have the business case.

Hugging Face Assistants

Enter Hugging Face Assistants platform, a direct competitor to the Open AI GPT Builder and the GPT Store [ I wrote a post a couple of weeks about the GPT Store phenomenon].

When I heard about this, I got very excited. I took my first ever GPT on the GPT Store and repurposed it to run on the Hugging Face Assistants Platform using LLAMA2 and the results were almost as good as the GPT4 in most cases. You can try it here. All our other agents including this one are on this page.

Over the last couple of months, I have built several agents for our customers on the Hugging Face platform to classify incidents, generate python code, reverse engineer COBOL code, update JSONs.

Please try it build your own agents and share your experiences here. Now will this kill the GPT Store as mentioned in the cover image? Nah. That was just to make you read the post :)

Tiny Curator's desk - one of the biggest production use cases by way of business impact I have heard so far is this one from Klarna.

If you like this newsletter, please share it with your colleagues and friends.

#TMT3 #secretsofgenaidigitaldisruptors #genai