What's hot in AI ? "Not just a parlor trick". Feb 2nd, 2024

Like mentioned last week, I am going to pick things that matter to me personally and share thoughts. I care about performance jumps in models, new architectures if relevant and efforts to increase efficiency. Also products. Real products in people's hands. Not going to cover fundraises.

Let's start.

1. There is a lot of drama in the world of AI , and a ton more expectations related to getting our hands on the first open source models that can beat or rival GPT4. We are all anticipating it. Recent leaks on different channels and Arthur Mensch's confirmation indicates that there is a model update and that it is promising. Mistral delivering at incredible speed is no news, I guess. The so-called ''Miqu'' is speculated to be a hint of the model using quantization. This technique is crucial for deploying AI models more efficiently, especially on devices with limited processing power or memory, such as mobile phones or embedded systems. Given there are claims Miqu can match GPT4 on certain tasks or is even better, quantization would be logical and in line with Mistral's overall mission of bringing cutting edge AI into the hands of every developer. I really want a super powerful open source model. I want it to be efficient too. Quantization comes with shortcomings and at the cost of accuracy and performance, hence curious to see if we actually have reached a milestone in delivering large model performance at local costs and latency. The Holy Grail of AI today.
2. Amazon's Rufus is another big company attempt to deliver generative AI at a large scale. Given their massive distribution and the potential utility of Rufus, adoption trends would be very curious to follow. I do not think asking Rufus what to buy is better than searching and getting hinted on what other people bought. The unimaginative approach of building another chat assistant is a bit meh. I may be wrong. A lot of the big releases have been flops so far. Most of the first assistants introduced by the big boy club of tech companies are forgotten, while we return to Chatgpt and Perplexity for our mundane affairs. This is a first move and scratching the surface. Of course we are excited to see what Amazon has in store.
3. Every week I play with GPTs in the GPT store . Rarely do I find anything that I go back to twice. Even adding GPTs to my conversation is not going to change it for me. But I am hopeful. I keep looking. This week I decided to make another one of my own. I went to the good old idea of making a chatbot, acting as a therapist. Understanding all the ethical implications, let us omit them for the purpose of this exercise. After all, the world's first chatbot was a therapist. Hi Eliza ! ELIZA is the predecessor of all chatbots and it was made in the 60s. ELIZA functioned through pattern matching and substitution methodology, where it would recognize certain keywords or phrases input by the user and respond with pre-programmed responses. The most famous script created for ELIZA was "DOCTOR," which simulated a Rogerian psychotherapist. This script allowed ELIZA to engage in conversations where the chatbot would primarily reflect questions back to the user, a technique that encouraged the user to continue talking. This method of interaction was surprisingly effective, leading some users to attribute human-like understanding and empathy to ELIZA, despite Weizenbaum's insistence that ELIZA was nothing more than an elaborate parlor trick without any understanding of the conversation.So what is possible today? There are many GPTs in the store acting as a therapist. I did not like any one of them, so I prompted Chatgpt with a simple instruction to make sure it does not interrupt me, listens and asks appropriate questions each time going deeper into my responses. I also incepted it with ideas from Gestalt therapy where the focus is on exploring what the client is feeling in the moment, via the lens of client-therapist relationship. After a 20 minute chat, I was in tears. The fact that a matrix multiplication can make you cry is insane to me . Also incredibly exciting. If you are building in this space, I think there is an incredible opportunity to craft something meaningful. Given psychotherapy is a scare resource and therapist matching is very hard, the opportunities to scale with this technology and transform our relationships with ourselves are vast. It's not a parlor trick any more, it is real.

4. Last but not least, if you are in NYC next week, we are hosting "LLM-s x Finance" panel at Barclays Rise with builders from the space, including Hyperplane and Basis co-founders. Deployment issues, feature discovery, reliability, price, fine-tuning or RAG and more questions will be unpacked.

If you are curious about my thesis about Vertical AI, here is what I think:

SaaS is not dead. Selling SaaS products will get increasingly more expensive because of the barriers of building in-house systems lowering and procurement getting more sophisticated. SaaS + AI , ie a system of intelligence on top of a system of record is a valid play, only if the intelligence you are delivering has real core product utility vs is adding marginal impact.

Vertical AI is a term I dislike, however I believe there are teams of people building deep in the vertical woods, who are going to unlock performance and utility and figure out the unit economics of this whole thing way faster than the horizontal ops players.

My second observation is that if you can unlock close to infinite customization of software and product suites based on the user persona and the actions they are performing, you are not only not dead, but you have paved the future of SaaS and B2B software. Chamath's 80/90 game is way harder than he thinks. It is not about 80% of the functionality, but cheaper, it is about custom functionality speaking to internal systems seamlessly at the same price and speed. We would pay more, as it would still end up cheaper, than doing it ourselves, no matter how good Gihub Copilot and Cursor AI are. Focus is an operational leverage.

Rant over. Have a fun weekend ! We have been doing a deep dive on RAG and unit economics at Ntropy ! Will share more next week.