Ntropy

In the last ten to twelve months, as a company and as founders and individuals in the LLM space, Ilia Zintchenko and I have been talking about cost of inference a lot. We also have been working on making sure you can bring that cost down at scale for highly valuable, high throughput and complex use cases where bigger models outperform small and specialized ones. Today, looking at recent performance of SOTA models that are better yet cheaper and faster, many will extrapolate that inference is solved and the costs are trending down. Competition artificially pushes the costs down even more and many companies are still bleeding money on this, trying to get batch sizes in the longer run. However, our argument stands and there is a massive problem to solve here still. Here is why 1. We have not seen major reasoning leaps in new models yet , and all the compute build-up means scaling this architecture is still on the map. Current LLM-s are good enough for many things , yet have a long way to go. There is a very high probability that the next model that leaves GPT4 in dust not just by shaving off a few benchmark percentage points here and there, but by seriously unlocking things that were not possible before, is going to be larger and much more expensive to run. 2. The cheaper the inference gets, the higher the demand, hence more things become possible that were economically prohibitive before, expanding this market even further. There is a reason to make things that are becoming cheaper already, even more accessible. 3. Finally, the most valuable LLM outputs and products are going to be context dependent and very recursive (e.g. agentic workflows and interactions) hence they require many LLM calls back and forth meaning that the trend of inference becoming cheaper has a smaller net effect on those bills. We need orders of magnitude faster and cheaper options here, while not sacrificing accuracy and while ensuring reliability. How's your Saturday going?

When you have smth of a huge utility and horizontal utility in fact, and the price of it goes down significantly ; you are going to see increasing amount of demand , you are going to see use cases never deemed feasible , everyone is going to want much more of it . Jevon’s paradox and all that . The world has never had this much of affordable intelligence at our finger tips and today it’s the worst it’s ever going to be. This does not make compute and compute spend overhyped . In fact it makes it clear how much more inference we’re going to need at every corner , in everything we do, big or small. Never been more excited about NVIDIA and the whole space. This just means we’re going to be building more and scaling more, my friends ?? Image courtesy goes to Dylan Patel from SemiAnalysis

Automating manual and tedious tasks that humans are not good at and do not want to do is the core idea behind RPA. This has been more recently at the center of the LLM/AI conversation too. Particularly in the media. Many consider this as an opex improvement or a way of turning opex into capex, unlocking the ability to scale infinitely with capital. While this is true and is certainly a solid version of the future, there is a big perspective missing. Using models and agents vs humans for these tasks achieves one thing that is universally agreed on : acceleration. Acceleration has super interesting second order effects. Imagine every process you do, you could accelerate the by 100x , 1000x, 10000x. Suddenly, you can do more in parallel. You can unlock invention and new outcomes. There are paths to new insight that are either located in between cross silo data interactions or the time spent on higher levels of abstraction. They were previously less possible to reach because of the energy spent on the lower level task completion. If you are an enterprise thinking about why and how you should consider using LLM-s and automation beyond the PR value on your stock, here are a few points to consider: - How can I accelerate across every single function? If we were to do x ten times faster, what does that unlock? - If every single one of my employees can operate on a higher level of abstraction, essentially being promoted to an architect of systems vs operator, what does that unlock? - What are the type of people I can hire and how will they impact the business if I can make everything 10x or a 100x faster?

This was one of the most insane years of my life . Made things happen that I never thought possible. Failed and got up more than ever thought capable of. Had the darkest and the brightest hours. At Ntropy, we built and launched and changed and shrunk and grew. We parted with team members through spring and summer. This was rough. We broke even for the first time in August. This was fun. We fixed our gross margins, improving from negative to over 70% positive Burn multiple dropped below 1 for the first time. Revenue grew over 3x We signed our first customers that are public companies We launched an end to end caching infrastructure allowing us to get the outputs of the largest models over 100x cheaper and faster The market we were building for, machine learning engineers and models consuming financial data was big, yet slow to adapt and very tricky for probabilistic inputs. The market we ended up in, financial services workflows and agents consuming unstructured financial data, is brand new and is going to be larger than anything we thought of before. We updated our priors multiple times and are continuing to do so. Just like our LLM caches that keep growing and improving. We have never seen such hunger for adoption, particularly from enterprises. They have never wanted to buy like this. This is the moment to make your data LLM aka agent ready. And, we happen to be the perfect place to do exactly that. Here's what I am excited about for in 2025: - Seeing and experiencing new ways to consume and produce knowledge (LLM powered reader apps, notepads etc). - Cost of software making dropping to the ground, further barriers being removed. - More mental sparring with o1 like partners. I cannot live without o1 pro atm. - Seeing the first AI workers and workers' collectives getting contracted and paid for outcomes - More 10M+ ARR seed businesses with a couple humans and lots of compute at the wheel. - More hacking of agentic / LLM SEO and distribution. - More curation businesses - Better auth tooling - Better design - More companies hiring for design engineers and deployment strategists - New UI patterns and interfaces for humans and computers to commune Long AI workers (obvious) , short horizontal SaaS, super long Middle East (Abu Dhabi, Dubai and Riyadh), long London (non-obvious ), long SF (obvious). Short most glue code and infra, short new generation RAG and enterprise AI companies, long services and consulting led GTM, long Perplexity Shop, Short Perplexity finance, long BNPL and card networks , long prediction markets inspired products (private company research , investigative journalism, creator platforms, etc.). Thanks for to everyone for being here, next to us, with us, behind us, in front of us, paying us, investing in us, talking to us, supporting us, closing doors on us, saying no to us, saying yes to us, pushing us, setting the bar high, believing and sticking with us. Here's to 2025 and to winning. ??

This was one of the most insane years of my life . Made things happen that I never thought possible. Failed and got up more than ever thought capable of. Had the darkest and the brightest hours. At Ntropy, we built and launched and changed and shrunk and grew. We parted with team members through spring and summer. This was rough. We broke even for the first time in August. This was fun. We fixed our gross margins, improving from negative to over 70% positive Burn multiple dropped below 1 for the first time. Revenue grew over 3x We signed our first customers that are public companies We launched an end to end caching infrastructure allowing us to get the outputs of the largest models over 100x cheaper and faster The market we were building for, machine learning engineers and models consuming financial data was big, yet slow to adapt and very tricky for probabilistic inputs. The market we ended up in, financial services workflows and agents consuming unstructured financial data, is brand new and is going to be larger than anything we thought of before. We updated our priors multiple times and are continuing to do so. Just like our LLM caches that keep growing and improving. We have never seen such hunger for adoption, particularly from enterprises. They have never wanted to buy like this. This is the moment to make your data LLM aka agent ready. And, we happen to be the perfect place to do exactly that. Here's what I am excited about for in 2025: - Seeing and experiencing new ways to consume and produce knowledge (LLM powered reader apps, notepads etc). - Cost of software making dropping to the ground, further barriers being removed. - More mental sparring with o1 like partners. I cannot live without o1 pro atm. - Seeing the first AI workers and workers' collectives getting contracted and paid for outcomes - More 10M+ ARR seed businesses with a couple humans and lots of compute at the wheel. - More hacking of agentic / LLM SEO and distribution. - More curation businesses - Better auth tooling - Better design - More companies hiring for design engineers and deployment strategists - New UI patterns and interfaces for humans and computers to commune Long AI workers (obvious) , short horizontal SaaS, super long Middle East (Abu Dhabi, Dubai and Riyadh), long London (non-obvious ), long SF (obvious). Short most glue code and infra, short new generation RAG and enterprise AI companies, long services and consulting led GTM, long Perplexity Shop, Short Perplexity finance, long BNPL and card networks , long prediction markets inspired products (private company research , investigative journalism, creator platforms, etc.). Thanks for to everyone for being here, next to us, with us, behind us, in front of us, paying us, investing in us, talking to us, supporting us, closing doors on us, saying no to us, saying yes to us, pushing us, setting the bar high, believing and sticking with us. Here's to 2025 and to winning. ??

What a season in AI. 12 days of OpenAI launches, the prompt to code tools are improving and we have a new Llama! At Re:Invent we got to sit down with enterprise AI leaders. Here are my top conclusions, if you are an enterprise already working on LLM initiatives. - There are going to be two types of companies after this wave. Those who transformed and are winning and those who did not survive. The impact is much bigger than pure software transformation, so your thinking needs to change from incremental to survival mode. - Many leaders are still having to measure impact to get budgets and create "business cases" and logic that will justify the investment. This makes a lot of sense, however slows down transformation. The framework of thinking should move from the the "economist mind" to "game theory". What will happen if I do not do this today and everyone else does it and wipes me out? This mentality top down will create the right urgency to transform the enterprises from the data all the way up to use cases. - Your cost base and margins are super ripe to be disrupted by LLM-s. From the more obvious opex like customer support and legal, to more complex and bespoke line items. Here you need to apply the "dishwasher proof" technique. If you want to find out in the fastest way possible which items are dishwasher proof, throw everything in. You will find the stuff that is, and throw out the rest. If you want to find areas of labor that can be completely transformed with LLM-s, you need to apply them to everything. Most of those will stick and deliver serious ROI, those that do not, will remain outliers and will be justified by the lift you will get from elsewhere. - Everyone talks about data. Data lakes, data transformation, data labeling, fine-tuning with data, proprietary data. Your data is your first step to successful LLM deployment and investing in that foundation now is the least you can do to be future proof. Instead of justifying costs going into data quality and normalization, think about it as a commodity and a must-have to make sure you can do anything else on top. - There is no reason legacy systems of record need to exist. There is no reason your CRM data should live someplace else from your HR data and your support data and your legal data. They can all exist in one place and be easily queryable and accessible by agents and humans. - We will see the rise of the new ERP, that is not an ERP, but agentic workflows created directly on top of your data warehouse / storage. - Deployment as a service or implementation as a service is going to be the default go to market for most, replacing product led growth. Companies that hack this motion will come out winning - Finally, outcome based pricing, or take rates will be a game changer. You are not simply optimizing existing markets and making things cheaper. You are making the pie bigger and taking a cut. Happy Friday! going to play with some new tools this weekend