#8 Positioning in an AI world
This week we have Llama 3.1, Chip Huyen 's post on building a GenAI platform, SearchGPT, and DeepMind getting Silver at the International Math Olympiad.
Llama 3.1 simply performs
Meta launches a GPT-4 level LLM for free (and 8B and 70B models too).
We knew this was coming but more than just releasing the weights, they detailed a lot of interesting decisions.
They chose to use a decoder-only architecture - steering away from Mixture of Experts - and focused on producing very clean data and data boundaries to prevent contamination in the training data. Showing simplicity is valuable.
They also allow for synthetic data generation and model distillation, something current frontier models didn't allow in their TOS. I could see an explosion in competition toward data companies like Scale AI or Nurdle AI .
They're furthering support for security and safety tooling like Llama Guard and Prompt Guard which is awesome.
Which leads to their announcement of Llama Stack - their open source ambitions of standardizing interfaces to LLMs. Something LlamaIndex and LangChain will definitely keep an eye on. React and PyTorch shows Meta is really good a building developer communities.
Groq is fast and fast is key
This demo really clicks why speed is key to unlocking a bunch of use cases.
Speak, get an answer instantly, tweak, iterate. Then, it becomes reducing the number of iteration loops. To do that, you'll want personalization and specialization. Very exciting times.
Blueprint for building your Generative AI stack
Chip Huyen 's blog hits again. A very good resource for how to design your system incrementally. And it reminds us that this sort of stuff takes a lot of engineering to get it reliably into production.
Somewhat tangental but look back at Llama 3. They don't detail (at least that I know of) the size and talent of the team.
Which leads to two avenues if you want to keep your team lean - banking on raw model performance improving and having less engineering required or relying on vendors and frameworks to provide high quality results.
领英推荐
OpenAI is coming for your search bar
OpenAI is taking on Perplexity with their own enhanced real-time search bar called SearchGPT. Partnerships with publishers is the first step but is this a trojan horse in a 20 sided game of prisoners dilemma? Did publishers already lose?
We'll probably see more of this type of commercialization as rumors are that they're set to lose $5B this year.
DeepMind gets silver in International Math Olympiad
I love how DeepMind quietly work on breakthroughs. OpenAI was born out of being the foil to DeepMind.
They use Lean, a formal solver, and searches for solutions until they get to one that works. One of the questions took minutes and others took three days.
Using formal solvers has been hinted at by many including Terry Tao.
It's very exciting to see this happening in real-time.
What's our takeaway?
In reinforcement learning, there's a concept of a reward function - letting your program know if what it did was good or bad. This lets it learn how to do things quicker or better.
Using Generative AI apps, we are the reward function. We tell it good or bad or tweak the result.
We all have personal preferences too. Can we model our personal preferences?
In a way, we have - in the form of social media algorithms.
We can also pre-populate and cache things we might want answers to if we know possible questions.
If you had infinite compute and energy, what could you do and who is best positioned to do it?