A token of AI progress —AnthropicAI breaks new ground and why it matters
Hi, I’m?Azeem Azhar . As a global expert on exponential technologies, I advise governments, some of the world’s largest firms, and investors on how to make sense of our exponential future. Every week, I share insider's perspective to help you make sense of the exponential age. This week, I look at the expansion of AI's context window: from goldfish to elephant.
?? Thanks to our sponsor?SynBioBeta, the Global Synthetic Biology Conference
Context windows for large-language models (LLMs) are growing rapidly. A context window is the length of text that an LLM can process and respond to. In a sense, it is like the “memory” of the system for a given analysis or conversation.
Anthropic released a new version of?Claude, their LLM, ballooning its context window from 9,000 tokens to an immense 100,000 tokens , or about 75,000 words. Larger context windows allow the systems to have much longer conversations, or to analyse much bigger, more complex documents. Claude’s new 100,000-token capacity means it can easily process the entirety of?The Great Gatsby, and answer queries about it in under 30 seconds.?
This opens up a range of incredibly useful applications. I have found the 8,000 token limit on GPT-4 a constraint. It prevents me from analysing documents that are longer than about 7,000 words (and most book chapters are longer than that). A whole slew of useful material like academic papers, legislation, business reports or smallish databases is within Claude’s reach. A long context window also opens up the possibility for greater personalisation of an LLM. A 100,000 token limit could store an individual’s preferences in significant granularity and leave space to engage in helpful conversation relevant to them. Finally, longer windows may reduce the frequency of certain hallucinations.
领英推荐
Is 100k even the limit we’ll see for context windows? It seems unlikely, judging by computer innovations of the past. Between 1981 and 2001, the average RAM of a computer increased by roughly five orders of magnitude. In the 20 years from 1997, wifi chip speeds jumped more than 1000-fold. Similar patterns would suggest immense context windows in the near future. This could mean processing or creating coherent outputs across very long research projects. Imagine.
?? Today’s edition is supported by?SynBioBeta
The cost of reading, writing and editing DNA is falling exponentially. Synthetic biology is being used to create new materials, foods, medicines, consumer goods and more. Join 1800+ industry leaders at?SynBioBeta, the Global Synthetic Biology Conference,? May 23-25 ?at the Oakland Marriott. Featuring genome pioneer Craig Venter, LanzaTech’s Jennifer Holmgren, xenotransplantation expert Martine Rothblatt and?Fantastic Fungi?host Paul Stamets.?
Exponential View?readers receive a 20% discount using code?EXV20?on checkout.
This comes close to something I was dreaming about since around 2014. Imagine all the newspapers, books, studies you've ever read. Then you work on a text and think: I once read something about this problem. Where was that? When was it? On which page? And now I just have to address my "second brain". I mark the passage I just wrote and ask my "second brain": Find matching input in all of my saved documents, in my 2000 books and 10000 papers and all of my personal notes. And instantly 5 or 10 books and articles are opened in tabs besides my work, right on the pages where something is written that could be important for me in my current context. Big vision. But the point is: Who is the super brain here? Is it the computer? Or is it me who commands a second brain like this? It seems obvious to me that human intelligence can grow exponentially the more AI can take your personal context into account. 100,000 tokens is not so important as progress for AI. What it does is it enables speeding up the progress of human intelligence. Don't fear AI, fear the strongly amplified HI ;)
Corporate Sustainability/ESG Consultant, Professor Associado na FDC - Funda??o Dom Cabral, Advisor Professor at FDC
1 年Sharing in Linkedin group "Shareholder Engagement on ESG" - linkedin.com/groups/3432928/
Founder of World's Fair Company
1 年Brilliant!
Innovation Coach for Redesign performance on Sales & Marketing Areas, Recently empowered with several ongoing A.I. Studies in top Universities and Agro B2C Comercialization proyects in Central America.Open for Contact
1 年Thanks for posting