GenAI Weekly — Edition 4

Your Weekly Dose of Gen AI: News, Trends, and Breakthroughs

Stay at the forefront of the Gen AI revolution with Gen AI Weekly! Each week, we curate the most noteworthy news, insights, and breakthroughs in the field, equipping you with the knowledge you need to stay ahead of the curve.

? Click subscribe to be notified of future editions

Cognition introduces Devin, an AI Software Engineer

From Cognition’s blog:

Cognition

Meet Devin, the world’s first fully autonomous AI software engineer. Devin is a tireless, skilled teammate, equally ready to build alongside you or independently complete tasks for you to review. With Devin, engineers can focus on more interesting problems and engineering teams can strive for more ambitious goals.

I don’t know if this is the future of software development, but I know it’s not going to be very different from this either. Developers of the future could simply be doing “spec management”— a specialization of prompt engineering.

What I learned from looking at the 900 most popular open-source AI tools

From Chip Huyeh’s blog:

Chip Huyen

I searched GitHub using the keywords gpt, llm, and generative ai. If AI feels so overwhelming right now, it’s because it is. There are 118K results for gpt alone.

Chip’s blog is always a pleasure to read. This analysis is no different.

ASCII art elicits harmful responses from 5 major AI chatbots

Dan Goodin writing for ArsTechnica:

Ars Technica

Researchers have discovered a new way to hack AI assistants that uses a surprisingly old-school method: ASCII art. It turns out that chat-based large language models such as GPT-4 get so distracted trying to process these representations that they forget to enforce rules blocking harmful responses, such as those providing instructions for building bombs.

We did cover this earlier, but this article simplifies this important subject and is worth a read.

Building Meta’s GenAI Infrastructure

Kevin Lee, Adi Gangidi, Mathew Oldham writing on the Facebook Engineering blog:

Marking a major investment in Meta’s AI future, we are announcing two 24k GPU clusters. We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. We use this cluster design for Llama 3 training.

I’ve always been impressed with how open Meta has been with their Open Compute Project. This is a good step forward.

OpenAI announces Transformer Debugger

From the project’s Github page:

TDB enables rapid exploration before needing to write code, with the ability to intervene in the forward pass and see how it affects a particular behavior. It can be used to answer questions like, "Why does the model output token A instead of token B for this prompt?" or "Why does attention head H attend to token T for this prompt?" It does so by identifying specific components (neurons, attention heads, autoencoder latents) that contribute to the behavior, showing automatically generated explanations of what causes those components to activate most strongly, and tracing connections between components to help discover circuits.

AI is accused of being a black box. Tools to help peer into that are always helpful.

Spreadsheets are all you need.ai: A low-code way to learn AI

spreadsheets-are-all-you-need.ai:

Spreadsheets-are-all-you-need implements the forward pass of?GPT2?(an ancestor of ChatGPT that was state of the art only a few years ago) entirely in Excel using standard spreadsheet functions.

What a way to learn about Transformers. And how beautiful and powerful spreadsheets are!

Using LLMs to Generate Fuzz Generators

Toby Murray writing on the Verse Systems blog:

Toby Murray

Specifically, we could consider performing the following steps:

While we already have systems that assist developers in writing code and understanding code, the holy grail is letting LLMs write whole programs for you. But, on the way to that goal is using them to test code in all kinds of manners. Most vulnerabilities are linked to buffer overflows and parsing bugs (and our devices parse all the time: email, HTML, CSS, JS, images, video, PDFs, etc.). Fuzzing is an important way to ensure we have robust parsers. Any help we can get from LLMs there is superb value add.

Yi: Open Foundation Models by 01.AI

Alex Young et al on Arxiv:

We introduce the Yi model family, a series of language and multimodal models that demonstrate strong multi-dimensional capabilities. The Yi model family is based on 6B and 34B pretrained language models, then we extend them to chat models, 200K long context models, depth-upscaled models, and vision-language models. Our base models achieve strong performance on a wide range of benchmarks like MMLU, and our finetuned chat models deliver strong human preference rate on major evaluation platforms like AlpacaEval and Chatbot Arena. Building upon our scalable super-computing infrastructure and the classical transformer architecture, we attribute the performance of Yi models primarily to its data quality resulting from our data-engineering efforts. For pretraining, we construct 3.1 trillion tokens of English and Chinese corpora using a cascaded data deduplication and quality filtering pipeline. For finetuning, we polish a small scale (less than 10K) instruction dataset over multiple iterations such that every single instance has been verified directly by our machine learning engineers. For vision-language, we combine the chat language model with a vision transformer encoder and train the model to align visual representations to the semantic space of the language model. We further extend the context length to 200K through lightweight continual pretraining and demonstrate strong needle-in-a-haystack retrieval performance. We show that extending the depth of the pretrained checkpoint through continual pretraining further improves performance. We believe that given our current results, continuing to scale up model parameters using thoroughly optimized data will lead to even stronger frontier models.

01.ai is headed by Kai-fu Lee and is a startup to watch. Also, their work is open source, which is great.

For the extra curious:

• Circles And Slashes
• Stealing Part of a Production Language Model
• Defining open-source AI
• How Photos Were Transmitted by Wire in the 1930s
• Why in some watches and clocks 4 is IIII and not IV
• Apple’s new AI paper
• Microsoft’s paper on AI-powered software development