New Computer Architecture with a Capital A

Too busy to read? ?? Listen to this episode now on my Humans of AI channel.

I spent the week with nerds. It felt like home.

30 Years ago, I took my first Architecture Studio at MIT.? In those days we built models with our hands and in a wood shop. My first model was a small angular space filled with columns so close that a miniature human could barely navigate the space.? My critic (this is what we called professors) didn’t like it very much.? But looking back, it perfectly characterizes how I’ve attempted to understand the world - through the balance of structure and space.

In 1994, when I was a student, the School of Architecture and Planning at MIT was located in Building 7, one of the original classically designed buildings by William Welles Bosworth.? I would enter Building 7 off Mass Ave to get to my studio. After class, I'd walk along the infinite corridor?to get to Kendall Square, passing through building 11, 10, 12, and 8.? As I would catch glimpses outside of the “quad” it would be barren. This snaking 825 foot long hallway through history was the cultural and social hub of the campus.

You would have thought the critic would have understood that my project wasn’t about the columns, it was about the latent spaces that the humans would fill in. ?

Architecture is all about the balance of structure and space.? Too much structure, you kill space.? Too much space, you kill culture. ?

Culture is cultivated by obstacles.

In my current research on the balance of rules & chaos, it turns out that human creativity thrives on aligning the right structure with the right mindset to solve the right problems.

Which brings me back to computers and architecture.

The Imagination in Action Conference was hosted at MIT’s Media Lab, an interdisciplinary research institute combining technology, multimedia, sciences, art and design.? It was founded in 1985 and is actually a part of the School of Architecture and Planning.? But in 1994, I had no idea it existed.? If I had, I would certainly have changed majors. At 50, I’m contemplating returning for a PhD in creativity science.? I think that would be a fun way to spend a decade.

I spent the first hour of the conference listening to talks by current PhD students.? I was on my first cup of coffee and the headliners–Stephen Wolfram, Yann LeCun and Lex Fridman weren’t on until the afternoon, so I decided to start by hearing the research that’s being done by some of today’s most brilliant youngish minds.? I’ve shared some of these through a series of human-AI portraits: Hope Schroeder , Nikhil Singh , and Kushagra Tiwary were three standouts.?

Everyone was talking about the need for new architectures for generative AI. ? For the majority of the population, we are barely a Fortnight (thank you Taylor) into the current architecture of LLMs and it’s already quite obvious that despite the unbelievable work these predictive language models can do, this is not a structure that is likely to be sustainable (these models are gas guzzlers) nor adaptable (language models don’t do math, period, end of sentence) nor actionable (they talk, but they don’t do).? So while the big tech companies are playing oneupsmanship games, the academics and the little tech guys wearing big brimmed cowboy hats are thinking about architecture. And so is this small town girl.

Now, I’m a turtle on most things, so I surround myself by hares.? My rabbit friend Mark Foggin saw Hamilton at the Public Theater and told me I had to see it.? I got opening week tickets on Broadway on my birthday.? Thank you, Mark!? So when my new speedster friend Eric Fraser said I need to try out Groq (not Grok by the brilliantly disturbed Elon Musk), but Groq, the on-device open-source LLM interface, I knew this was an AI Hamilton moment.

The IIA conference was one of the most poorly organized conferences I have ever attended, which is how I knew it would be awesome. Men have this way of smoke signaling their way through networking that both amazes and befuddles me. Beauty is always discovered in the chaos.? There was no conference app or organized way other than a single page PDF file that you could barely read with the photos of people sorted like a taxonomy of species.? I could tell from the image that the tigers, lions and bears were arriving at 15:00 hours, so like a dog in heat I found my way through the crowd (structure) to the second row seat (space) to take in the spectacle.? I peed on my territory with a large fake Prada bag that I picked up on Canal Street, while my physical body removed itself periodically to get water, go to the women’s room, and charge my phone (thanks, Chris McKay ).? I had my MacBook Pro and a copy of my book, TEMPERATURE , in the bag. I figured if someone stole the bag, at least they’d read my book.

The hype man of the event, John Werner (ladies think of him like the Erin Gallagher of AI who wears a tie….yeah, no, that’s a terrible analogy….scrap that).? Anyhow, John was hosting this event like Jimmy Kimmel at the Oscars bringing on stage one superstar after another with some off-color jokes interspersed with his genuine Tenderheart Care Bear lovableness. On the later, in true rock concert spirit as the headliners were coming on stage he saved room on the floor for all the students to filter in and have a front row seat. Seeing two pre-teen girls sitting at the foot of the stage listening to Stephen Wolfram talk about math and the power of narrative broke this feminist’s cold hard heart.

Computing, yes, computing.? I promise we’re getting there.

So now we come to our opening act, which IMHO is always the most interesting part of the show, right?? I mean picking the opening act is a big signal.? It has to be someone good enough that the headliner admires their work, but not big enough to overshadow their glory.? An up and comer, an indy star, a reddit sensation, a TikTok (RIP) phenom, a nobody until they’re everybody’s hero. ?

Wait for it, wait for it.

Yeah, you guessed it: Groq . But not the CEO of Groq, the C-T-O of Groq, Dinesh Maheshwari , who walks onto the stage like the Marlborough Man and damn, he stole the fucking show.

Now, to put this in context, let me just remind you of where we’re at in time (because many of you will read this asynchronously).? Just a mere 30 days ago the Bengal tiger of computing, 英伟达 held their annual conference announcing their biggest computing chip Blackwell at the tune of $10B dollars a piece.?This shift towards massive computational resources is expected to transform industries from healthcare with its complex diagnostic algorithms to autonomous vehicles that require real-time processing at unprecedented scales. We are currently living in a world where bigger = better. More compute is more power, more data, and more capability.

It’s also more electricity.? A hell of a lot more.

GPUs, for those unfamiliar with the acronym, these are Graphics Processing Units.? They are big ass computers with multiple cores designed to process vast amounts of data - they were designed to run video games and they are now running the biggest LLMs on the planet. ?

But the big question has been, do we really need the same amount of compute to ask who was the President of the United States in 1994 as we do to understand the complexities of global climate change?

Probably not.? As I said before creativity is all about alignment of the right structure with the right mindset to solve the right problems.? In computing, that means having the right architecture and the right model to solve the right problems.? It makes no sense to run a language query with a rocket engine. That’s the problem that Groq is out to solve.

And they are doing it in the most elegant way possible with an architecture so simple it can be explained to a five year old.

Groq is like the Henry Ford of AI.? Rather than assembling the entire car at once, it’s done on a line - processing piece by piece.? The outcome is what we call “low latency,” or what can more technically be called wickedly fast.? And because it’s so fast, it doesn’t need to happen in a big server farm.? It can happen on-device.

Let me say this a different way. NVIDIA is Taylor Swift.? Groq is Lana Del Rey (who she brought up on stage with her to celebrate her Grammy win…and if you listen to Fortnight, it’s all LDR’s sound in Taylor’s wrapper.)

So why does this matter to you?? Because we are moving now into the world of craft beers of AI.? If you fancy yourself someone whose a purveyor of taste, who doesn’t just follow the masses, who can spot a needle in a haystack - you are someone whose going to want to pay attention to Groq and the future of LPUs.

As an architect, I find this fascinating.? From a user perspective, ChatGPT is still the all-utility player whose performance is unmatched with regard to its multimodal, mixture of experts approach to natural language prompted computing.? This is like the multipurpose room that in the 1990s was all the rage because it was a big wide open space where people could do anything.? But remember when I said that too much space kills culture?? Lack of specificity in space, a misfit of the architecture to the task at hand cannot only be ecologically wasteful, it can mean that the kinds of stuff that happens there gets watered down and boring.? Banal. ?

This is why I find Groq worthy of our attention: the right architecture + the right model to solve the right problem = efficiency....and efficiency = sustainability.

P.S. Like Taylor, within 12 hours they released a bonus side: you can try out Groq on your phone, where you can run Meta’s latest open source model (Llama3-70b-4096). Remember building 7….engineers like numbers. Shrug. It will still hallucinate, but that has more to do with Llama3 than it does with Groq.? If you want to read more about LLM Hallucinations, I recommend Gary Marcus 's Humans versus Machines: The Hallucination Edition .

I'm Lori Mazor. I teach AI with a Human Touch.? I'm reinventing how we educate, strategize, and build the future one article at a time. If you enjoy this newsletter,

• You will love the book, TEMPERATURE: Creativity in the Age of AI, debuted #1 New Release in Human-Computer Interaction and now available on Amazon.
• You can join one of our executive bootcamps with our May cohorts. The next Level I cohort kicks off on Monday, April 29 (5-7PM EST) and Level II on Wednesday, May 8 (5-7PM EST). This is the last week to register.
• Bring our 10-Hour Corporate Training Program to your Company , empowering your employees to identify their own problems, strategize, and build their own solutions as you organically incubate and grow enterprise level applications.
• I'm going on tour! Our team is planning our five city book tour to DC, Boston, LA, Chicago, Miami and San Francisco and we'd love to visit you for a reading, workshop, or creative jam session. Media bookings and inquiries to: [email protected] .