Artificial Neural Networks Should Work Like the One in Our Head

Artificial intelligence is deepening its influence in all aspects of life in the digital age: from our personal activities online and business applications in industry to accelerating research on grand challenges such as climate change and vaccine development.

This phenomenon has inspired many tech companies to build AI solutions that are intended to accelerate machine learning (ML) and other forms of advance compute. The ones that are likeliest to break free from the pack and win are focused on optimizing performance in three areas:

• Efficiency: solutions that only compute what is necessary and doing that on the fly (dynamically)
• Massive scalability: achieved through architecture that tightly integrates computation and networking
• General purpose: the flexibility to accommodate a wide variety of computational/ML tasks

To that end, one of the most promising new companies in this space is Tenstorrent. Founded in 2016, Tenstorrent solves each of those challenges with a full-stack solution comprising innovative hardware and software that, together, makes AI computing much more efficient and accessible for organizations large and small. Making deep learning more scalable and less resource intensive are critical to democratizing AI and unlocking its full potential.

To understand how Tenstorrent achieves this, let’s look at how the artificial neural networks that enable AI have evolved in industry versus in us as humans. Titans of tech like Alphabet, Amazon, Facebook and Microsoft have all built vast networks of data centers that occupy untold acres of land — and are rapidly sprawling as compute needs grow.

This is because, currently, there's more of a direct correlation between the size of ML models and the amount of hardware needed to supply the computational horsepower required. But not all of that processing capacity is necessary for most AI-related tasks. In human terms, it would be like having 100 percent of our brain activated every time we tie our shoes, eat a meal, or watch a movie. Beyond maddening, it would be biologically unsustainable.

Instead, our nervous system has evolved in a way that we typically use just a sliver of our brain, only engaging the most relevant areas for a given task. Tenstorrent has achieved this kind of efficiency with its AI solution, which carries out "fine-grained conditional computation."

The flexibility and scalability of this approach enables faster inference and training across a wide array of use cases — from data centers to edge devices. On the hardware side, this is achieved through Tenstorrent's unique processor architecture:

Tenstorrent debuted this expansion card, called Grayskull, at the Linley Spring Processor Conference in April. It’s the company's answer to the computer processing units (CPUs) and graphics processing units (GPUs) currently used for AI applications. But GPUs, as the name suggests, were built to rapidly process heavy mathematical computations for rendering graphics in products such as video games. They were not designed to develop AI algorithms.

And yet, the GPU remains the industry standard. These processors are used to train the vast majority of today's neural networks, even the largest ones like BERT and GPT-1, 2 and 3, which are used for natural language processing.

Computer scientists train these networks by feeding them massive amounts of data to tune their millions — and even billions — of parameters so they can accurately do things like recognize faces and images or generate coherent and convincing text. Some have been trained on all of Wikipedia, or in other cases, by setting up a web crawler and feeding the entire World Wide Web into the training process.

The resulting AI capabilities become the backbone for functions such as recommendation engines, content moderation and fraud detection, in sectors ranging from e-commerce and social media to finance and healthcare. The array of industries Tenstorrent can impact is indeed great.

The latest OpenAI model, GPT-3, is capable of amazing feats, such as writing news articles, viral tweets, even poetry. But the model also requires massive amounts of compute. Whereas its predecessor, GPT-2, could run on a few GPUs, training GPT-3 required thousands of GPUs.

And that comes with multiple costs. Financially, only the biggest companies can afford to train networks that immense, let alone pay for the amount of electricity required to run all that equipment.

And that's the other cost: to the environment. As AI usage grows, its energy consumption and carbon emissions are becoming increasingly clear. According to one study out of the University of Massachusetts, Amherst, the electricity needed to train a transformer — a type of deep-learning model — can cause more than 626,000 pounds of carbon dioxide to be emitted. That's nearly five times the emissions of an average American car.

A Computerworld article published last year presented the environmental impact in global terms:

“Recent predictions state that the energy consumption of data centres is set to account for 3.2 percent of the total worldwide carbon emissions by 2025 and they could consume no less than a fifth of global electricity. By 2040, storing digital data is set to create 14 percent of the world’s emissions, around the same proportion as the US does today.”

As Tenstorrent Founder and CEO Ljubisa Bajic puts it, getting a neural network to toddler-level intelligence would require a city full of machinery. So, by replacing GPUs with Tenstorrent's solution, we can meet the growing demand for AI and ML computation, without filling entire municipalities with computers.

Throughout his career, Ljubisa has driven innovation at some of the most dominant computer-chip makers in the world. Before Tenstorrent, he headed power and performance architecture at leading semiconductor company Advanced Micro Devices (AMD). Prior to that, he worked as a senior architect at Nvidia, which has been dubbed "the house that GPUs built."

Across the company, Tenstorrent's architects and engineers bring decades of experience developing products at Altera, AMD, ARM, ATI, Bio-Rad, IBM, Intel and Nvidia. They have helped build industry-defining products that are still in use today. So, there's no better team to tackle the $18.5 billion AI chip market. Gartner expects sales to reach nearly $44 billion in 2024.

At the conference last spring, Tenstorrent presented performance numbers showing how conditional execution enabled Grayskull to perform many times better than today's leading solution. Results from early pilots with potential customers have also born out Tenstorrent's promise to spark a paradigm shift in advance compute.

When we first partnered with the company, Ljubisa had two goals: One was to build a solution that outperforms existing products on the market, which they've demonstrated. The other, far more ambitious vision was to make neural networks run as efficiently as the human brain — and ultimately, create a more sustainable and achievable future for AI.

Tenstorrent is closer than ever to achieving its loftier goal and now wants to recruit large, influential institutions like top-tier research universities and federal agencies as allies in this mission to course correct AI's unsustainable trajectory. One widely respected industry watcher who attended the Linley event — Karl Freund, senior analyst at Moor Insights & Strategy — later shared his thoughts about Tenstorrent in Forbes, clearly bolstering the company's case:

"To my eye, this announcement marks a shift from chips with lots of fast cores and on-die memory & fabric (which describes most of the entrants to date) to a new approach of smart computing where the software, training, and inference chips all coordinate knowledge of the network to reduce the amount of computation. Brute force is great, until something better comes along, and I think it just did."

__________________________________________________________________________

Follow Eclipse Ventures on LinkedIn. We publish new articles frequently!