Chip 5: What is the way out after Moore's Law fails?

In yesterday's content, we talked about several major challenges in the current computing field. These challenges may lead to chips becoming more expensive and computing performance improving slowly after 2012.

But now, there have been some breakthroughs in these difficulties, namely the emergence of brain-like chips.

Today, I will introduce to you what brain-like chips are all about.

The Dual Dilemma of Hardware and Software To clarify brain-like chips, we need to review the difficulties in the computing field.

In terms of hardware, because the volume of individual transistors is too small, quantum effects become more and more serious.

Electrons no longer flow according to the rules we hope for, so in order to further constrain quantum effects, the structure of a single transistor has evolved from planar to three-dimensional, and from three-dimensional to complex three-dimensional.

Combining hundreds of billions of such complex structure transistors together makes the manufacturing process much more complicated than before. Today's 5 nm process chips have 3-4 times more manufacturing steps than a decade ago.

The constraints on hardware directly lead to the cost of individual transistors decreasing by half every 1-2 years in the past few decades, to a slight increase each year. Therefore, future top-tier chips will be more expensive than the previous generation.

Don't just look at each generation being only 20% more expensive, but at this rate, the top flagship CPU in 2042 will no longer be 5,000 yuan, but 190,000 yuan. And people's demand for computing will continue, so how can the cost be resolved 50 or 100 years later?

The above are the challenges encountered in hardware, and the challenges in software are actually not small.

Although about 10 years ago, new algorithms such as deep learning and neural networks appeared, significantly improving the performance of machines in image recognition, semantic understanding, automatic translation, and chess games. But these improvements are uneven.

For example, machines now far exceed humans in chess games, but in areas like autonomous driving and semantic understanding, they can only be said to be close to humans, far from human experts in these areas.

How can computers outperform human experts in these weak links?

It seems that there isn't much room for breakthroughs in improving algorithms, as there was a major breakthrough on this path not long ago. Therefore, today, both technology experts and capital giants are working on another path, which is to stack devices and computing power.

With money, of course, this can continue, but the problem encountered in software is: to improve performance by 10 times, the cost is not just 10 times more, but possibly 1 billion times more.

The cost of achieving current machine performance is already in the hundreds of millions of dollars. If it increases by 1 billion times, it would be completely beyond people's capacity.

Moreover, a harder limit comes from energy consumption. In fact, using current algorithms to improve software performance by 10 times, that energy consumption might be the sum of the world's energy consumption over several decades. Not to mention whether there is so much energy, even if there is, the accompanying impurity emissions and carbon emissions during consumption would be enough to completely collapse the living environment.

The Way Out: Stepping Beyond the Von Neumann Architecture When both hardware and software are stuck, where is the way out?

This requires us to step out of the current chip structure, that is, the von Neumann structure that has continued from the late 1940s to today.

Regarding this structure, if you take a computer course or look it up in an encyclopedia, the result is this: first, it will tell you that a computer is divided into a processor, controller, memory, input and output devices; then, qualitatively introduce what these modules do.

But if I say the same thing, it will be hard for you to understand the benefits of stepping out of such a structure. So, I want to re-describe the von Neumann structure according to the characteristics of brain-like chips.

The von Neumann structure is based on the proof of the genius mathematician Turing and was finally realized by von Neumann in reality.

At that time, Turing invented an operation that could simulate all logical calculations. This operation was completed by a machine consisting of a controller, a read/write head, and an infinitely long tape.

The tape is used to store information; the read/write head can read the information on the tape and also write the calculation results on the tape; the controller is used to move the tape left and right or erase the data on the current read/write head. Thus, as long as the actions of pulling the tape left and right and reading the tape information are coordinated reasonably enough, no human intervention is needed in the middle process, and the numerical result of the logical operation can be obtained after a finite number of operations.

Afterward, the work of some computer scientists was to think about how to arrange the actions of pulling the tape and saving intermediate results to complete addition calculations. Even, how to arrange the sequence of actions to complete the calculation of the hyperbolic cotangent function? These computer scientists gradually evolved into today's algorithm engineers and programmers.

Another group of computer scientists worked on how to shrink the small unit of tape plus read/write head smaller and smaller, so that 20 billion such devices could work at the same time, wouldn't that be much more efficient? These computer scientists gradually evolved into today's chip design and manufacturing scientists and engineers.

The computing tasks that humans encountered 70 years ago could be completed with pen and paper, perhaps with some calculators. So when the von Neumann structure appeared, with the enrichment of algorithms and the miniaturization of computing modules, the first 60 years of the 70 years maintained stable and rapid development.

But in the last 10 years, the speed has slowed down because of the bottlenecks I mentioned at the beginning in both software and hardware.

The problem in software can be called a bottleneck, because people might still find some ways to increase computing performance by thousands or tens of thousands of times.

But the problem in hardware can no longer be called a bottleneck, but rather hitting a ceiling—chip technology cannot shrink indefinitely, and no one knows how to do below 1 nm, and now we have reached 3 nm.

Among many solutions, brain-like chips are the most promising. This idea appeared as early as 1990, with breakthroughs not in hardware or software, but in a fundamental restructuring of the computing structure. That is, no longer using the von Neumann structure, so the hardware and software ceilings inevitably encountered by the von Neumann structure no longer exist.

The inspiration for brain-like chips comes from the human brain. We know that the human brain must be performing a large amount of computation, and the complexity of this computation far exceeds that of man-made computing devices, and its performance in many aspects also significantly surpasses that of man-made computing devices.

So, if we can borrow some new structures from the human brain to man-made computing devices, perhaps there can be breakthroughs.

Research in this area is divided into two schools:

One school believes that to create a new computing structure, one must first understand the working principles of the human brain. This school is still stuck in the mire to this day. Because after more than a hundred years of research, humans still have no clear understanding of the basic principles of the human brain.

But this school firmly believes that this path is correct, because the von Neumann structure was originally developed in a top-down manner—first, Turing mathematically proved the correctness of the operation, and then von Neumann found a way to implement it. After that, the computer industry exploded like a nuclear bomb for 70 years.

The other school believes that it is not necessary to thoroughly understand the principles. First, use the existing biological knowledge to create some structures based on the basic units of the human brain and see what can be done with them.

This school also has precedents. For example, more than 20 years after the Wright brothers first flew, von Kármán's aerodynamics theory fully explained the principle of why airplanes could fly.

Similarly, the use of radio during World Wars I and II, which was actually dependent on the ionosphere at the edge of the Earth's atmosphere reflecting electromagnetic waves multiple times, allowing people thousands of kilometers away to receive information. This principle was only clearly explained more than 70 years after the invention of radio. But before that, people did not refuse to use it because they did not thoroughly understand the principle; they used it first and explained it later.

So, the second school, according to the basic structure of neurons, gave some mathematical descriptions to brain-like chips, built models, and produced several operational chips. As a result, today the second school is temporarily victorious over the first.

The relevant mathematical descriptions can be found in some books related to AI chips, so I won't go into detail. The general description is as follows:

A basic unit of a chip is similar to a neuron in the human brain; it is a small sphere. Several long lines connect it to distant neurons.

In brain science, these long connections are called "axons." Additionally, the surface of the small sphere has some sparse hairs. They may not all be useful, but when certain conditions are met, a few of these hairs will shake hands with connections extended by other small spheres. These hairs are called "dendrites" in brain science.

The handshake is not the concept of wired connection in traditional circuits, but similar to the concept of "synapses" in brain science. That is, there is a gap of a few micrometers between the extended axon and the local dendrite.

In the human brain, the transmission of signals within a single neuron is through the form of discharge; but the transmission of signals across multiple neurons relies on chemical substances secreted at synapses, i.e., neurotransmitters.

Only when the concentration of neurotransmitters is high enough will it cause the neuron at the other end of the handshake to be highly activated and start discharging; when the concentration of neurotransmitters is not high enough, it does not activate the subsequent neuron.

The use of neurotransmitters, instead of direct electrical connections, is because electrical signals lose too much in transmission through biological tissue. If the distance exceeds a few tens of centimeters, the electrical signal almost disappears. This is a feature that emerged in biological evolution.

In the mathematical description of brain-like chips, "synapses" are replaced with the concept of "weight" in mathematics, "whether a neuron can be activated" is replaced with the concept of "threshold" in mathematics, and neurons are described by mathematical activation functions.

The Difference Between Brain-like Chips and Traditional Structures

That's enough about the principles. It might be a bit brain-burning, but it's okay if you don't understand; we just need to know that it is built by mimicking the human brain.

Our question is, what is different between these new models and the traditional von Neumann structure?

First is the integrated memory and computation structure.

What is this? Here, I'll just briefly describe it as another structure parallel to the von Neumann structure. Its reading, storage, and computation are all completed in the same unit, which is the most basic structural change. As for the specifics, we'll talk about them separately in tomorrow's content.

Second, the connections between units have changed.

Whether an input will cause multiple units to act is similar to whether a neuron will be activated. When the concentration of neurotransmitters is not strong enough, that neuron is not active, or in other words, does not reach the threshold. This characteristic is called "event-driven."

Today, such chips are already used in dynamic visual analysis. For example, in the process of autonomous driving, some judgments do not need data from the entire panoramic image; knowing which objects are moving at the moment is enough.

Traditional chips analyze the entire video frame, while in brain-like chips, only those active pixels activate the processing. By analyzing only this active content, the amount of computation is naturally greatly reduced, thereby greatly reducing chip power consumption. This feature is a bit like the visual processing of frogs, which are also more sensitive to moving objects.

How much will power consumption be reduced?

We can refer to the match between Lee Sedol and Alpha Go back then. That year, their competition was neck and neck, but Lee Sedol's brain consumed about 20 watts, while Alpha Go was about 1 million watts.

Although today Alpha Go achieves the same performance with only a few hundred watts, seemingly close to the human brain's 20 watts. But in fact, the human brain's 20 watts of energy consumption is not just for processing Go strategy, but also for processing vision, hearing, balance, maintaining emotional stability, and many more tasks much more complex than Go.

Therefore, the human brain is still much more efficient than traditional computers. Brain-like chips could potentially significantly reduce power consumption and computation in many similar calculations.

Brain-like chips represent a new structure that, once popularized, will create another "computer era."

That's all about brain-like chips for today. Tomorrow, I'll talk in detail about one of its important innovations—integrated memory and computation.

See you tomorrow.