Can machines think?

Chapter 1

Can machines think?

It is the beginning of the 17th century, Netherlands. A young boy receives a present from his father. It is a toy in the form of a soldier, about one feet tall, and has all the detail of the uniform including an accordingly sized sword. The toy soldier can stand on its own but is made of soft materials, making its arms and legs bendable using a little force exerted by the boy’s fingers. For this, he holds his fingers around the shoulder and upper arm of the soldier. This allows him to swing the soldier’s arms as in a forceful march. He has also another grip where he holds the legs and is able to bend them just enough to make the soldier walk upstairs.

It is the only toy of such refinement that the boy owns. It is only the dawn of the 17th century. Industrial revolution has not yet begun. Toys, like everything else, are exclusively made by manual labor. The soldier becomes a significant part of his life. Especially at bedtime, just before he falls asleep, he would close his eyes and his imaginations would let him wander about wild ideas: ‘What if his soldier could walk on its own?’ He imagined placing the soldier on a table and whispering a command. He also imagined his sister standing by as a witness. Oh, would that make him proud! He then comes up with an even wilder idea. “What if the toy comes to life every time he is asleep?” The soldier could walk around the room and explore the chair, the table, his clothes. The boy gets so taken by his imagination that he starts worrying about consequences. The most worrisome idea was if the toy walks out of the room and gets lost or stolen.

The creative imagination of this boy, let us call him René, is nothing unusual. Kids born long before and long after him came up with similar ideas. Nowadays, this imagination is spurred by toys we call robots—those that have both human-like and machine-like features. Back in the 17th century, they did not have as elaborate machines as we do today. Nevertheless, they had elementary ones. The basic concept was there. This was enough to drive imagination. In fact, a famous philosopher named René Descartes, may well have been the boy who lived in the same times as our boy. Descartes had a theory that all the living systems worked like machines. He also had the then unorthodox idea that the center of our passions is the brain, not the heart, as had been widely held. Although Descartes is widely known for his work on the mind, he was also a physiologist building his own dissection lab—which was at those times, a popular source of biological knowledge; if you wanted to know anatomy, you had to build and run your own anatomy lab. It was these observations that made him think that our bodies worked like machines—he thought movement of muscles traveled through nerves. To prove his point, there is also evidence that Descartes built a small robot that could do back-flips. The robot was, it seems, suspended on a tight rope and was able to do impressive actions—much like those that our child-René dreamed of just before sleep.

But Descartes was not the first one to build a robot. Some half century before him, there was a fellow in Italy who also, like Descartes, had many talents. Leonardo Da Vinci is best known for his art, but his work on robotics is less known, even though he built multiple ones. Da Vinci was a skilled craft-man building all kinds of machinery. Luckily, being a visual artist too, he left us with accurate drawings of his inventions, allowing us to re-build many of them in modern times. He gave a shot even on creating flying machines resembling a helicopter and an airplane. His design for a parachute was even tested from 3000 meter jump. And it worked! A human-like robot that he created was also a work of a genius. The outer skeleton of the body was conveniently borrowed from a medieval knight armour. The inside of the armour gave Da Vinci plenty of space to design a system of pulleys and ropes that could move arms and legs in a human-like way. René would have loved it. But René likely never saw the works of Da Vinci. Italy was far away in those times. Unlike then, today everyone can see Da Vinci’s robot. It has been rebuilt and is on display in a museum.

Europe is not the only place where seeds of robotics have been sown. In 18th century Japan a human-like machine was created that was able to served tea—or, at least, much like the European equivalents, illustrated the possibility that something like a tea-serving machine may one day be possible. We can be almost certain that many other efforts in building robots existed, many likely lost due to the necessarily selective nature of the discipline called history. Only a fraction of what has happened remains on records.

Since those individual germs of robotics have seen the light of day, notable progress has been made. The robots of today would, no doubt, captivate all the robotic pioneers such as Descartes, Da Vinci and the Japanese masters. Unfortunately, however, these pioneers may also be somewhat disappointed. They would probably quickly note that modern robots made of metal, plastic and electric motors share a critical flaw with their original much less elaborate inventions made of wood, ropes and pulleys. The problem of modern robots is still that their flexibility in motion is not nearly being matched by a flexibility of the mind controlling that motion. In other words, when we apply the criteria by which we judge the intelligence of humans to robots, or even animals, the robots of today remain utterly dumb.

And it is not that we haven’t realized the problem. We are fully aware of the issue, and we are working on it. To address the matter, the scientific community has even created an entire research discipline called Artificial Intelligence (AI). Much like robotics, AI is a young discipline waiting for prime time. The hopes are high that what we are observing today is a dawn of a true AI bonanza, and not a hype that will end up in disappointments, as has already happened in the past. The good news is that today, one no longer needs be a maverick to work on robots or on their minds. Education on AI has been institutionalized. Governments pour money into research and development. Businesses compete on how to best put this technology to use.

It has been estimated that today over twenty thousand individuals hold a PhD that qualifies them as working in AI. Importantly, all of these people needed motivation and inspiration to get their PhD and also to continue working post PhD. I want to describe an event from my own childhood that seemingly greatly affected my adult life. I have a vivid memory of watching a science fiction movie at the age of six, or so. Although I cannot recall the movie plot, I clearly remember the scene of watching the movie on our family’s black-and-white TV— and yes, I am old enough for that. There was a male character who had a machine at home that could talk to him and give him advice. The machine was much more articulate and wittier than any AI assistant we have today in a form of Cortana, Siri or Alexa. The machine understood what its owner was going through and was able to give him a good advice. The two appeared to be friends. My father was also watching, so I asked him: “What is that machine?” This is when I heard for the first time this magic word that would inspire much of my child life: “It is a computer.” A computer?! A machine that can think, talk to you and be your friend is called a computer!? How awesome! A computer became in my imagination the most important object of desire. Suddenly, all other toys and games paled in comparison to what one could possibly do with a computer. A lot of fantasy occurred at bedtime about what I could do with a computer.

It took many years to realize that I was misinformed. Only when I grew older did I get my fingers on an actual computer and learned how to instruct a computer to do things by applying if-then statements and for-loops. A big gap became apparent—on one side there was what computers were capable of, and on the other side, there was a child’s fantasy created by a Sci-Fi movie. I believe it is exactly this discrepancy, a disappointment one may say, that has driven my educational and professional life afterwards, leading me into studying AI technology, investigating the functions of human brain and the principles of human cognition. This exaggerated expectation that lasted for years may be the main reason this very book is being written. Perhaps, one can understand the present book as an attempt to repair a child’s broken dream.

But what motivates one to fathom a robot or AI in the first place? Ideas of machines with human features seem prevalent across cultures. Why do these ideas pop up in children minds? And why do they have the power to leave strong traces as to preoccupy the person’s intellect well into adulthood? I don’t know, but there may be something about powers that we hope robots would give us. I think we humans will always be inspired to build robots. There is a strong inherent desire. A need for such machines is a part of being human.

Apart from our motivation to create intelligent machines, a question arises of whether it is in principle possible to create machines of human-like intelligence. Do we even have a chance of building an AI that matches us? In the present book I propose the next technological steps we need to make to get us closer to this dream. A lot in this book will have to do with the resources required to reach that goal and finding tricks to reduce the need for those resources. When one has a goal of epic proportions—like it is to build a human-level intelligent machine—one not only needs ask how a small-scale proof of principle can be created, but also whether this proof can be applied at scale. Is there enough material, energy, labor? How many pulleys does Da Vinci need to make a robot that climbs stairs? Will there be enough space within the armour? And how many pulleys for a robot that can take care of a household? Would there be enough wood on the planet Earth?

To illustrate the importance of assessing the demands for scaling, let us consider another endeavor, intellectually less challenging, and yet of epic proportions. Suppose the entire mankind is being ruled by a crazy king Uraganus and the king orders moving the entire mountain range of Alps down to the deserts of Saudi Arabia. He insists on flattening out southern Europe, no matter the cost, and rebuilding the Alps down in the desert.

A small-scale proof of concept would require loading a truck with stones from Alps, driving down to the desert and dumping the load. The question of scaling is: would completing of such a task be even possible with all the technological resources the mankind has on its disposal today? To answer, let us do some calculations. The Alps weigh some 4.8 x 1013 tones. This is 48 followed by 12 zeros, like this: 480,000,000,000,000 tones. If we engage all the trucks that exist on our planet, can we do the job? Today, there are some 240 million operational trucks on the planet Earth. If, on average, each truck can load 10 tons of cargo, it follows that in just one round trip from the heart of Alps to desert of Saudi Arabia all the trucks in the world can transport 2.4 billion tons of material—a nine-zero number. Now it is easy to get the number of round trips needed to move the Alps by dividing the two numbers. This gives us a seven zero number, or more accurately, 2.4 x 107 round trips. Note that dividing such big numbers is as easy as first-grade math; to approximate, all one needs do is subtract the zeros: thirteen zeros minus nine zeros gives us seven zeros.

So, can mankind handle a seven-zero count of round trips? Let’s compute the number of years this would take. The shortest car route from the center of Alps to Saudi Arabia is 5,000 kilometer long, which translates to around 100 hours for a round trip under maximally optimal conditions. This gives us 109 hours or 260,000 years. Mankind as a whole likely cannot ever achieve a task of that scale. To put this in perspective, the oldest traces of human civilization that we know of are only some 6,000 years old. This is less than three percent of the prospect in the future that we are looking at. Even the entire period that the species of homo sapiens has been inhabiting the planet Earth is shorter, some 200,000 years. Thus, using today’s technology, the entire planet does not have enough resources to complete the Alps moving project now, or ever.

When it comes to planning ambitious AI projects, Ray Kurzweil was one of the rare people who ran similar numbers. Kurzweil, among other things, invented a synthesizer of musical sounds. He is also a futurist and has assessed when computers will have enough power to run an equivalent of a human brain. He founded his assessments on Moore’s law, which states that the computational power available to mankind roughly doubles every two years. This law basically asserts that the power of our computers grows exponentially; every two years we add as much to our number crunching capabilities as we have accumulated throughout the entire preceding history of computation technology. Kurzweil calculated that it will be at around the year 2045 when our computers will have enough power to run as many calculations as our brains.

What Kurzweil did not offer is the AI technology needed to make these super-powerful machines intelligent. His assessment was merely about having sufficient capabilities to execute raw number crunching—in supercomputer jargon this is often referred to as bare metal. Will we in 2045 only have bare metal or will we also find ways to take advantage of that immense capability to compute? Besides bare metal we will need, in addition, software that matches the hardware. Otherwise, it would be like Uraganus having enough trucks to transport the Alps within just one decade—a fleet of some six trillion vehicles—but not having enough people to drive those trucks or enough gasoline to supply them with.

The present book is about the AI technology that we will need—the software to be run in year 2045. The key issue is scaling. Biological brains seem to scale their intelligence well. The jump from chimpanzee intelligence to human intelligence is a big one, large enough to support our language, science, technology and art. And yet our brains are only about three times the size of those that chimpanzees carry in their skulls. This is an example of a good scaling capability of intelligence implemented in biological neurons; the shift from living on trees to creating an entire civilization is made by increasing the resources by less than a factor of ten. We humans have some 8.6 x 1010 neurons up from 2.8 x 1010 in chimpanzee brains. That is, not even a single zero needs be added to the number describing the needed resources. Similarly, if we compare a mouse brain to a human brain, the difference is not that big. We are talking here about a factor of one thousand, or three zeros.

To expand on the idea that we need to seek intelligence technology that scales well, let us recall further a fact that, across species, brain size is affected grossly by the body size of the animal maybe more than by the intelligence of that animal. Elephants, whales and even dolphins have brains bigger than humans. Bottlenose dolphin (Tursiops truncatus) beat us by about 200 grams of raw brain material. That is quite a chunk—one half of the mass of chimpanzee brain. Sperm whales beat us big time. This giant has a brain about six times bigger than ours! And yet our kids can learn timetables and whale offspring can’t. Obviously, sheer brain size does not guarantee that a species will invent science and technology and build a civilization. There are more factors to intelligence than just throwing resources at it. Neuron counts alone may have a similar role as the bare metal has for computation; it is a necessary but not a sufficient requirement. Neurons with their synapses and all their other proteins and molecules may be just the bare carbon, the foundation on top of which one yet needs build intelligence.

In contrast to biological intelligence, our AI technology does not scale well when it comes to attempts in increasing its intelligence. In Chapter 4, I will present numbers showing that today’s best AI technology has poor scaling properties. In Chapter 8, I will assess the target numbers—the levels that a successful AI would need to reach if it is to match the intelligence of humans. These calculations will give us similar disappointing numbers as Uraganus received when he asked to move Alps down to Arabian Peninsula. The AI technology of today is no match for the level of ambition we have. The machine learning tools of today are like trucks for Uraganus’ plans: a great help on a smaller scale, but cannot bring us where we want to be. Moreover, similarly to Uraganus, we do not have all the time in the world, as Kurzweil predicted that we will have enough computer power by the year 2045, and he may be right. It would be a pity to leave these resources unused. Given that a great number of people have been thoroughly inspired as children by science fiction books, movies, and computer games, and are properly trained in math and machine learning to assist the effort, it seems that all the preconditions for building a human-level AI are fulfilled. Except that one link is missing: we do not have blueprint plans on what exactly needs be done.

In this book I describe how I think we principally need to approach the creation of an AI that reaches human-level intelligence. We need to start building a whole new generation of AI, and I suggest we will do this through distributed task switching. This technology solves the scaling problem by avoiding the capital shortcoming of today’s AI: trying to pre-ready a machine with a full load of knowledge telling the machine what to do in all the possible situations that the machine may ever encounter. As it will turn out, this full-knowledge-load approach does not scale well. Chapters 3 and 4 will help us understand how this scaling problem manifests itself already today. In Chapters 5 to 10, I will cover our biological brains and explain step-by-step the tricks that our minds use to circumvent this scaling problem. The actual algorithm that implements this new type of AI, I will describe in Chapters 10 to 12. In the remaining parts of the book we will explore the implications of this new approach. First, we will cover the advantages, and some disadvantages, that this new approach has. These will be Chapters 13 to 15. A big issue is training some parts of that new algorithm, which we will address in Chapters 16 to 19. Finally, it will be necessary to consider the implications on our society and how this may change our lives, and my views on these questions will be covered in Chapters 20 to 23.

By the end of the book the reader should have a good idea on how I think we can make come true the imaginations of our boy René and how to fulfill the predictions made by Ray Kurzweil. The answer to the question of whether machines can achieve human-like intelligent will be decidedly affirmative. However, it will also become clear that the effort that we will need to invest will be considerable. Intelligence will not come easily to silicon devices. Every new milestone will require more sweat on our human side. The reward at the end, however, I believe will be great.