How We Learn - A Book about Learning in Human Brain and Machines

Recent developments in Machine Learning and Artificial Intelligence have ignited more interest and research on how humans learn. Recently I came across a book that compares human learning to how machines learn: 'How We Learn' by Professor Stanislas Dehaene. Overall it is an interesting book with intuitive explanations of how our brains work, how we learn at different stages of life and?how Machine Learning and neural networks mimics the biological learning process.

Here's a very brief summary of the key concepts in the book.?

The human brain is an extraordinary machine. Its ability to process information and adapt to circumstances by reprogramming itself is unparalleled, and it remains the best source of inspiration for recent developments in artificial intelligence.

Stanislas Dehaene starts the book by explaining that natural selection favored?the?emergence?of learning. He argues that genetic hardwiring helps.Humans possess some highly sophisticated learning algorithms that can refine those early skills according to our experience. Today, neural networks are not able to make the best use of data (therefore the need for massive or large foundation models). Compared to that a baby brain wins, hands-down.

Emerging theory is that our brain acts as a statistician, superior to machines. It was amusing to read that "even babies understand probabilities".?

Dehaene believes that neural networks mimic only a small part of our brain's function - the first few stages of sensory processing. This is where our brain operates in unconscious?matter (very fast) in the first 200-300 milliseconds.?

The book has three parts:

1 - What is learning - in silicon?and neural circuits - unconscious simulation - mental models - compare performance of computer algorithm vs brain - honing on theory of optimal learning, best learner operates as a scientist who makes rational use of probabilities and statistics

2 - How Our Brains learn - using psychology and neuroscience, the author explains why babies are the best learning machines! Nature and nurture join forces to help us learn.

3- The Four Pillars of Learning:

There are four universal pillars which massively modulate our ability to learn. These four pillars are:

a - Attention - a set o f neural circuits that select, amplify and propagate the signals that we view as relevant, multiplying their impact by x100. Note that attention is one of the key to success of transformer-based models like ChatGPT.

b - Active engagement - active generation of hypotheses with motivation and curiosity

c- Error Feedback - when world violates our expectations, error signals spread throughout our brain

d- Consolidation over time - brain compiles what it has acquired and transfers it into long-term memory, thus freeing neural resources for further learning. Repetition plays an essential role in this consolidation process. Sleep is not passive, brain revisits its past states at a faster pace and recodes the knowledge acquired during the day.?

Seven Definitions of Learning

This, I believe, is the unique value of this book. Here Stanislas Dehaene introduces the biological learning mechanisms and provides examples and explanations of Machine Learning techniques that correspond to each of these learning.

He starts by a definition: Learning is to form an internal model of the external world. Then he presents seven ways to think learn.

Learning?is:

1. adjusting the parameters of a mental model
2. exploiting?a combinatorial explosion
3. minimizing errors (he uses the example of a hunter adjusting telescope)
4. exploring the space of possibilities
5. optimising a reward function (backpropagation, actor/critic, adversarial learning)
6. restricting search space
7. projecting a prior hypotheses (its not nature vs nurture - how baby brain knows the difference between people and inanimate objects, via millions of years?of evolution)

I highly recommend reading the book for detailed explanations.

Overall I enjoyed reading the book and benefited from it. There is room for improvement though. There were a few stories in the book where I thought the author overplayed the capabilities of ML (for example, DeepMind playing games) and it would have been better to provide more information about its limits.

I hope that this summary was helpful and welcome any comments and thoughts on the fascinating topic of how to accelerate learning for humans or machines or even, together.