Making My Peace with Information

By?Geoffrey Moore

Author –?The Infinite Staircase: What the Universe Tells Us About Life, Ethics, and Mortality

For a long time, I have been uncomfortable with the way writers in technology, biology, and mathematics have been using the term information.?Here are some of my “issues”:

• Information theorists, following the lead of Claude Shannon, often talk about information as the obverse of entropy.?That is, the more ordered something is, the more information it contains.?I get the idea, but it plays havoc with the physicist’s definition of entropy as derived from the Second Law of Thermodynamics.?That is, entropy in the context of the Second Law is completely indifferent to the amount of order or disorder there is in any system, nor does it have any interest in information.?All it cares about is the distribution of heat.
• Geneticists talk about the information in the genetic code.?Now, from a human perspective, it is clear that we have extracted an enormous amount of highly specific information from all sorts of DNA simply by treating it as a code for specifying amino acids, and we can use that information, in turn, to better understand the manufacture of proteins and the molecular biology of the cell.?In other words, we have clearly been informed.?But to say the cell has been informed anthropomorphically overstretches the term.?Cells do not have anything even resembling a brain, so there is simply no one there to inform.?All cells can do is detect and respond to chemical signals, be they internally or externally generated.?They do not make choices.?They involuntarily enact a cascade of chemical reactions, all driven by the Second Law.
• Developmental biologists talk about how fetal development is informed by genetic instructions.?They describe highly specific sets of interactions that unfold against very precise sets of locations so that there is every appearance of an intelligent operator, but once again, there is no supervising agent anywhere to be found.?We say the genome codes for these developments, but that implies that biological processes are driven by intention, and we know that is not the case.?

So, what the heck is going on?

I got a clue from a book I have just started reading, The Self-Assembling Brain, by Peter Robin Hiesinger.?Hiesinger distinguishes between two kinds of information:

1. Endpoint information, which is extracted from completed artifacts and exists independently of them in the form of abstract descriptions which can, in turn, be used as blueprints to create more of the same artifact, and
2. Algorithmic information, which is a sequence of prescribed actions that drive the assembly of an artifact without the use of any endpoint information, as exemplified by agent-based computer simulations like the Mandelbrot Set.?

The former I would term “information as a noun,” whereas the latter is more like “information as a verb.”?There is a parallel distinction in classical philosophy, contrasting natura naturata, nature as phenomena observed in the world, and natura naturans, nature engaged in the process of creating those phenomena.?There is a less perfect but still relevant parallel to note as well, this one between being and becoming, the former set apart from the flux of time, the latter deeply embedded in it.?

Regardless, for my purposes, algorithmic information is the essential concept.?It is the defining ingredient in the dynamics of complexity and emergence, two ideas that are core to my understanding of the universe described in The Infinite Staircase.?Algorithms can evolve through natural selection without intelligent design because Darwinism selects for behaviors that create competitive advantage, regardless of understanding or intent.?Complexity in this view is an unintended consequence of algorithms.?As for the algorithms themselves, what drives them ultimately is the Second Law—the universe’s unceasing drive to disperse energy in order to cool down.???

To bring this closer to hand, however, Hiesinger’s distinction between endpoint and algorithmic information captures the distinction between traditional artificial intelligence and machine learning.?GOFAI (Good Old-Fashioned AI) seeks to recreate cognition, emulating the various lobes of the cerebral cortex, whereas machine learning seeks to recreate autonomous systems, emulating the cerebellum and brain stem.?In The Master Algorithm, Pedro Dominguez presents five candidates for the role of master algorithm, three of which align with the cerebral cortex—Symbolist, Bayesian, and Analogizer—and two with the cerebellum and brain stem—Connectionist and Evolutionary.?His point, as well as my point here, is that in the end there must be a fusion of all to create a complete artificial intelligence.

At present, machine learning has outpaced GOFAI by a long shot, which to my mind means we need to redirect our attention to the other side of the equation to make the next big leaps forward.

That’s what I think.?What do you think?

Follow Geoff on LinkedIn?|?Geoffrey Moore Main Mailing List?|?Infinite Staircase Mailing List

__________________________________________________________________________

Geoffrey Moore?|?Infinite Staircase Site?|?Geoffrey Moore Twitter?|?Infinite Staircase Twitter?|?Facebook?|?YouTube