Borges, Freud & AGI

The great Argentinian writer, Jorge Luis Borges, was said to have had an extraordinary memory. Fearing that he would turn blind quite early in his life, similar to his parents and grandparents, he had trained his mind to memorize great works of literature and could recount them at will. Borges was truly fascinated with memory. In his short story "Funes the Memorious", he writes about one Ireneo Funes who, after an accident where-in he fell from a horse and hit his head, acquires the extraordinary ability to remember everything. Funes, Borges writes, could recollect the shape of clouds at dawn from childhood, learn multiple languages without any effort, and could reconstruct a whole day from memory. Funes says that "I alone have more memories than all mankind has probably had since the world has been the world".

But how is Funes developing new memories? He lies bed-ridden in his room and only recounts his past. His new memories are a recollection of his old memories. Every time he recollects his past, the hippocampus in his brain manipulates the original memory and creates a new one: not an exact replica but still bearing a strong resemblance. So the next time Funes remembers the event, he isn't remembering the original event but a memory of the original event. What happens after a hundred times recollection of that event? Does it bear a strong resemblance to the original or has it been distorted sufficiently to highlight only certain aspects of that original memory? Is this what Freud referred to as the screen memory: something that suppresses unhappy aspects of memory and highlights only the happy ones, primarily used as a survival technique from trauma? Or is it like Theseus's ship: the memory has been replaced by bits and pieces of other memories even though it looks like the original memory?

AGI seems to have a similar burden. The dataset on which AGI can be trained and that has been created by humans is finite, whereas the appetite of AGI seems to be infinite. Sooner or later, AGI will need to be trained exclusively on test data created by AGI itself. Every time the AGI generates data, there would be some distortion inserted at the fringes: something not alarming but still not the original. And the AGI trained on that slightly distorted vision of the world will generate a somewhat more distorted vision. And so on and so forth. After a million iterations, would it still bear resemblance to the world as we know it? Or, by every repetition, do we get acclimatized to the new vision of the world, just like we replace our old memory with it's new version?

Only time will tell. But I feel that Borges, being the clever man that he was, was alerting us to the potential curse of an all-knowing sentient being when he made Funes to say that "My memory, sir, is like a garbage heap". Information replication without the power of abstraction and moderation can, indeed, become an intelligence wasteland.