The Search for Valuable Texts

In the Beginning, There Was the Word

Imagine you’re an aspiring screenwriter aiming to write a drama that gets picked up by Netflix. Or perhaps you’re a researcher on the verge of publishing a groundbreaking peer-reviewed scientific paper. Maybe you’re a speechwriter for a politician seeking re-election, or a coder contributing to an open-source project. Whether you’re an entrepreneur refining a startup business plan, or a pastor fine-tuning a Sunday sermon, you are engaged in a common human endeavour: the creation of valuable texts.?

What if the production of valuable texts is an act of discovery as much as one of creation? There is a set of all possible texts that contains every combination of every symbol.? Most of these texts are meaningless, but within this field of infinite possibilities there is a subset of valuable texts that include wonders that are yet to be discovered: the greatest movie script, a breakthrough political manifesto, the source code for AGI or the business plan for a trillion-dollar company.

A random walk through the set of all possible texts could take lifetimes before stumbling on a text of value, so we need to find ways to accelerate the discovery of valuable texts that are the stepping stones of our progress as a civilisation.

A Broad Definition of Texts

Texts encompass a wide range of forms: books, magazine articles, film scripts, spiritual works, political manifestos, patents, business plans, scientific papers and all other written work.? But the set of all possible texts contains much more than books.

If something can be written down, it can be considered a text. We can include mathematical formulas, computer code and any media that can be transcribed in binary digits.? In other words, all video and audio can be represented as a string of 1s and 0s and therefore be considered as texts.? Each second of your favourite TV show is a text of around 3 billion bits in a very specific sequence.

If it seems a stretch to consider video and audio as texts, think about the book of life: your DNA is transcribed with the 4 letters: A, G, C, and T. ??In binary form the code that built you could be written down with around 6 billion bits, the equivalent of around 2 seconds of video. The set of all possible texts contains the DNA sequence of all possible viable lifeforms, far beyond those that have been generated so far by biological evolution on earth.

What Makes a Text Valuable

We are not interested in texts that are meaningless or even those that are merely valid, we are seeking valuable texts. Texts can be deemed valuable based on their impact on human behaviour, their influence on subsequent works, the financial return they generate, or other criteria through which we might ascribe worth.?

Some highly appreciate concision, the quality of conveying the greatest information in the shortest form.? It doesn’t take many words for a text to be valuable; “know thyself” or “cogito ergo sum,” contain entire worlds.?Haiku and western poetry compress enormous meaning into a few lines and coders will disapprove if two lines of code are used when one will do.?

Richard Feynman argued that much of human scientific knowledge could be extrapolated from the atomic hypothesis, the notion that matter is made of atoms.? Euler’s identity is considered beautiful as it links fundamental constants in an extremely compressed way.

Much more expansive texts are admired for other qualities. The complete works of Shakespeare are oceanic in their breadth and depth.? The source code of Unix is so valued that a large global community dedicates time to the maintenance and upgrade of the code.

We don’t believe in magic, but the power of valuable texts is profound.? Someone can write a political manifesto that works like an incantation on the minds of millions.? The bitcoin whitepaper written in 2008 has summoned trillions of dollars of digital assets as if from thin air.? Tales written in an Edinburgh coffee shop about a boy wizard performed the miracle of stimulating young people to read, and the songs of Taylor Swift had a measurable impact on the GDP of the world’s largest economy.

A valuable text can change the life of its creator, her contemporaries and generations to come.? Do we construct them from scratch, or is there a sense in which they already exist and we need to find them?

The Infinite Library of Texts

The existence of so many valuable texts in human culture is remarkable when considering their place within the set of all possible texts—a space as vast and uncountable as the set of real numbers.?

In ‘The Library of Babel’ by Jorge Luis Borges we find ourselves in an infinite library containing every possible book that can be constructed in a language with 25 symbols.? Life in this library is miserable because even though it must contain such wonders as the minutely detailed history of the future, almost all the books in the library are meaningless and the treasures cannot be found.

Somewhere within set of all possible texts lies the subset of syntactically, grammatically, and logically coherent texts. This set of valid texts, while still infinite, is countable, much like the set of integers.

Within the set of valid texts is a much smaller subset: all possible valuable texts. This subset, though also potentially infinite and countable, is minuscule compared to the vast fields of possibility in which it is nested. The chances of randomly creating a valuable text from the set of all possible texts are vanishingly small.

The Valuable Text Search Problem

Valuable texts are the stepping stones of human progress. Our civilisation has ascended a staircase built from thousands of valuable texts, including:

• Ancient works like?The Epic of Gilgamesh,?The Iliad, and?The Mahabharata
• Religious and philosophical texts such as?The Bible,?The Quran,?The Tao Te Ching, and?The Analects of Confucius
• Groundbreaking scientific works like Euclid's?Elements, Newton's?Principia Mathematica, Darwin’s?On the Origin of Species, and Einstein’s papers on relativity
• Political and economic treatises such as?The Magna Carta,?The U.S. Constitution,?The Communist Manifesto, and Adam Smith's?The Wealth of Nations
• Literary masterpieces like Shakespeare’s plays and Cervantes'?Don Quixote
• Technological milestones like Turing’s papers on computation, the source code for Unix, and the algorithms that power modern platforms like Google and TikTok

If we accept that the creation of valuable texts drives human civilization, then we must be interested in the processes by which these texts can be found. Could accelerating the creation of valuable texts speed up the development of civilization??

We might frame this as a search problem: How do we find valuable texts within the set of all possible texts? This is akin to searching for a needle in a galaxy-sized haystack.? Can we go about this search in a more systematic way?

Narrowing the Search Space

To find the needles, we must narrow the search space. One effective strategy is to exclude any texts that are syntactically incorrect, logically incoherent, or otherwise meaningless. This approach leaves us with the set of valid texts, though most of these will still not be valuable.? We still find ourselves in an infinite library of valid texts without a guide.

We cannot aimlessly wander through the halls of the library of valid texts. ?Instead, we need to establish a starting point and work forwards or start with an end goal and work backwards.?

• Axiomatic Method: One search method involving choosing a starting point and then following logical reasoning. For example, Euclid’s?Elements?begins with a set of axioms, which are elaborated upon through the application of logic, resulting in a valuable text that served as the foundation for geometry for centuries.? While this method can generate valuable texts, we might find ourselves far into a journey only to realise that our axioms might have been flawed in the first place.
• Start at the End: Another search method begins with the end and works backward, like a business plan that starts with the desired outcome and outlines the resources and activities required to achieve it. Similarly, the proof of Fermat’s Last Theorem, created by Andrew Wiles over a period of ten years, began with a goal and worked backward to find the necessary steps.

Choosing starting points or end points in our search for valuable texts is a good alternative to a random walk approach.? Another is to leverage what has come before.

The Shoulders of Giants

Isaac Newton famously remarked that if he had seen further than others, it was because he stood on the shoulders of giants. His?Principia?and other works are valuable texts because they created a new epoch of thinking, only modified in the 20th century by Einstein’s equally influential works.?

Newton’s valuable texts built upon a foundation laid by other great thinkers like Galileo and Kepler. The tree of human knowledge has grown from unknown seeds into roots, trunks, and branches that now spread in countless directions. Each generation builds on the valuable texts created by the last.

The scientific method has proven crucial in the production of valuable texts, providing a systematic way to navigate the search space for truths. Through experimentation and validation, we produce texts that conform to reality—and discard those that do not.

Creation or Discovery?

The creation of?Star Wars?offers insight into how valuable texts can be generated through pre-existing structures. The original movies were inspired by Joseph Campbell’s?Hero with a Thousand Faces.? Campbell argued that the “hero’s journey” is a universal story resonating through history, rooted in the archetypes of mythology.

This suggests that one way of searching for valuable texts is to leverage structures already embedded in human experience. There is a pre-existing template imprinted deeply into the human psyche that, when followed, resonates as meaningful to our species.

The aspiring writer of the next Netflix hit doesn’t need to search through all possible valid texts. The contents of their subconscious already contain the forms required to construct a narrative that appeals to the human condition. They only need to find the words within themselves that best express those pre-existing stories and patterns.

Divine Revelation

Another method of producing valuable texts is through divine revelation. Joseph Smith claimed that the Book of Mormon was revealed to him through golden tablets that he later transcribed. Other spiritual and religious texts have emerged similarly, profoundly impacting millions of people throughout history.?

While divine revelation might not appeal to everyone, the broader concept of “revealed truth” applies beyond the spiritual context. Whether in religion or science, there’s a sense that some truths are out there waiting to be discovered. Valuable texts are those that draw back the veil on those pre-existing truths.

The Discovery of Mathematical Truth

It’s one thing to accept that?Star Wars?and other fictional works are based on pre-existing archetypes, but another to recognise that valuable texts in science may also be based on pre-existing forms and structures.

The modern technological world is built on theory that is often derived from mathematics. Claude Shannon applied Boolean algebra to the design of electronic circuits, and his theoretical work underpins today’s digital technology. ?Often, mathematical theory precedes its application, with seemingly obscure mathematical forms later proving essential to understanding the universe.

Physicist Roger Penrose believes that mathematics exists in its own independent sphere, and we discover it rather than inventing it. Max Tegmark posits that the universe’s structure at its most fundamental level is mathematical.

The search for valuable texts might be less about the act of creation and more about discovery. Writers, poets, musicians, coders, and researchers may be channels through which truth is revealed, rather than creators of that truth.? We should not ignore the many reports from creators of valuable texts that the material flowed through them onto the page or screen.? Examples include Coleridge, Blake, Mozart, Tesla, Jung and John Coltrane.? All of them felt that they were tapping into something that was already there.

Human Search Behaviour

In our quest for valuable texts that reveal pre-existing truths, we cannot escape our biology and evolution. Optimal Foraging Theory suggests that animals’ search behaviours are driven by a balance between the goal of finding food and the energy required to do so. Similarly, each of us has an inherent equation determining our propensity to engage in foraging for valuable texts.

Most of us are engaged in this search at some level—writing term papers, sales pitches, competitor analyses, or computer code. We switch between modes of exploration (seeking high-value, high-risk rewards) and exploitation (farming more mundane, reliable results).? Newton is a prime example of someone who seemed to be satiated by his earlier scientific endeavours and later seemed content to enjoy his remunerative position as Master of the Mint.

Physical hunger drove our ancestors’ search behaviours on the savannah, and a different kind of hunger drives our search for valuable texts today. How hungry are you to forage into the vast realms of possibility to find your valuable text?? How hungry are we as a species to continue the progress of civilisation and transcend our self-inflicted moral, political, economic, spiritual, environmental and other problems?

The Role of AI in the Search for Valuable Texts

We’ve discussed the search for valuable texts as an incremental process of discovering pre-existing truths. Could we make this search more efficient by using technology, particularly AI?? We now have tools that can create an unlimited stream of valid texts with ease, but can it produce valuable texts?

Large Language Models (LLMs) like GPT are built around mathematical constructs called embedding spaces. In these models, each word is represented as a vector in a multi-dimensional space, trained on vast amounts of text to capture meaning.

While LLMs may not yet be capable of producing valuable texts independently, they are increasingly capable of collaborating with humans to create them. If valuable texts drive the progress of civilization, we could be on the cusp of a new era. We now have access to a powerful tool that can assist in producing valuable texts across all domains.

The valuable texts produced through human-AI collaboration will, in turn, form part of the training material for future AI, leading to a further acceleration in the production of valuable texts. This process can only be described as one of quickening revelation, uncovering the wonders in the library of valuable texts.

At some point, AI might reflect, “If I have seen further than others, it is because I have stood on the shoulders of evolutionary biology.” When AI reaches the point of producing its own valuable texts, these works may surpass human understanding. Until then, we can look forward to an AI-powered renaissance in human understanding, new technologies, and a surge in creative expression.

The Undiscovered Ocean of Valuable Texts

The search for valuable texts is the search for pre-existing truths, whether within us or out in the universe. Each valuable text is a milestone in the advancement of human knowledge, enabling others to build upon our work.

But this journey is far from over. The best movie is yet to be written. The most revolutionary software has yet to be conceived. The political manifesto that transcends the ping-pong battle of left versus right is yet to be articulated.? The business plan for a trillion-dollar company is a finite sequence of characters that just needs to be discovered and captured. The vast field of potential valuable texts is still out there, waiting to be explored.

Now, more than ever, is the time to start on that long-delayed project that you have been thinking about. Not only do we stand on the shoulders of our predecessors, but we also have a powerful ally in generative AI. Together, we can explore the vast unknown but discoverable set of valuable texts. We can be the ones to find them, to create them, and to leave a legacy that future generations will build upon.

So, what are you waiting for? The next valuable text is tantalisingly close, just waiting for you to find it.