Asking the right questions

In the ancient land of Cornycobania, there stood an idol of the goddess of wisdom, Ann. The local tribe had long revered this idol, and they would come to her in times of need, seeking guidance and wisdom. This year, the tribe had a question that weighed heavily on their minds. They needed to decide which crops to plant for the coming season. They put their question to the idol in different forms, they made sacrifices, and they prayed to Ann. But the idol remained silent.

The men grew desperate. They began to believe that the goddess thought them no longer worthy of her guidance. One by one, they turned away, until only a small boy remained to face the idol.

The boy approached it and simply asked, "Potatoes or corn?" And to everyone's surprise, the idol finally spoke, uttering very clearly: "Potatoes." The tribe was amazed, they realised that the goddess had answered their question, but they had not asked it in the right way. The harvest that year was plentiful and the tribe thrived, while the boy grew up to become a famous ventriloquist in Europe, but that's a completely different story

We all know, that to get a useful answer, one must have a useful question to ask. Wisdom comes to those who ask the right questions, and clarity and precision in asking are the key to getting the quality answers we need. In the world of artificial intelligence, the process of posing the right questions to AI systems is critical to obtaining potentially useful answers.

Asking the right questions to an AI system requires a deep understanding of the technology, the data, and the problem that needs to be solved. It requires a clear understanding of what information is needed and how it can be obtained. It also requires a critical eye, to ensure that the questions are relevant, precise, and unbiased. Digital computers understand numbers and nothing else. To communicate with them, one must speak their language, the language of math.

To represent real-life phenomena as numerical data, various techniques are employed, such as encoding, feature extraction and feature engineering. These techniques are key to the successful application of AI. Encoding is the process of converting non-numerical data such as text, images, or sound into numerical data that can be processed by the AI system. Feature extraction is the process of identifying relevant characteristics of the data that can be used to represent it numerically. And feature engineering is the process of using domain knowledge to create new features that can improve the performance of the AI system.

It is important to keep in mind that not all data can be represented in a numerical form, and not all numerical representations of data are equally effective. The process of representation and encoding of data is an art that requires skill and experience. It is essential for both training and employing neural networks. The exact ways in which data is encoded also impacts radically the performance and precision of the network.

An interesting example of representation technique is Parsons Code, used to quickly identify music from a tiny fragment of it. This code works by concentrating on melodic motion, the change of pitch which. This would be generally considered a form of feature extraction and it allows for a very short yet unique representation for any musical composition as it becomes a sequence of numbers. Applying the same representation technique to just a fragment of music, allows to search for the shorter sequence of numbers, representing the fragment, within the longer sequences representing full compositions. This is basically how various music identification services work. Parsons Code is a great example of reducing the data to a bare minimum needed for the task at hand, obtaining an efficient, reliable and replicable method.

Representation and encoding techniques play a crucial role in managing and storing data, such as images and music. Commonly used techniques, that we all use every day, include JPEGs for pictures and MP3 for music, which have been developed to provide a highly efficient means of storage and transmission. These techniques have been optimised over time to keep up with the changing needs of society, but the search for new techniques never ends. This search is driven by the need to keep up with the ever-changing demands. From the desire to store and transmit more data, to the need for higher quality images and music, the search for new and improved encoding techniques is a vital part of the digital world. Data representation is the "Secret sauce" of any AI application as it facilitates new and innovative ways of representing data, which in turn lead to improved performance and more accurate results. As the field of AI advances, the search for new and improved techniques is ongoing, driven by the desire for better results and the need to keep up with the ever-changing demands of society.

Feature extraction, of which the aforementioned Parsons Code is a good example, is a powerful tool for identifying hidden patterns in various types of data. Its use has led to a great deal of progress in various fields, including medicine, where it is used to identify hidden patterns in medical data, such as those related to diseases or treatments. It is also used in physics, where it helps to identify patterns in large amounts of data generated by experiments, and in business intelligence, where it enables the machines to extract insights from large amounts of data related to market trends, consumer behaviour and more. A lot of the innovation in AI comes as existing feature extraction techniques are refined and new ones are developed.

The ability to digitalise information efficiently and pose clear questions is essential for machines to provide us with useful answers. The process of data representation and encoding is a key part of this effort, and yet another hidden aspect of the AI magic. As we continue to push the boundaries of what is possible with AI, the importance of data representation and encoding will only continue to grow. It is an ongoing effort that requires a deep understanding of the data and the technology, and a commitment to finding new and better ways to represent and store it.