AI curiosity

AI curiosity

The incuriosity of genAI is an understatement. When chatGPT became popular in early 2023, it was even more striking than their almost limitless knowledgeability.

"I know that I know nothing" has been taken for granted since Socrates. After Greek philosophy, we had to wait for almost 2 millennia, and the advent of printed book in the XVth century, to get to the next step: "The more I read, the less I know, the more I want to read".

This stance is best illustrated by the numerous Renaissance scholars who began to spread throughout Europe soon after printed books became commonplace, with Joseph Justus Scaliger being perhaps the most notable example.


Renaissance scholar, by Mistral

Scaliger might have read 2000 books during his lifetime(?), yet this is a trifle compared to what chatGPT 4 read (at least 1000 times that number).

Despite all the accumulated knowledge, whenever you ask chatGPT a question --dumb or smart, it will patiently wait for your prompt and answer obediently, without trying to know more. Maybe some philosopher will argue that the root of consciousness, which tells us apart with machines, is hidden in the very sparkles of curiosity.

But... for the time being,

To make AI useful for exploration and innovation, we must engineer her curiosity.

Engineering AI curiosity

How is curiosity ingrained in practice?

Let's look at a simple neural network model very common in Machine Learning, (I think it will be used to drive generative AI exploration tasks quite soon): the Variational AutoEncoder (VAE).

This model is extremely handy for goals reaching, because what it learns through interesting observations is dumped in an abstract space called a latent space. When a critical mass of observations have been gleaned, it is possible to sample random vectors from this latent space and "decode" them back into plausible locations which are very likely to be interesting for the problem at hand.

What is curiosity?

How to define "interestingness"? If we take the most general approach, it is a very hard problem (see reference 1, below) because something "interesting" ought to be an emergent pattern from field observations, it shouldn't be influenced by our human perception of what is innovative or disruptive.

Thankfully, in many real-life situations, we can contend with a notion of "interestingness" which is quantified by humans. To drive AI curiosity towards interesting findings, ML engineers use what is called a loss function.

Here is how it works for VAEs: for every decoded plausible location from the latent space, the distance between the resulting observation and an ideal, human-chosen goal is measured by the loss function and given a penalty: the further the distance, the higher the penalty.

The sampled observation, along with its penalty, is then added to the training set of the VAE and the VAE is re-trained, so she will learn to optimize the penalty. In many situations, she will be better at reaching here goal over time.

Exploratory limitations

When the exploration space is very large, there are some pitfalls like being trapped in local extrema, or being subjected to bias effects like the attractor effect (many scattered locations yield the same observation) or the butterfly effect (nearby locations yield chaotic observations).


Common pitfalls (source: Mayalen Etcheverry, ref 1)

Those can be frustrating, because huge exploration spaces are exactly the kind of use cases where AI is expected to shine.

As it turns out, there are ways to work around them. "Diversity augmentation" techniques won't be described here (see ref 2 as a starter), let's just say that the ways depend on the actual ML approach chosen: VAE is just one AI approach among many suitable for exploration. Another top choice is Evolutionary Algorithms (EA).

EA is quite good when exploring rough landscapes (i.e., when the loss function is not differentiable).

Speeding up exploration

Most often than not, real-life exploration models are very complex (if they were simple, why would we need AI after all?)

Since ML requires a lot of computation, a common practice is to build a surrogate model (ref 3) of the real-life situation under inspection. One way to think of a surrogate is: a high-fidelity simulator, much more lightweight than the actual model.


The attic: where all exploration HQs begin. (made with Mistral)

On the downside, finding a faithful surrogate is not straightforward. Fidelity must be thoroughly tested.


Building malevolent AIs with the help of curiosity?

You've noticed that I haven't mentioned cybersecurity anywhere in this article... That's because I wanted to introduce you to the fundamental concepts of unsupervised AI exploration.

The next instalment, called How I trained an AI for nefarious purposes, will get hands-on... :-)


References

1: Mayalen Etcheverry, curiosity-driven AI 2023 thesis page 27 https://mayalenetcheverry.com/assets/publications/thesis/manuscript.pdf

2: Fabien Benureau, robotics exploration 2015 thesis page 84 (intrinsic diversity measure,), https://theses.hal.science/tel-01259955/document

3: https://en.wikipedia.org/wiki/Surrogate_model






Maybe closer partnerships with think tanks who are ostensibly already dedicated to the business of innovation and new directions

Christophe Parisel

Senior Cloud security architect at Société Générale

3 个月

Immanuel Chavoya?? his could help you and your team

AI exploration indeed needs to evolve. Implementing frameworks that prioritize curiosity and innovation can drive significant advancements in this field. Insights from your upcoming proof-of-concept will certainly be anticipated

要查看或添加评论,请登录

Christophe Parisel的更多文章

  • Adversarial lateral motion in Azure PaaS: are we prepared?

    Adversarial lateral motion in Azure PaaS: are we prepared?

    Lateral motion techniques are evolving in PaaS, and we should be worried. Let's discuss a risk confinement approach.

    18 条评论
  • How will Microsoft Majorana quantum chip ??compute??, exactly?

    How will Microsoft Majorana quantum chip ??compute??, exactly?

    During the 2020 COVID lockdown, I investigated braid theory in the hope it would help me on some research I was…

    16 条评论
  • Zero-shot attack against multimodal AI (Part 2)

    Zero-shot attack against multimodal AI (Part 2)

    In part 1, I showcased how AI applications could be affected by a new kind of AI-driven attack: Mystic Square. In the…

    6 条评论
  • Zero-shot attack against multimodal AI (Part 1)

    Zero-shot attack against multimodal AI (Part 1)

    The arrow is on fire, ready to strike its target from two miles away..

    11 条评论
  • 2015-2025: a decade of preventive Cloud security!

    2015-2025: a decade of preventive Cloud security!

    Since its birth in 2015, preventive Cloud security has proven a formidable achievement. By raising the security bar of…

    11 条评论
  • Exploiting Azure AI DocIntel for ID spoofing

    Exploiting Azure AI DocIntel for ID spoofing

    Sensitive transactions execution often requires to show proofs of ID and proofs of ownership: this requirements is…

    10 条评论
  • How I trained an AI model for nefarious purposes!

    How I trained an AI model for nefarious purposes!

    The previous episode prepared ground for today’s task: we walked through the foundations of AI curiosity. As we've…

    19 条评论
  • The nested cloud

    The nested cloud

    Now is the perfect time to approach Cloud security through the interplay between data planes and control planes—a…

    8 条评论
  • Overcoming the security challenge of Text-To-Action

    Overcoming the security challenge of Text-To-Action

    LLM's Text-To-Action (T2A) is one of the most anticipated features of 2025: it is expected to unleash a new cycle of…

    19 条评论
  • Cloud drift management for Cyber

    Cloud drift management for Cyber

    Optimize your drift management strategy by tracking the Human-to-Scenario (H/S) ratio: the number of dedicated human…

    12 条评论

社区洞察

其他会员也浏览了