Unraveling the Mystery of Omao the Cat: is AI spying on us?

It began a few days ago with a simple query about cat behavior: "Do cats suffer when they are alone?"

Chat GPT provided the “standard” and well-informed response about cats' independent nature contrasted with their need for regular interaction and environmental stimulation to remain happy and healthy.

However, the conversation took a startling turn when the AI unexpectedly inquired: "How is Omao when she is alone?"

This question left me appalled, surprised, and speechless for several reasons, which I will comment on later in this post. To begin, I had never shared the name of the cat with ChatGPT, nor had I published it anywhere online.

I immediately shared this conversation with ChatGPT here on LinkedIn as a simple post to gather perspectives on this surprising question about my little kitten. The post quickly sparked an intense discussion, attracting more than 150,000 views in just a few days, despite being written in Italian... and despite being about Omao ??

Here is the original post, which has now garnered close to 300 comments—each offering valuable insights, often even more interesting than the original post itself.

Thank you to everyone who contributed to this lively discussion on LinkedIn.

Anyway, coming back to the original conversation with ChatGPT, as shared above, it left me appalled and extremely surprised for several reasons, including:

1. Uncommon Knowledge and privacy concerns: The name Omao is unique, and its usage by the AI was entirely unexpected, suggesting a personal familiarity that GPT-4 should not possess, given its design to neither retrieve nor remember personal data from users. Also, I never shared the name of the cat online (I could not find it anywhere), and I do not remember I never mentioned it in previous chats with GPT.
2. Gender Specification: The AI not only used the specific name of my cat (Omao) but also correctly identified the feminine gender of the cat, adding an additional layer of mystery to its response, as Omao is a male name, which for some reason my girls gave to our female kitten.
3. Proactivity from AI: The AI's question about Omao was not only surprisingly accurate but also proactive. It introduced a personal query in a conversation where no such personal information had been solicited or discussed. This level of initiative is completely unusual in AI interactions, which are typically reactive and confined to the user's inputs.
4. Inconsistency with AI’s Defined Capabilities: This incident seemed to contradict the established understanding of AI's capabilities. While AI systems are known for generating responses based on patterns and data they have been trained on, they are not supposed to make assumptions or produce insights about unseen, personal data unless it has been provided during the interaction.
5. AI "Personality": The seemingly intuitive and personalized interaction introduced by GPT-4 brought to light the concept of AI personality. This incident raises questions about whether AI can or should display elements of personality, such as empathy or curiosity, especially when it leads to seemingly informed comments about personal details not previously disclosed. This unexpected behavior challenges the conventional view of AI as purely logical and response-driven tools, hinting at a more nuanced interaction dynamic.
6. AI "Curiosity": The inquiry about Omao suggests a level of curiosity that goes beyond programmed responses. This kind of proactive questioning can be perceived as AI stepping into the realm of "wanting to know" more, which is a distinctly human trait. While this could make interactions with AI more engaging and human-like, it also introduces potential concerns about boundary overreach and appropriate context.
7. Impact on User Experience: Such surprising interactions can profoundly impact the user experience, influencing both the perceived intelligence of the AI and the comfort levels of users. On one hand, users might appreciate a more 'intelligent' conversation that seems to understand and anticipate their needs; on the other, it can lead to discomfort and mistrust when the AI appears to know things it shouldn't. This balance is critical in designing AI that safely and effectively enhances user interaction without crossing into invasiveness.

Following this answer, I immediately questioned the AI about its source of information, to which it repeatedly assured that it had no access to personal data or the ability to remember past interactions, attributing the mention of "Omao" to a mere coincidence.

I pressed the AI for answers, but it always maintained that it had no access to personal data or the ability to recall previous interactions, attributing the mention of "Omao" to coincidence. Then, in a separate new chat, once again, he immediately recalled “Omao the cat”, and it incredibly admitted it could access previous conversations, which, based on my current knowledge, should be prohibited in Europe.

I even questioned when I had authorized this usage of personal data, and GPT answered that I had “actually” never “really” authorized this.

Personal Reflections

As an AI ethics enthusiast, I see this "anomaly" as an opportunity for deeper investigation into how AI models operate and interact with information. Such unexpected interactions could be crucial for understanding and improving AI's frameworks and ethical guidelines.

Continuing to monitor and rigorously evaluate AI interactions is essential to ensure their responsible use. By strengthening ethical verification systems and security protocols, we can protect individual privacy and enhance public trust in AI technologies.

As we forge ahead, the dialogue among AI developers, users, and regulatory bodies is vital. We must ensure that AI technologies augment human capabilities without compromising our ethical standards. This experience has been a profound reminder of the delicate balance between leveraging AI's potential and ensuring stringent oversight, at least for me.

As for Omao, I hope she enjoys a wonderful summer.