Harnessing the Boundless Power of Unstructured Data

In a world increasingly shaped by artificial intelligence, understanding human communication in all its complexity has never been more critical. Speech serves as a powerful lens into this complexity, weaving together tone, rhythm, and emotion to uncover layers of meaning beyond the words themselves.

These paralinguistic elements—non-verbal features such as intonation, speed, and vocal quality—convey emotions and intentions that often remain unnoticed in traditional analysis. By decoding these subtle cues, multimodal machine learning drives innovations across fields like healthcare, security, and the creative industries.

In Greece, researchers are making significant strides in this domain, showcasing the country’s potential for global innovation. Dr. Theodoros Giannakopoulos and his team at the Multimedia Analysis Group of the Computational Intelligence Lab (MagCIL) at Demokritos are pioneering multimodal machine learning techniques that analyze unstructured data, such as speech and images, to reveal new dimensions of communication.

While tech giants like Google and Meta lead in automatic speech recognition (ASR) for transcribing audio to text, MagCIL goes deeper, focusing on how people speak rather than what they say. “At MagCIL, we develop algorithms to analyze tone, behavior, voice clarity, and other paralinguistic features,” Giannakopoulos explains.

This approach offers novel insights that can enhance public health, accessibility, and human-computer interaction. For instance, MagCIL’s algorithms can identify speech patterns linked to cognitive or emotional states, paving the way for mental health diagnostics and personalized learning solutions.

MagCIL is also making waves in the field of music with an innovative algorithm designed to identify songs played in public spaces. Targeted at businesses, this tool helps ensure that revenue reaches the rightful artists. Unlike conventional systems, this algorithm focuses on recognizing cover songs, live performances, mashups, and remixes—categories often overlooked by conventional music recognition technologies.

The lab is also home to FaRADAI (Frugal and Robust AI for Defence Advanced Intelligence),? a flagship program funded by the European Defence Fund. This program enables researchers to develop algorithms that require minimal data for training, a concept known as few-shot learning or frugal AI. This approach is particularly beneficial for researchers in data-scarce environments, enabling them to achieve high-performance results despite limited resources.

Theodoros Giannakopoulos is particularly interested in using speech analysis to identify psychiatric conditions and cognitive conditions, such as depression, anxiety, and Alzheimer’s disease. By analyzing speech patterns, the lab aims to detect markers linked to these disorders. While promising, this application is still in its infancy, as researchers face hurdles related to data accessibility and compliance with strict medical protocols. "Acquiring data involves navigating complex rights and regulations, which can be a significant challenge for researchers," Giannakopoulos notes. "But these safeguards ultimately ensure a more secure and ethical framework for users," he adds.

The lab has also developed specialized datasets, including one from computer science conferences annotated with audience engagement levels. “We recently analyzed voice characteristics in successful TED Talks, investigating how traits like tone and modulation correlate with long-term impact, such as views and influence,” Giannakopoulos shares.

Another area of exploration involves evaluating the communication skills of public speakers. The team studies how these factors influence audience engagement by analyzing elements such as pauses, vocal clarity, and modulation. “This type of analysis could be developed into a consumer-facing product,” Giannakopoulos suggests.

Although many of these applications remain experimental, the lab collaborates with industry. One notable collaboration is with Behavioral Signals, a company specializing in behavioral and speech analysis. This collaboration serves two primary purposes:

1. Deepfake Detection: Leveraging advanced deep learning techniques to identify manipulated media (deepfakes).
2. Behavioral Insights in Business: Analyzing emotions and behaviors in business interactions, such as call center environments. This analysis evaluates customer engagement and assesses how effectively agents handle calls—not based on content but on delivery and tone.

This work has evolved into a commercial product generating revenue, showcasing how behavioral analysis (or signal processing) can optimize call center operations and enhance business outcomes.

Looking ahead, Giannakopoulos envisions a path of innovation and sustainability for his team. "One of my aspirations is to create a spin-off from the lab to transform our research ideas into real-world solutions," he shares. He highlights the supportive environment at Demokritos, particularly inspired by successful spin-offs launched by other researchers at Lefkippos. "With the concentration of AI expertise here, now is the perfect time to take this step," he remarks.

“Communicating the work being done is more important than ever," Giannakopoulos reflects. "By nature, research aims to make an impact—whether by transferring its results to industry or benefiting the public sector. Clear communication of its purpose, whether commercial or social, is key," he concludes.

The work of Theodoros Giannakopoulos and his team exemplifies the intersection of scientific innovation and real-world applications, spanning fields from mental health diagnostics to business optimization. Their challenges highlight the critical need for sustained investment in groundbreaking research. As Giannakopoulos strives to translate his team’s innovations into meaningful solutions, his vision underscores a greater goal: harnessing technology to serve society while cultivating environments that empower scientific discovery.

Τhank you to Dr. Eva (Steiner) Karatairi for additional research and reporting on this article.

I am the Director and Chairman of the Board of the National Centre for Scientific Research “Demokritos” one of the largest research institutions of Greece, driving scientific discovery and technological innovation to create global impact in artificial intelligence, advanced materials, and quantum computing for sectors like energy, telecommunications, environmental and biotech. I am Advisor at EY on Emerging Technologies and Defence Innovation.?