Study shows low efficiency of AI in finding accurate sources of information
Vitaliy Khomyak
I help tech companies with tailored software solutions and share expertise across numerous fields (too many to list!). As a tech enthusiast, I aim to keep LinkedIn exciting by sharing fresh, interesting news daily.
The Tow Center for Digital Journalism, which studies modern media, has expressed doubts about artificial intelligence's ability to search for information on the Internet effectively. As part of the study, scientists tested the accuracy of various AI-based chatbot models.
The experiment selected 200 articles from 20 different sources. The AI was tasked with identifying the correct article, edition, and year of publication, using exact citations from each piece as clues.
The results were not so encouraging: the AI was able to correctly identify the source of the quote only 60% of the time. In comparison, a manual search found the source 100% of the time.
Two versions of X's Grok AI performed particularly poorly. Grok-3 Search made errors 94% of the time, while Microsoft's Copilot failed to process 104 out of 200 queries, and of the remaining ones, only 16 were correct.
Even the paid versions of the chatbots showed only a slight improvement in search quality compared to the free ones. The AI mostly either couldn’t find the exact source or provided incorrect links. The DeepSeek model incorrectly identified authorship in 115 out of 200 cases.
Against this backdrop, Microsoft announced the launch of new advertising formats aimed at users of the Copilot AI service.