AI’s Impact on Information Curation and Knowledge Acquisition in Scientific Research

Contents for this article were extracted from the book chapter Artificial Intelligence for Curation of Information and Knowledge Acquisition authored by Christopher Farrow, PhD and Alex Chabot-Leclerc, PhD of Enthought. Download the full chapter here .

Advancements in technology have revolutionized materials research and product development. The demand for digital tools and solutions in materials R&D is escalating, driven by the need for rapid innovation in areas like manufacturing, product development, and novel materials discovery. Central to this technological revolution is the efficient curation and acquisition of knowledge.

Researchers today have access to an (over)abundance of digital resources, including public and private document servers, domain-specific databases, and online search services—not to mention their own internal and proprietary data. These tools have significantly broadened the scope of accessible information, enhanced the ease of information retrieval, and improved the speed with which specific data can be found. However, while these technologies have transformed information accessibility, they have not fundamentally changed how researchers process, reason about, and apply this information.?

The advent of artificial intelligence (AI), particularly Generative AI, promises to drive a deeper transformation in how knowledge is curated and acquired.

Curation → Knowledge Acquisition

To understand AI’s transformative potential, it is essential to break down the process through which information evolves into knowledge. This process can be divided into two main stages: “curation” and “knowledge acquisition”.?

In the curation stage, information is collected, organized, and prepared for application to specific research tasks. This involves identifying relevant sources, extracting useful data, and storing it for future retrieval. The knowledge acquisition stage occurs when this curated information is reinterpreted within a specific context, allowing researchers to draw new conclusions and transform data into actionable knowledge.

While many technological solutions currently exist to assist with information curation—most notably, search engines—knowledge acquisition remains a frontier for innovation. AI has the potential to bridge this gap by mimicking human intuition and creativity, enabling more sophisticated information processing and knowledge generation. It can actually enhance both curation and knowledge acquisition, fundamentally reshaping research processes.

Technology-Assisted Curation

Curation—traditionally performed manually—involves selecting and organizing information to make it manageable and relevant for specific tasks. Researchers often begin the process of knowledge building while curating information, interpreting data on the fly. However, this manual curation process has limitations, primarily due to the inherent information exclusion that occurs when only selected data is stored. This exclusion can hinder future knowledge acquisition, as discarded information is not available for re-interpretation in new contexts.

For optimal knowledge acquisition, curation processes should aim to retain as much information as possible, only filtering out irrelevant data during retrieval. In this light, the ideal technology-assisted curation system is one that functions as an advanced search engine, capable of sourcing, extracting, and storing vast amounts of data, which can later be selectively retrieved and reinterpreted as needed.

Search engines are designed to handle vast amounts of digitized information, with physical limitations on storage and retrieval speeds being largely mitigated by modern technology. However, most search engines are text-centric, focusing on retrieving text-based data in response to text-based queries. This search engine approach does not fully accommodate the diverse types of information—such as images, graphs, tables, and molecular formulas—that are crucial in scientific research.

Scientific Search Systems

The next evolution in search technology—what we call scientific search systems— involves incorporating scientific information directly into search queries and results, reducing the need for researchers to manually study non-text data for relevance. This requires a search engine that is context-aware, capable of understanding the scientific significance of various data types.?

Enthought’s technical teams have developed such scientific search systems, designed to integrate and categorize different types of data, enabling researchers to input complex queries that include both text and non-text elements. Enhanced by natural language processing (NLP) and other AI-driven technologies, these systems can interpret and respond to queries in a way that aligns more closely with the researcher's intent, providing a more nuanced and useful set of search results.

Generative AI for Knowledge Acquisition

While search technologies have significantly enhanced the curation of information, Generative AI offers new possibilities for knowledge acquisition. Large Language Models (LLMs) represent a significant advancement in AI's ability to synthesize information and generate new knowledge. These models can answer complex research questions, perform tasks, and even make connections that a human researcher might miss.

Retrieval-Augmented Generation

One of the key techniques that enable LLMs to answer research questions is retrieval-augmented generation (RAG). This approach combines a search engine with an LLM, allowing the AI to generate responses based on up-to-date and accurate information without requiring constant retraining. The search engine retrieves relevant documents based on the user's query, and the LLM then synthesizes this information to provide a coherent answer.

This method allows LLMs to function not just as knowledge bases, but as dynamic tools that can reason about information in context, providing answers that are more relevant and nuanced than those generated by traditional search engines.

Multimodal Embeddings

Beyond text-based information, Generative AI can handle multimodal data, integrating various types of information—such as images, audio, and text—into a single high-dimensional space. This capability enables AI to make connections between different types of data, further enhancing the process of knowledge acquisition. For example, a researcher could input an image of a material's microstructure and receive predictions about its properties, based on the AI's understanding of the relationships between image features and material characteristics.

AI as a Research Assistant

Generative AI models, particularly LLMs, can act as sophisticated research assistants. They can answer complex queries, perform data analysis, and even generate hypotheses or research directions based on the information they process.?

While these AI systems are not infallible and can sometimes make mistakes, they represent a significant step forward in the automation of knowledge acquisition. For instance, domain-specific AI agents, like those developed in the ChemCrow project , can carry out complex research tasks by planning actions, selecting tools, and iterating on their processes until the task is complete. These agents can synthesize information from various sources, perform calculations, and provide detailed research plans, significantly reducing the workload on human researchers.

And AI is getting better and better by the day. OpenAI has just released o1, “a new series of AI models designed to spend more time thinking before they respond. They can reason through complex tasks and solve harder problems than previous models in science, coding, and math.”

Conclusion

Artificial intelligence, particularly in the form of Generative AI and LLMs, is transforming how researchers curate and acquire knowledge. While traditional search tools have enhanced information retrieval, they have not fundamentally changed the way knowledge is created. Generative AI offers new possibilities for synthesizing information and generating new knowledge, making it a powerful tool in the hands of researchers.

As AI technologies continue to evolve, their integration into research processes will only deepen, offering more sophisticated tools for curation, search, and knowledge acquisition. Research of the future will undoubtedly be fundamentally different due to these advancements, as AI becomes an increasingly indispensable partner in the quest for new knowledge and innovation.

Download Artificial Intelligence for Curation of Information and Knowledge Acquisition authored by Christopher Farrow, PhD and Alex Chabot-Leclerc, PhD of Enthought.