The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed

In August, the RAND Corporation, a well-known research organization, published a paper titled?"The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI."?The paper explores the reasons behind the high failure rates of AI projects and offers strategies for success. Despite the growing investment and interest in AI across various sectors, the paper claims that many AI projects fail to deliver the expected results, with a failure rate significantly higher than that of other IT projects.

While I didn’t find anything revolutionary in the paper, I believe it summarizes the issues well and could be useful for everyone. Before I provide a summary of the publication, I want to comment on some statements made in the paper (read the entire text if interested):

• “80 percent of AI is the dirty work of data engineering.”?– This is true for any data analytics project and is not specific to AI.
• “By some estimates, more than 80 percent of AI projects fail—twice the rate of failure for information technology projects that do not involve AI.”?– It is unlikely that this is a fair comparison. I doubt that anyone has actually done the math. But even if this is true, AI projects are inherently more complex and contain many more unknowns than most other projects.
• “Finally, several interviewees (10 of 50) expressed the belief that rigid interpretations of agile software development processes are a poor fit for AI projects.”?– I completely agree with this. There is a huge difference between software development and data science. Few managers understand this difference and, therefore, expect the same predictability from data science projects as from software development projects.
• As for academia, “even if a technical problem leads to a more promising research agenda, the interviewees noted that the project would still be considered a failure unless it resulted in an immediate publication, such as a conference proceeding or paper.”?– Academia has its own set of problems. According to one source, it is estimated that there are currently more than 30,000 academic journals, and the number continues to increase by about 5%–7% per year. Over 2 million new research articles are published across all fields of study each year. Does the world need that many academic journals and all the papers they publish every year? The answer is likely “no.”

Now, back to the paper and its summary. There are more details in it, but I feel the following is the main message.

The paper identifies five primary root causes of AI project failures:

1.???? Misalignment between Business Needs and AI Solutions:?Often, there is a misunderstanding or miscommunication regarding the problem that AI is intended to solve, leading to the development of solutions that do not align with business needs.

2.???? Insufficient Data Quality and Quantity:?Many organizations lack the necessary data to train effective AI models, or the available data is of poor quality. (AK: It's often cliché to cite 'poor data quality' as a primary reason for project failures. A more precise issue might be that organizations frequently underestimate the volume of data required to train AI models, and these models often fail to capture the right features.)

3.???? Technology-Driven Rather Than Problem-Driven Projects:?Projects often fail because they focus more on using the latest technologies rather than solving real problems.

4.???? Lack of Adequate Infrastructure:?Organizations frequently lack the infrastructure needed to manage data and deploy AI models effectively.

5.???? Overestimating AI’s Capabilities:?AI is sometimes applied to problems that are too complex for the technology, leading to unrealistic expectations and project failure. (AK: I feel this is even more true in the case of GenAI projects.)

Recommendations for Successful AI Projects

To avoid these common pitfalls, the paper provides several recommendations for making AI projects more successful:

·?????? Ensure Clear Understanding of Project Goals:?Both business leaders and technical teams need to clearly understand the project's purpose and the context in which the AI solution will be used. Effective communication between these groups is crucial to aligning the AI project with business objectives.

·?????? Commit to Long-Term Problem Solving:?AI projects require time and patience. Leaders should be prepared to commit to solving a specific problem for an extended period, typically at least a year, to ensure the project has the necessary time to succeed.

·?????? Focus on Solving Problems, Not on Technology:?Successful AI projects prioritize solving specific business problems rather than chasing the latest technological trends. The technology should be viewed as a tool, not an end in itself.

·?????? Invest in Data and Infrastructure:?Up-front investments in data quality, data governance, and infrastructure are essential. This includes ensuring that data pipelines are reliable and that AI models can be deployed and maintained effectively.

·?????? Understand AI’s Limitations:?Leaders should recognize that AI has its limitations and cannot solve every problem. Collaboration with technical experts is necessary to assess whether a problem is suitable for an AI solution and to set realistic expectations.

These recommendations aim to guide organizations in successfully implementing AI projects by focusing on problem-solving, ensuring alignment between technical and business goals, and investing in the necessary infrastructure and data quality.

Again, read the paper if interested in more details.