Why AI Innovation Projects Fail in Industrial Company?

Hi everyone!

Exciting to see so many of you interested in this newsletter, I wasn't expecting this kind of reach!

For this week's issue, I want to dive into one of the topics that is in my opinion most lethal to any digitization effort within industrial or manufacturing companies.

The AI innovation projects (and why they fail).

I've consulted and helped quite a few industrial companies in some kind of AI-related project they already started.

Sadly, most of these AI innovation projects were doomed to fail and have most likely killed all motivations for new future ones. I invariably got brought in too late.

By reaching some decision makers out there with this post, my hope is that some of these AI innovation projects get corrected before getting launched.

Let's dive in!

Main Issue with AI Innovation Project in Industrial Companies

The main problem I've seen with industrial companies having an AI project running is that the project should never have been started.

There was simply not enough data of the right kind to green light it in any form.

Yet, unlocking some budget and starting the procedure to get the project going required higher management involvement. In the industrial sector, this isn't the kind of random project getting started by a team with too much time on their hands.

Meaning that people steering the company decided that this project should be prioritized for X or Y reasons.

I guarantee you that trying to fix up a data-problem 6-12 months down the line is not going to end well. At best it requires more data collection (i.e. more money into that project) at worst you have to kill it (i.e. sunk cost fallacy kicks in pretty hard at that point).

At the surface level all of the movements looked great:

• Management hired a consultant to make an ROI assessment.
• They looked at the competitors and what the tech companies were doing.
• They read plenty of white papers about why this kind of project would be the next best thing for their company.
• They had a copious amount of meetings to scope the project.

The only thing missing was a very hard look at their own data they were gathering.

Ironically, if there is no link between the data you have and the desired result everything else is of little use.

Furthermore, not all data are created equal. Meaning that even if you ask your team "do we have lots of data?", they might say yes. What your team thinks is a lush data forest might in practice be a data barren wasteland with very little information.

Even more so, not all AI projects require the same amount of data. For some projects a few thousand data points will be more than enough and others will not be feasible even with millions of training data points.

This high failure rate of AI innovation projects in industrial companies compared to tech companies is mainly driven by the fact that higher management might not have enough data background or support.

This is highly understandable because their deep expertise is in another domain (like hardware or engineering).

The point remains, AI innovations can 100% benefit industrial companies and should be attempted. However, higher management needs to understand that they may lack enough expertise to even check if their project is feasible.

So how can you check if you have the right kind of data to start that project without deep expertise in data science?

Framework to Check if the Project is Feasible - The Excel Test

Here is my very simple pragmatic framework to check if an AI project is even feasible. I've already discussed it in a separate LinkedIn post, but I'll detail it more over here.

The core process is simple, ask your team (IT or whoever has access to your databases) this simple question:

"Could you generate me an Excel spreadsheet with each row being one instance of our training data for this AI project? We will use this to evaluate the feasibility."

This is powerful because you will be able to answer all questions you need to check if the project should even start and where you should invest in your data stack before even doing an AI project.

I also strongly advise to get an independent data scientist with a data engineering background in the loop as a consultant for this exercise (If you need contacts hit me up).

The answers you might get are the following:

?? It's impossible to generate as the data can't be fit into rows in an Excel spreadsheet!

Don't green light this project! Even images and text can be fit into rows (i.e. each pixel in the image being in a column and each word being one-hot-encoded).

Your issue might lie into what kind of data you thought you had. For instance, if you have thousands of PDF or Word documents and you labeled them as "data" you will quickly see that they are next to useless for your project.

These formats are notoriously difficult to extract information from, especially if they always change (The two words found in a project brief most feared by data scientists are "OCR" and "PDFs").

At this point, you should first and foremost invest more into data standardization or data collection. The return on investment will be much higher than forcing a tortuous AI project to a cliff edge.

Once that basic plumbing will be put in place and piping that data into an Excel format becomes feasible, think back about the project.

?? It is taking a long time to generate

If the team took a very long time to generate the data, you have a big red flag that there is too much pre-processing involved.

To verify that, ask your team to generate 10X more data they output. If they all groan at the idea, you have a data-engineering issue!

This might happen for multiple reasons:

• very hard to access the data for security reasons.
• data is all scattered everywhere and needs to be hunted for.
• The formats are a mess to work with and your team cleaned them manually.
• Your organization litterally have little data being collected on that front.

Instead of the AI innovation project, invest in figuring out how to store, transform and manipulate the data for the department impacted by your project. (hint: This is where your independent Data Science consultant will be useful).

You might also have to review some security protocol in order to enable digitalization at your organization. This might look like a mild improvement, but any data scientist that had to sit on their hands for 3 months before they got their data will love the gesture.

?? Data is generated, but people are confused as to what to put in the rows and are arguing

Don't green light this project yet.

Ensure that you have clarity about what you are trying to do with this AI project with your team. What are the core features and what is the end label to predict?

This might sound silly, but some projects just can't work with the data collected. Some critical feature or label is missing making the rest of the data collected nice, but not enough.

This is one of the most problematic situations. At a first glance, your project looks feasible because you have "lots of data ??". Everyone is excited and your project is moving fast at the beginning.

Then inevitably, when you get to a modeling phase, nothing will come out of it. The people that worked hard to get that data will start to blame the poor intern working on the case with insightful tips like "try harder".

The best thing to do early on is to get that data in front of you, get your team in a meeting with your data science consultant. They will be able to assess exactly what is minimally missing and your team will be able to tell exactly how difficult it will be to get it.

At this point, making a call to greenlight the project will be much simpler.

? Your team is asking you where you want the data and if they can split it into multiple file (also they put it in .csv)

You are on the right track!

If this happens it means that you have lots of data, your team was able to manipulate the raw input properly to generate the training data and that they know what the project is about.

You still should verify if you have the right sort of end label, but the signs are good for green lighting it.

If it's your first AI innovation project, I would advise getting a second pair of eyes on the case just to validate the probability of success!

?About getting internal talent

Getting internal talent to run a full AI innovation project might not be the best when you are just starting out in your digitalization journey.

Since these talents are currently in high demand, having one coming in for a project that is doomed to fail is not a great first impression or good use of resources.

I would advise to get the right talent to instead ensure that your data piping and tooling is properly set up. This will allow your organization to gather more data of the right kind and enable the future AI innovation projects.

It will also build the foundation for your data-driven culture that you should put in place as soon as possible.

You Have an AI Innovation Project in Mind?

Well then reach out to me before you start!

I'm always happy to give some feedback on a project and put it on the right track (anything to give that poor data science intern some breathing room)!

Let's continue the discussion in the comment and if you have data war stories to recount I'll love to hear them!

Have a great week! ??

Yacine Mahdid