When it comes to implementing AI solutions, you can do better than fitting a square peg in a round hole.

Are you struggling to get the best out of the data product you purchased? you are not alone.

So you wanted to make use of the latest technological leaps in the field of machine learning, big data, and all those good things happened in the last decade. You found the sweet spot business problem to solve, spent couple of millions or more to to purchase the market leading platform/product. The honeymoon is over, and now you are not quite sure about the solution you purchased. Something tells that the "off the shelf product" is not fitting quite well. It works, but it could have been working better, had it been using some additional data which is specific to your business. It could have been working better, had the product been making use of the secret sauce business understanding specific to your company. You are not alone, read on..

Computing for process automation vs computing for data insights

If you look at, how computing have been helping businesses (starting int the early 80s), you will see that, the journey started with the use of computing to help in automating business processes. This was the first wave. In the first wave, companies have been producing data as a result of process automation. Data oriented systems such as DBMS were helping to collect, organize and integrate the data into the automation process. Then came the second wave, where companies started to gain value out of the data generated as part of the first wave. This was largely helped by the technological advances happened since early 2000s, in the field of BIgData, Machine Learning, etc.

By definition, processes are logical flows which naturally fits into computing world. So all you have to do is, to mimic the process into computing, provide hooks for extension/alteration of the process and you are ready to automate. That is why in every business vertical there is a market leader in process automation, and they do very well.

Why tech companies are interested in "platformizing" a solution?

Because it is in their interest to write the solution once and sell it to many customers. Tech companies want to follow the "write once, sell anywhere" mantra. It suits to their bottom line. This idea is not directly in line with their customer's interest, which is to solve the business problem at hand (with a platform or without a platform). Customers have not been complaining, because traditionally, "write once, sell anywhere" approach has been working reasonably well. This is because it is not difficult to provide a generalized platform solution which automates a business process. For example, a hotel reservation system, travel management system, billing management system, etc works across customers and industries. Yes, the customizations are still done on those products. Those customizations, are process customizations. Product companies build those customization hooks in to the solutions (for example SAP) and allow customers to do customizations within a general process framework. And the whole echo system worked well. Tech companies made profits, end customers did not have to complain. Every one is happy.

Automating a process vs automating data insight generation / decision making.

There is a fundamental difference in trying to have a generic solution for data insight generation vs a generic solution for process automation.

In general traditional "off the shelf" platforms starts from a business process and automates it. Data is an outcome of the platform. But solutions which are generating insights from data starts from data and generates additional data to enable a decision making process. Here data is not a byproduct, but it is the seed and meat of the solution.

Why off the shelf data products do not work best?

Because semantic meaning of the data and the context of the data changes from business to business. Even the very notion of customer and the behavior and context of the customer changes from business to business. When you develop a generic AI solution, you create an algorithm which works against an abstract notion of the context of the data. For example, let us say, there is an algorithm which predicts customer journey. What would be a customer journey in an apparel industry will be different from say food & restaurant industry. The parameters which governs the customer behavior will also change. Even in the same industry segment the meaning and context of data changes. In addition to this, there will be specific business knowledge and USPs which makes the business run, in the segment they are in. The generic AI algorithm will have to take these into consideration to be successful. Off the shelf products fails to incorporate these differences into their product. Unlike process automation oriented platforms, merely providing a hook for customization is not enough. Because there are no hooks into data. You can not alter it, you can not extend it. Data is what it is. Often it is dirty, incomplete, contextual and tied to the business domain. AI companies are struggling with this tension. Providing a customized solution vs providing a platform. While providing the customized solution is best for the interest of their customers, it is not in the best interest of the company as it is against the "write once, sell anywhere" notion.

Below is a Glassdoor review from an employee of a silicon valley company who provides a platform for understanding customer preferences, which illustrates this tension. I cropped out the name of the company to protect its privacy.

Opportunities for "platformizing" AI solutions

There are still avenues for providing platforms for AI solutions. This is mostly around providing algorithmic building blocks towards the final solution, before the solution touches the business specific data. Examples are face detection, NLP, speech synthesis, etc. AI players such as Microsoft, Google, Amazon, etc provides these solutions which can be used as building blocks for the final solution. In short, problems , which do not touch the business and the contextual data, is a good candidate to be solved using a generic platform.

Does that mean, all those AI companies which are trying to solve a business problem as a platform solution is doomed? Absolutely not. What is required is a mix of consulting and platform approach. Consulting is to understand the business and the data of the customer. Platform should be coarse set of loosely connected algorithms, which can be configured, re-wired and fed with data. Here platform is for the sake of not reinventing the wheel on the basic solution approach. The difference should be in the way AI solution provider approach the customer to help them. The difference is in asking "buy my stuff" vs "let me help to solve your problem". Because at the end of the day, businesses needs to solve their business problem, be it with using AI or not using AI. Overtly focussing on a generic platform solution, should not be at the cost of providing less than best solution to the success of the customer.

Epilogue

During my career, three times, I implemented or were part of a team which implemented a custom in-house data solution, which eventually replaced or intended to replace an "off the shelf" platform.