An approach to leveraging AI/ML

As the saying goes, don't use a cannon to kill a fly! AI is not a solution to every problem. As always, focus on understanding the problem, the tasks, the opportunity, and the impact, which has to be measurable and incremental as AI comes with a cost.

Start with heuristics and get a baseline.

Generally speaking, it is always better to start with heuristics, as it can be implemented quickly (1-2 sprints) and require a smaller investment. Once it starts showing the impact, smaller incremental updates can be made to optimize, leading to good baseline results.

Use a simple AI model to start.

Following a good baseline, start with a simple and interpretable machine learning algorithm to establish the first ML baseline. Use it to compare with the baseline collected with the heuristic model. Use this approach to evolve the model slowly.

Heuristics vs. AI model and leveraging dark mode testing.

Now, a great way to test your models in production without impacting your business is by testing the algo in dark mode (in production). Fire the ML algorithm in the background for every action where heuristics are used and save both the heuristic-generated and ML results. Over the next few days/months, this data will give you insight into whether the ML model is indeed effective and meaningful.

Feel good, you found a winner!

Once you see a sustained and consistent positive performance of the ML model over heuristics, you may have your winner.

Wait, what did it cost?

Now that you have a successful model, before you deploy it, add up the cost of people, software, computing, and data collection to understand the cost vs. incremental gain as you move forward.

Now, deploy, celebrate, and watch results like a hawk!

Now, deploy your winner to production. Remember, the journey from heuristics to finding the first successful AI model using dark mode can take months. Bring stakeholders along to explain the journey and why this approach is paramount. Share progress with stakeholders every week as you generate data in dark mode. Transparency and communication through this process are critical to the support you will need through this journey.

Find the right mentors to guide you through this journey.

Find the gurus, discuss what you are doing, and take feedback. Speaking to people who have been through this journey can open your eyes to new ideas, reduce the pain, and avoid cost!!

Let me know if you like this and want more. I would love to get your perspective.

Big thanks to Mark Gillett and Valliappa Lakshmanan as I started my journey. I would not have been able to learn these lessons without their mentorship.