How do you communicate machine learning data and information to non-technical stakeholders?

My communication strategy centers around explaining in layman's terms how a neural network or algorithm works, and detailing how we go from model to outcome. For example, with large language models, I explain how they're able to form associations (spatially, like how the words "rain" and "umbrella" would be close together in high-dimensional space) between words based on patterns they find in text data, and so when a prompt is entered, the algorithm is spitting out a prediction of what the most likely answer or response to that question would be. I always think explaining things conceptually to a non-technical audience is best; just break it down into a series of digestible steps and most people will get it!

My own communication strategy is to not give the end audience more than what they need to know. Some stakeholders or business users will be much more engaged in the technical details, but that’s typically not the case. The value of the model outcome is from the value it brings to the business! Much like how models contain layers, so do the end-users of the model results. I think about how to show high-level results first with the key metrics or numbers. From there, end-users can drill down into more details (time series or categorical summaries for example). Finally, there are more granular views that might appeal to the most technical audience or a specific segment of business users.

Sometimes you won’t know this information prior to actually talking to your intended target audience - imagine bumping into a decision maker or team leader in the hallway at the office for example. In these situations - aim to be engaging and avoid diving into details without first gauging if your improptu audience is interested or has time to talk further. Personally I’ve found that simple analogies work great to explain relatively complex topics but it takes some work and experience to come up with these, even if you try to prepare them in advance. My tip for coming up with better analogies is to get your repetitions in and keep trying to improve your story telling skills and how you can connect the dots for a wide range of audiences.

Drawing from my experience teaching machine learning to high schoolers, here's a tip: steer clear of the complex stuff. Focus on the 'why' and 'how' of the model, the data it uses, and the outcomes it generates. Use everyday examples, like Netflix recommendations, to explain concepts. Remember to talk about how this can positively impact a business, say, through increased sales or improved customer service. This not only highlights the practical side of machine learning but also helps to build trust in its recommendations. Of course you have to be much more intricate based on the audience but the key thing is to generate interest in subject then followed by explaining the specifics. Employing Feynman technique here would be a good idea.

Machine Learning , like any other tool has different perspectives. Knowing the audience type enables an Applied scientist to present the most suitable perspective to him/her. A data scientist would be more interested in knowing the model being used, parameters landscape and tuning startegies etc. A Software Engineer would like to know the inferencing capabilities and non-functional strategies to optimize the cost and latency. A Machine Learning Engineer would like to know about data engineering, model effecicncy or model compression techniques, pruning etc. A business stakeholder or Senior leader would like to know about the business impact, feasibility and profitability of the Machine Learning solution.

Asking the right questions to key decision makers, end users and the wider team who will help you implement the final solution is super critical in this stage. Highlighting value is definitely recommended, but it is important to emphasize the areas which might be more aligned to a stakeholders interest and incentives to help gain their trust and build momentum. Sometimes the solution phase will not be very clear at the start of the project and you will need to take this as a sign to keep asking more questions about the business problem. How will success events be measured? What time frame do we need to action the insights? What sources of data are available at training and serving time? What data lags are there? The list goes on!

There are many ways of conveying machine learning or deep technical concepts to non-technical stakeholders. In my experience, I have found three simple techniques to achieve this effectively. 1. Storytelling: Transform complex data into compelling, memorable narratives—Utilise real-world scenarios to demonstrate the impact of your work. 2. Simplify Language: Avoid technical jargon and present information clearly and clearly. Think how you would explain this to a child. 3. Demonstrate Impact: Highlight real-world applications and potential ROI. This will solidify the value of your work and make the benefits tangible to your audience. These simple techniques executed well will help build trust and advocacy for your machine-learning projects.

A machine learning solution can best be described with data points, statistical techniques or visualtizations etc. Introduce the problem and do deep-dive using the facts and figures. Continue the story-telling and then arrive at the solution. As famously said Machine learning is both science and arts so is the explanation.

We often have a saying that is relevant to any data and machine learning problem - “Garbage In, Garbage Out” This is definitely the case when we start exploring our data and analyzing it for the right starting place to build any machine learning solution. Feature engineering is going to be very helpful - as will making sure that you have the correct target variable that doesn’t exhibit any artificial selection bias or sampling skew that deviates from the same population you will use at model serving time. Likely, supervised machine learning labels or target variables are not always directly available in the input data - you have to create this yourself with whatever tools you have on hand. Statistics knowledge is key here!

While the simplification is important for effective communication, not losing the relevance is paramount in geting the RIGHT message across. As Sir David Spiegelhalter brilliantly put it, for example, Brexit supposed to save 350 million pounds per week EU costed UK. However, considering the UK population size, it translates to only about 80 pences a day per person; i.e., a packet of crisps. The moral of story is that one should present stats that are relevant, since stats can easily be misrepresented and misinterpreted.

Absolutely! When it comes to visualizing LLMs, I use Streamlit and/or Gradio. They have revolutionized how we demonstrate and interact with language models. Their rise in platforms like Huggingface spaces underscores their effectiveness in simplifying and visualizing complex ML outputs. They not only streamline the process of turning data science scripts into shareable web apps but also enhance the experience for the end-users. I've also found them invaluable for client presentations, as they allow for real-time interaction and tangible demonstration of model capabilities.

Feature importance and any explainability tools such as SHAP and LIME will help to build trust in your model - especially when you are using features which still have some business logic or understanding baked in. Model evaluation scores can vary from problem to problem and even which specific f1, sensitivity, specificity and recall metrics should be optimised is also defined by the actual problem statement the ML model was designed to solve in the first place. Model optimization is increasingly becoming automated where you can leverage frameworks or packages to automatically find you the best model based off your certain optimization metric such as loss, recall etc. Note that there is a time/cost axis to how much you should tinker!

It’s important to communicate with colleagues on both the technical and business stakeholder fronts. Attend meetings of the business end users for a model to present it to them. In those meetings solicit and receive feedback on what works and doesn’t work for how they perceive the model to add value to the business.

Communication is a two-way street. As much as you need to communicate about data and results, you also need to listen for what your business stakeholders are saying, asking, and understanding. There’s something to learn about each other’s domain if you’re willing to look for it…

Another key consideration is managing client expectations effectively. It's vital to communicate the intricacies of model accuracy, the significance of training data size, and the ongoing nature of fine-tuning and model updates. Often, clients have lofty expectations of AI capabilities and may not fully grasp the intricacies and effort required for production-ready projects. By keeping them informed and engaged, we can ensure a more realistic and satisfactory collaboration.

Know the landscape and the stakeholders. Drive your metrics towards their interest. Sometimes 99% of accuracy don't tell anything to them, but rewiring metrics to a business logic framework, how much money can be saved or earned is a powerful stand against these situations.