How do you tell stakeholders if your data is bad for machine learning?

First of all, use descriptive analytics to understand what aspects of data is bad and build respective visualization. Also do an impact analysis on what other processes & functions are affected by this bad data. While meeting your stakeholders, articulate a couple of things to the stakeholders 1. what a good data looks like versus what is at hand. 2. What can and will happen if you build the ML model using bad data? 3. How bad data is already affecting while it is being used by other team members? Finally, let them know - 1. What is leading to bad data? 2. What corrective actions can be taken and how much time, resources and cost will it take? 3. How much time will it take to accumulate the right data that is required for Machine Learning?

Ask your data science team about what data they want to use in their machine-learning pipelines. If you are already discussing data quality with the data science team, it is because the data is already there, but you don't trust it. So, configure data quality checks on the tables and columns that will be used for training and run the checks to profile the data. Share what you have found. The best way is to share a data quality report with a list of issues. A better solution is to show them a data quality dashboard listing all issues.

A metadata framework would need to be established initially and governance policies and rules extrapolated and applied across the metadata. These rules will certainly utilize accuracy, consistency, cleanliness, timeliness and other inherent data quality measures. Once the underlying data quality measures and deviations are noted and fixed iteratively, underlying fixable data would be corrected and any remaining deviant invalid data separated from the clean learning set. It’s only upon this clean learning set that true machine learning can occur.

Establish quality metrics in your organization to be able to access the usefulness of the data for a given business objective. Once such metrics are established, then you can showcase how the data quality improvements result in a positive impact on machine learning model performance. You can then share these results that demonstrate improved accuracy, reduced bias, and better generalizability, with your business and technology stakeholders. This process will open the doors for periodically engaging with your stakeholders in a collaborative fashion to help drive better insights & business outcomes from high quality data. Listen to the stakeholder concerns and feedback, & work together to develop a plan for improving data quality over time.

Applying Business/domain context is key for every part of the data lifecycle- right from Ingestion, cleansing, pre-processing, seed data population. Without the business context and a proper understanding of the use case, data is meaningless.

In ML Projects, data quality significantly influences model training, algorithm selection, and the overall feasibility of achieving desired outcomes. Strategic challenges can arise when data quality is not aligned with the specific needs of the ML algorithms, which may require highly accurate or specifically formatted data. From a people perspective, a lack of understanding or commitment to maintaining data quality can lead to poorly maintained datasets, which directly impacts the performance and scalability of ML models. Overcoming these issues involves not only technical solutions but also organizational commitment. Ensuring that all team members understand the importance of data quality and its direct impact on ML success is crucial.

In the current data climate, it is almost a given that very few people/teams have trust in their data. That is because data trust has not been prioritized to the level it needs and deserves. One insane way to identify impact is to turn off the data spigot and see who complains/screams ;) Then, work backwards to understand the value and impact of data.

There are different levels of "badness" in data. 1) Are you able to even create a model? 2) Is the model's accuracy in question? 3) Is the model slightly off on precision? These all require different data solutions.

Gauge the impact of the bad data and be transparent about the findings to your audiance. This helps set expectations and gives clarity on what could go wrong.

The quality of data is not merely a factor; it is the foundation of the entire machine learning process. The choice of algorithms, model performance, and the feasibility of the project depend on data quality. Poor data can lead to inaccurate results, affecting everything from model training to real-world applications. Quantifying this impact through metrics such as accuracy, precision, and F1-score is a great way to measure it. It does not only highlight the importance of good data, but also provides a tangible way to track improvements. In machine learning, your model is only as good as the data it acquires from. Invest in quality data, and the results will speak for themselves!

Implicitly, machine learning is an investment in making decisions and finding insights. Your stakeholders should WANT to know if the data is bad. The last thing you want to have happen is to put confidence in a new technology to take you to the promised land and then you fail miserably. Sometimes, we assume that people don't want to hear "this is not optimal" or other negative things. In this particular situation, I want them to understand that my assessment of the issue is sound. I want them to trust me. As such, I shouldn't just say "it's bad". I want to convey if something is inherently fixable and I also want to convey that just because the data is bad for application x, it's not necessarily bad for some future application.

Articulate the impact in factual supportive terms that your audience will understand and appreciate based on their role. You can also use this as an opportunity for shaping the organisation data strategy and investments

Communicating about data quality requires you to be specific. A dashboard with a visual saying customer data is of poor data quality is insufficient. You need to be specific on what customer data has poor data quality. It could be data that's essential to the project, so the project needs to be deferred until the issue can be resolved. Alternatively, it might be data that the project doesn't need at all and the project can go ahead without any issues. You want to make sure your stakeholders go into the project with eyes wide open, knowing what data they can use without any issue and what data to avoid or use with caution.

In my observation trust in data is a major factor in any analysis. Conveying a few things like the timeline of the data, source of data, and data definition, in a clear message and simple language that is easy to interpret will provide more confidence to stakeholders in using the final output.

Build your data observability dashboard on your data - Let data insight provide more insight about data quality issues . Traditionally this is how your business complained about Data quality, because they did not see it in line with what they expect to see.. use the same analogy to provide insight on DQ issues

Establishing a common understanding of data quality objectives and standards is the first step towards a cohesive approach. It's not just about setting benchmarks, but also about achieving them. Involving stakeholders in the process, soliciting their feedback, and making them a part of the solution fosters a sense of ownership and responsibility. This collaborative approach ensures that data quality is not just a concern of the data team, but a shared organizational objective. Regular reporting on progress and outcomes ensures everyone is informed and engaged. Celebrating success is more important than learning from failures. It's these learnings that lead to continuous improvement and improvement.

Present potential solutions, such as data cleaning, feature engineering, or collecting additional data, to address data quality issues. Clearly outline the necessary steps to enhance the data and make it suitable for machine learning. If issues are severe, discuss the feasibility of obtaining new, high-quality data. Set realistic expectations with stakeholders, being transparent about the limitations imposed by current data conditions. Explain that addressing these issues may require additional time and resources but is crucial for building reliable and effective machine learning models.

Ideally, identify the CDE that are required, discuss with the business to identify the business rule and do a profiling of data. Build a report based on the business rules and sample data. Share the report with the business so that they understand the quality. Depending on the business need the DQ score can determine whether to go ahead with the dataset or it needs further cleansing.

Traditionally the raised eyebrow, pursed lips and sucked tooth. But if that doesn't work, just giggling hysterically with your colleagues works a treat. If all else fails, just remember you can string it out for several months or even years before they cotton on, but which time you'll have it on your CV and be ready to move up and out.

Sometimes it's not the quality or accuracy of the data that makes it insufficient for ML algorithms, but the completeness of the data. Are all attributes required to accurately predict a target even available ? For example, I can't expect to consistently and accurately predict your weather this afternoon without knowing your latitude and longitude. Are all the attributes (features) required to predict an outcome (target) available ?

Let’s tell them what needs to be done going forward so that scenarios like the one posted here is less likely to occur. Good Data Quality starts with good user capture techniques. Give someone a free text field to enter attribute x and you’ve just kissed attribute x goodbye for being able to gain insight or metrics from it. This necessitates a data quality assessment of any new or changed data models for an application database.

Effectively conveying data unsuitability for machine learning is vital for transparency. Key steps involve transparently addressing data quality issues, specifying problems, and elucidating their impact on model performance. Educate stakeholders on the pivotal role of data quality, utilize examples for clarity, and propose actionable improvement strategies. Discuss data collection processes and governance, advocate collaboration, and underscore the long-term benefits of investing in data quality. Emphasize setting realistic expectations, highlighting the foundational role of addressing data quality for machine learning success. Note that data generation often relies on hypotheses, and synthetic data may be considered in certain scenarios.