Validating Steelmaking Prediction Models using SHAP

In steel production, especially in the steel melting shop (SMS), data-driven models are increasingly essential. They help predict key parameters like carbon content, energy consumption, and production yield, optimizing both quality and cost. However, while these machine learning (ML) models offer impressive predictive power, their complexity can make it hard to understand or trust their outputs. As a steel melting shop manager, knowing why a model makes specific predictions is critical to validating and applying its insights. This is where SHAP (SHapley Additive exPlanations) comes in, turning complex predictions into clear, actionable insights.

SHAP is an explainability tool designed to open up the “black box” of AI models, allowing managers to see how each input, like furnace temperature or alloy composition, impacts the prediction. Imagine you’re using a model to predict carbon content in steel after melting, based on variables like temperature, time in the furnace, and slag additives. SHAP breaks down the prediction, showing exactly how much each factor contributed. This makes it possible to validate whether the model is considering the right factors – a crucial step for trust and reliability.

• Understanding the “Why” Behind Predictions: When a model makes a prediction, such as a high carbon content in a steel heat, SHAP allows managers to see which factors led to that outcome. For instance, SHAP might reveal that a high furnace temperature or extended time in the furnace strongly influenced the prediction. By breaking down each prediction, SHAP lets you validate whether the model is basing its output on expected, logical factors, confirming that it aligns with actual steelmaking knowledge.
• Ensuring Key Process Drivers are Accurately Represented: SHAP also helps identify which variables consistently drive predictions. In steel melting, critical factors like furnace temperature, slag additives, and alloy ratios directly affect steel quality. By visualizing the SHAP values, a manager can see if the model is appropriately weighting these factors. For example, if slag additives are shown to have little influence on carbon content predictions in the model but are known to be crucial, it’s a signal to adjust the model or explore possible data issues.
• Detecting and Correcting Model Misalignment: SHAP can highlight when a model relies on unexpected or irrelevant factors. If a model, for example, uses factors like the specific day of the week to predict steel quality (likely an unimportant variable in SMS), SHAP makes this visible. This insight allows managers to question and refine the model, ensuring predictions are based on meaningful process parameters, not statistical noise.

With SHAP, AI models go beyond being just predictive tools; they become reliable partners in decision-making. By using SHAP to validate and interpret model predictions, SMS managers ensure that AI systems are not only powerful but also transparent and trustworthy, paving the way for smarter, more consistent production decisions in the steel industry.

SHAP in Action: Example for Validating Steel Quality Predictions

Consider an example of a basic predictive model trained to estimate carbon content in steel based on:

• Furnace Temperature
• Time in Furnace
• Alloy Composition
• Slag Additives

The SHAP summary plot below shows the contributions of each factor to a specific carbon content prediction, providing a clear view of what drives the model's decisions. Without knowing any technical details about the prediction model, SMS managers can interpret the plot as follows:

• Features with dots positioned further right have a stronger positive impact.
• Those with dots further left have a stronger negative impact.
• Each dot represents an individual prediction’s Shapley value for that feature.

This plot provides transparency into the model’s decisions, helping SMS managers understand which features influence predictions in steel quality and why. They can then apply their years of experience to validate the model output easily.

In this example plot:

• Temperature and furnace time have the strongest positive impact on the carbon content prediction, aligning with expectations for the SMS process.
• Slag additives show a slight reducing effect on carbon content, which may be expected based on process knowledge.

This visualization allows managers to validate that the model’s outputs are grounded in realistic, process-relevant factors, rather than arbitrary correlations.

By using SHAP, steel melting shop managers gain an intuitive view into each model's “thought process.” This transparency makes AI models more accessible, enabling managers to ask questions like:

• Are the right factors being emphasized?
• Does the model’s reasoning align with SMS process knowledge?
• Are there unexpected dependencies or biases that need correcting?

For an SMS manager, SHAP doesn’t just provide a prediction – it offers confidence in that prediction by revealing how it was formed. This empowers the SMS team to use AI tools effectively, making data-driven decisions that improve quality and efficiency across steelmaking operations.

[The views expressed in the blog is author's own views and it does not necessarily reflects the views of his employer JSW Steel ]