How can you make your statistical models more interpretable?

Feature selection, even via stepwise methods in linear regression, can simplify models but risks misleading results. It may cause loss of context, overemphasis on certain features, non-intuitive feature interactions, and complexity from feature transformations. Specifically, in stepwise regression, a data snooping effect can overestimate variable significance, multicollinearity might distort importance, model instability can arise, p-values may inflate due to multiple tests, and coefficients can become biased. Therefore, while useful for exploratory analysis, such methods require cautious interpretation and often need additional validation techniques.

The easiest way to achieve interpretability is to?use only a subset of algorithms that create interpretable models. Linear regression, logistic regression and the decision tree are commonly used interpretable models. By?reducing the number of features, like principal component analysis (PCA) you can avoid overfitting, improve performance, and make your model easier to interpret. By creating new features, you can capture important relationships, interactions, or patterns that can enhance your model's predictive power and explainability.

In Machine learning, we use different ML algorithms for drawing decision boundary in hyper-plane. Hyper-plane dimensions is decided on the basis of number of features. Decision boundary is identified on the basis of features, their interaction with each and other and with outcome. Feature selection techniques are used for selecting fewer features which are important. We can analyze interaction between the features and outcome; their correlations, if there are fewer features. It is easy to visualize few important features in a graph. Also, it removes features which are not important and doesn't play any significant role in ML. On the other hand, feature engineering steps are used to normalize data and bring all features on same scale.

Feature selection is one of the key concepts in interpretability. But to a certain degree of interpretability. For example you may use feature importance based on Gini or accuracy decrease to select important features. But after that you may need simple models such as logistic regression, linear regression, Poisson regression, cox ph regression or others to simplify the model with those selected variables. Also select features based on their interaction and adjustment potential too. Model goodness of fit, AIC and other criterions may be used too.

Achieving interpretability involves strategic feature selection. Metrics like accuracy decrease help identify key features, followed by simplification using models such as logistic regression or linear regression. Incorporating domain expertise is crucial, preventing challenges with highly correlated variables. Emphasizing easy-to-understand and maintain features, especially those with lasting predictive power, enhances ongoing model efficacy.

These techniques could be tricky, always rely on an statistical test over each parameter that backs-up them. Tests should also be included in the overall documentation of the model, if not someone might ask why a parameter was taken out.

In my view, achieving a "correct interpretation" stands out as the foremost objective. Relying solely on the interpretations of others, particularly in cases involving interactions in nonlinear models like logit and probit or various data transformations such as normalization, standardization, log transformation, or sinus hyperbolic, may not be advisable. Utilizing visualizations stands as the most impactful method for presenting the outcomes of statistical models. However, I maintain the perspective that while visualization certainly enhances comprehension of the model's operations, it doesn't equate to full-fledged "interpretation."

Se o público que irá receber sua análise n?o é especializado em dados, importante ter em mente que os gráficos aqui descritos n?o s?o facilmente interpretáveis. Lembre-se de criar o gráfico mais simples possível, explicá-lo com textos rápidos e usar de técnicas de visualiza??o de dados para destacar o que importa no gráfico.

Meaningful graphics highlight data patterns more than a table of numbers and stats. Keep in mind also to label with legends and appropriate axes

Lots of us humans are visual learners. Therefore tables and graphics are excellent tools for your "message" to reach the target audience. Chosing the right visualization depends clearly on the question you want to answer, the data you have, the models you used and so on.

Good visualization is key for the interpretation and deeper understanding of the results. At times, common visualization approaches will be easier to comprehend from newer approaches that might objectively provide a better view of the data. Make sure to balance well your ambitions to use the best visualization with your ambitions to provide familiar visualization that's easy to get by more people.

I would add that LIME and SHAP provide good estimates for Feature Importance but they can only be used to find correlated features to the target variable, no causal relationship. There's an interesting library called CXPlain which is trained on a causal objective and can quantify a prediction's improvement caused by adding/removing a feature.

leveraging Explainable AI (XAI) methods like LIME and SHAP is crucial for understanding and trusting complex AI models. These tools provide interpretable insights into model decisions, helping identify potential biases and uncertainties. For instance, LIME simplifies complex models for specific instances, while SHAP quantifies feature contributions based on game theory. Counterfactual explanations and Anchors further enhance interpretability by generating alternative scenarios and pinpointing crucial features for predictions. Incorporating XAI methods not only enhances model transparency but also aids in addressing challenges related to model complexity and opaqueness.

Best explainability-accuracy trade-off model and framework I find is Microsoft's GLM based explainable tree model and the surrounding interactive GAMChanger framework. Surprisingly it has not been used extensively in the community. Each statistician should have a look.

SAS modelops has this capability. It makes models explainable to both statisticians and business users. This has huge implications with regards to understanding models, and keeping them relevant

The definition of 'interpretable' is quite ambiguous here so I'll go with a wide variety of interpretations: 1) Try to keep n below 3 in n-way interactions for regression, or if you have to have a 3-way interaction, consider stratifying by one of the 3 variables for greater ease of interpretation. 2) Be very careful when interpreting and explaining logistic regression and survival analysis coefficients. Logistic regression coefficients are often misinterpreted. 3) Variable importances in tree-based models really do not indicate the variable's importance as a soloist but as a team player if you scrutinize what they actually are. Yet they are interpreted an overwhelming majority of the time as the marginal contributions of a solo variable

In my experience there is a golden rule for modelling: always start considering an easy model, or the one which is more 'interpretable', before moving to a more complex model. During my first years of professional experience, I used to go directly to complex parameters so I could demonstrate my knowledge, but it only caused that my work was hard to understand. Instead of spending hours or days explaining your work, start with simple things, sometimes these are good enough for the work.

I think we need to distinguish two categories of models: - models that will help you understand how a system works so that you can implement some changes (i.e, policy related) - models that will just speed up some task, but you do not really want to understand how a system works. What discussed in this article fits the second category, while a family of models called agent-based models fits the first. The two approaches can be highly interconnected (i.e., an ABM can generate unbiased data for ML; AI can be used to identify patterns in ABM results), and I personally think exploring this route will bring tremendous benefits to humanity in the nearby future.

"All models are wrong, but some are useful" is often attributed to the statistician George Box. Models are simplifications of reality, and so, inherently flawed, but they can still provide valuable insights. This statement underscores the importance of understanding a model's limitations and not relying on them blindly. "Art is a lie that makes us realize truth," a quote attributed to Pablo Picasso, suggests that while art may not always be a literal representation of reality, it conveys deeper truths and insights about life, emotions, and the human condition. Artists convey aspects of reality and human experience that may not be easily articulated through direct, literal means. "Models are lies that help us see the truth" - Me

Generally picking a good measurement is the best practice. For example, if one is concerned with, say fraudulent activity, Pick a metric that allows false positives, but minimizes false negatives. The integral of ROC may not be the best measure, maybe a better look is the precision recall curve, or simple minimizing the antitrace of the confusion matrix. In interpretation, explain why a particular metric has been chosen and what the various outcomes mean.