What are the methods to interpret the output of machine learning methods?

The methods of interpretability of machine learning methods can be classified into the following groups:

Post-hoc Interpretability

Post-hoc interpretability refers to the interpretability of models after they have been successfully trained. Here we include a list approaches by their overarching logic.

• Feature Analysis

  Accumulated Local Effects (ALE) aims to explain the impact of features on the outcome in average.

  Feature Interaction aims to explain how the interaction of features impact the model output.

  Permutation Feature Importance (PFI) evaluates the impact of feature on the output via random permutation of particular feature input.

• Model Inspection

  Scoped Rules (Anchors) aims to arrive at anchoring rules unaffected by changes in other features.

• Saliency Analysis

  Partial Dependence Plot (PDP) visualizes the average dependence between the target and a set of features.

  Individual Condition Expectation (ICE) visualizes the sample based dependence between the target and a set of features.

  Shapley Additive Explanations (SHAP) aims to distribute the total outcome as individual feature contributions.

• Model Simplification and Surrogate Models

  Global Surrogate is the approximation of the model globally via a simpler interpretable model.

  Local Surrogate(LIME) is the approximation of the model locally via a simpler interpretable model to explain an individual prediction.

• Mathematical Modeling

  These approaches building on some assumptions, formulate a mathematical framework to explain model outcomes.

• Explanation by Example

  Illustrate the workings of the model by studying representative samples and their corresponding output.

• Explanation Generation

  These methods aim to generate symbols or words explaining the inner working of the models.

Intrinsic Interpratability

Intrinsic interpretability refers to the interpretability that is built into the model. Two major approaches have been employed in intrinsic interpretability.

• Interpretable Representation

  A set of favorable properties such as monotonicity or sparsity are imposed on the model through regularization to arrive at more interpretable representations.

• Interpretable Architecture

  The architecture of the model is designed to increase its interpretability.

References

• Molnar, Christoph. “A guide for making black box models explainable.” URL: https://christophm. github. io/interpretable-ml-book (2020).

• Fan, Fenglei, Jinjun Xiong, and Ge Wang. “On interpretability of artificial neural networks.” arXiv preprint arXiv:2001.02522 (2020).

• Arrieta, Alejandro Barredo, et al. “Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI.” Information Fusion 58 (2020): 82-115.

• Du, Mengnan, Ninghao Liu, and Xia Hu. “Techniques for interpretable machine learning.” Communications of the ACM 63.1 (2019): 68-77.

• Gilpin, Leilani H., et al. “Explaining explanations: An overview of interpretability of machine learning.” 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 2018.

• Guidotti, Riccardo, et al. “A survey of methods for explaining black box models.” ACM computing surveys (CSUR) 51.5 (2018): 1-42.

• Adadi, Amina, and Mohammed Berrada. “Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI).” IEEE Access 6 (2018): 52138-52160.

• Zhang, Quanshi, and Song-Chun Zhu. “Visual Interpretability for Deep Learning: a Survey.” arXiv preprint arXiv:1802.00614 (2018).

• Lipton, Zachary C. “The mythos of model interpretability.” Queue 16.3 (2018): 31-57.

• Chakraborty, Supriyo, et al. “Interpretability of deep learning models: a survey of results.” 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). IEEE, 2017.