Sample codes for feature attribution (a.k.a. saliency map) based on data perturbation.
PertMap (Perturbation-based Saliency Map) highlights pixels in the input image that are relevant to the classification result of machine learning models (mostly, DNN/CNNs).
PertMap uses the size of the allowable data perturbation as the masure of relevance. It seeks the maximally invariant data perturbation that does not change the classification result. The details of the algorithms can be found in the following papers.
-
Linear programming, with linear approximation.
- Satoshi Hara, Kouichi Ikeno, Tasuku Soma, Takanori Maehara. Maximally Invariant Data Perturbation as Explanation. arXiv:1806.07004, 2018 [slide].
-
Stochastic optimization.
- Kouichi Ikeno, Satoshi Hara. Maximizing Invariant Data Perturbation with Stochastic Optimization. arXiv:1807.05077, 2018.
- Python3.x
- Numpy
- Matplotlib
- scikit-image
- Tensorflow
- CPLEX (only for linear programming)
The code also requires the VGG16 weight file in this repository.