CNN Explanation Technique Based on Contribution-Based Input Sampling and Block-Wise Feature Aggregation

Recently, deep learning models (DNN) have been introduced into various fields such as manufacturing and healthcare, but the 'black box' problem, in which the basis for a model's decisions cannot be known due to numerous parameters and nonlinear operations, has emerged. To address this, research on Explainable AI (XAI), which analyzes which features an AI model used to make its predictions, is becoming increasingly important.

Existing XAI methods have several limitations:

• Backpropagation-based: It is difficult to grasp dependency information between features, resulting in low interpretability.
• CAM-based: These use features from upper layers, but due to low resolution, the resulting heatmap can be blurry or highlight irrelevant features.
• Perturbation-based: Input sampling methods (such as RISE) have very high computational cost and produce random results, while input optimization methods (such as Extremal perturbation) have high computational cost and risk failing to find a global optimum.

To overcome these limitations, LG AI Research has developed a new algorithm called SISE (Semantic Input Sampling for Explanation). This technology aims to identify the weighted feature maps that represent the basis for a DNN's predictions during the image recognition process and visualize them in the form of a heatmap.