By Quanshi Zhang, Ying Nian Wu, and Song-Chun Zhu University of California, Los Angeles
Original article: arxiv.org/abs/1710.00935
Abstract
This paper proposes a method to modify traditional convolutional neural networks (CNNs) into interpretable CNNs to elaborate knowledge representation in high convolutional layer CNNs. In an interpretable CNN, each filter in a highly convolutional layer represents a specific object part (bird head, bird wing, etc.). This method does not require additional labeling and supervision. During the learning process, Interpretable CNN automatically assigns an object part to each filter of high convolutional layers. Experiments show that filters in interpretable CNNs are semantically more meaningful than those in traditional CNNs.
citation
In a traditional CNN, a high-level filter may describe a mixed pattern, i.e. the filter may be activated by both the head and the legs of the cat. This complex representation of highly convolutional layers greatly reduces the interpretability of the network. In contrast, filters in our interpretable CNN are activated by a certain part. In this way, we can explicitly identify which object parts in the CNN are memorized for classification without ambiguity.
Four goals:
1. Slightly modify CNN to improve its interpretability and be widely used in CNNs with different structures.
2. No need for additional labeling, automatically assigning modes to the filter
3. The loss function of the top layer is not changed, and the same training samples as the original CNN are used
4. As an exploratory research, the interpretability design may reduce the recognition ability , we want to limit this drop to a small range.
Method: Calculate the loss for each filter of the high convolutional layer. This kind of loss reduces the entropy of inter-class activation and the entropy of the spatial distribution of neural activation, that is, each filter must encode a separate object part (bird head bird tail ), and the filter must be activated by a single part of the object, not repeatedly in different object regions. Assuming repeated shapes in different regions tends to describe low-level textures (such as color and edges) more than high-level parts.
related work
Network Visualization: Visualization of filters in CNNs is the most straightforward way to explore patterns hidden in neural units. Such as quantifying neuron scores.
Pattern Retrieval: Several studies go beyond passive visualization to actively retrieve specific units from CNNs for different applications. Retrieves the patterns in filter.
Model diagnostics: Many methods have been developed to diagnose the representation of black-box models. LIME, Grad-Cam.
Learning better representations: Instead of pre-trained CNNs for diagnosis and/or visualization, methods have been developed to learn more meaningful representations. For example, adding some rules and labels during the training process.
Algorithm
As shown in the figure above, for an n n filter, design n^2 templates (one template per pixel), and each template is also n n. These templates are used to describe the ideal activation distribution when the target part (bird head and bird tail) triggers the i-th unit in the filter.
In the process of forward propagation, for each input picture, one of n^2 templates is selected to filter out the noise units activated in filter x.
The figure above shows the original image, the mask, and the activation area after submerging the processed image.
In the process of backpropagation, the filter is used to converge to a fixed target mode through loss. In other words, if the picture I belongs to category C, then the filter can be assigned to a certain mask. If it does not belong to category C, it is hoped that the filter will match a negative mask, so there are a total of n^2+1 templates.
We represent the loss of the filter as the interaction information between filter(X) and mask(T). The paper gives a detailed formula for calculating the prior probability.
The fit between filter x and template T is measured by the conditional likelihood p(x|T):
Learning
We train interpretable CNNs in an end-to-end fashion. During the forward propagation, each filter in the CNN passes information from the bottom up, just like a traditional CNN. During backpropagation, the filter of each explainable convolutional layer receives the final task loss gradient and its own filter loss:
We calculate the Lossf gradient of each component on the filter x:
this formula uses an approximation method to reduce the amount of calculation. This formula ensures that when the Image belongs to the target class of the filter, T will be assigned as a template, otherwise it will be assigned to a negative template.
Determine the target category of each filter: which category of graph activates the filter the most, and which category is assigned to the filter.
Understanding Loss
This section analyzes how to ensure low entropy when analyzing the above formula from the classification of filters and the positioning of patterns by filters.
Experiment
The above are part of the interpretability comparison of the filter, and the value represents the amount of semantic information contained in the filter (peer comparison).
Later, the instability of filter positioning is compared with the effect of superimposing the filter mode on the original image (application effect).
Summarize
In this paper, we propose a general approach to modify traditional CNNs to improve their interpretability. In addition to the classification ability, the interpretability of the network is also an important property. We designed a Loss to push the filter to represent the part of the object in the high convolution layer without any annotation. Experiments show that our interpretable CNNs encode more meaningful semantic information in high convolutional layers than conventional CNNs. In future work, we will design new filters to describe discriminative textures for one category, and design new filters for object parts shared by multiple categories to achieve higher model flexibility.