Table of contents
2. Why do we need interpretability?
3. What interpretability methods are available
1. What is interpretability
0x1: Generalized Interpretability
Interpretability in a broad sense refers to:
When we need to understand or solve a thing, we can get enough understandable information that we need.
For example, when we debug a bug, we need to locate the problem through variable review and log information.
For example, when facing a new problem in scientific research, we need to consult some information to understand the basic concept and research status of this new problem, so as to obtain a correct understanding of the research direction.
Conversely, if we can't get enough information in some situations, then these things are inexplicable to us. For example, the question "What is the purpose of the universe" raised by Hawking in Liu Cixin's short story "Words of the Morning" immediately stuck omniscient adventurers, because no advanced civilization can understand and grasp the Creator's creation of the universe. These ultimate questions will always remain inexplicable to us.
0x2: Interpretability in Statistical Learning
In traditional statistics, we investigate large amounts of data to construct and test hypotheses. We build models to build rules that we can incorporate into our models.
For example, a marketing firm can build a model that correlates marketing campaign data with financial data to determine what constitutes an effective campaign. This is a top-down data science approach where interpretability is key as it is the cornerstone of the defined rules and processes. Since correlation often does not equal causation, a strong understanding of the model is required for decision-making and interpretation.
As data analytics practitioners, we are often concerned with fine-tuning models for optimal performance.
Data science is usually defined as: 'Given data with X labels, find a model with minimum error'.
While the ability to train high-performance models is a critical skill for a data scientist, it's important to be able to see the bigger picture.
Interpretability of data and machine learning models is one of the crucial aspects in the "usefulness" of data science. It ensures that the model is consistent with the problem you want to solve. Being able to correctly explain your findings is data science. important part of the process.
0x3: Interpretability in the field of machine learning
1. Decision tree is currently recognized as the best interpretable non-linear machine learning algorithm in the industry.
Specifically in the field of machine learning, take the most user-friendly decision tree model as an example, each model Making a decision will go through a decision sequence to show us the decision basis of the model:
For example, the condition of male & unmarried & doctor & bald corresponds to the decision of "not interested".
Moreover, the screening variable criteria based on information theory that comes with the decision tree model also helps us understand which variables play a significant role in the process of model decision making.
So to a certain extent, we think that the decision tree model is a model with better interpretability.
2. The interpretability of the DNN deep learning model is not so obvious
. Taking the most user-unfriendly multi-layer neural network model as an example, what is the basis for the decision-making of the model?
Like 1/(e^-(2*1/(e^(-(2*x+y))+1) + 3*1/(e^(-(8*x+5*y) )+1))+1) is greater than 0.5 as the standard (this is already the simplest model structure), the superposition formula of this series of nonlinear functions makes it difficult to directly understand the "brain circuit" of the neural network, so the deep neural network Network habit is considered to be a black box model.
The human brain is not good at understanding nonlinear relationships, let alone complex nonlinear combinations with layers of combinations.
Therefore, a natural thinking strategy is to find a low-order approximate mapping with lower dimensions (that is, to find a low-dimensional representation method), so that the premise of not losing too much of the original high-order decision-making hypersurface space distribution Next, find a low-order decision surface. In this way, human beings can get a glimpse of the original high-order decision hypersurface from the low-order decision surface.
The author's thinking: According to this idea, we can actually see that the visualization after t-SNE and PCA dimension reduction is a low-order approximate mapping.
2. Why do we need interpretability?
In a broad sense, our need for interpretability of deep learning models mainly comes from insufficient understanding of problems and tasks.
Although the high degree of nonlinearity endows the multilayer neural network with extremely high model representation capabilities, it can achieve very gratifying performance on many problems with some appropriate parameter tuning techniques.
An uncomfortable problem is that although we have built a machine with extremely high accuracy, we can only get a bunch of seemingly meaningless model parameters and judgment results with a very high degree of fitting in the end. As model developers, we obviously cannot just be satisfied with a good model.
In fact, the exploration of model interpretability is to explore the principle of model work well. If you understand the principle, you may not need to do a lot of useless model tuning work, and the direction of optimization will be more directional.
From the perspective of pattern recognition, the model itself also means knowledge (or the model itself stores some pattern knowledge), and we want to know what knowledge the model has learned from the data (expressed in a way that humans can understand) to generate final decision. Can it help us discover some potential associations.
For example, I want to develop an application based on a deep learning model to help doctors determine the risk of a patient. In addition to the final determination result, I may also need to understand the factors that the model makes such a determination based on the patient's considerations.
0x1: Different motivations for interpretability
1. Identify and mitigate bias
Bias can exist in any data set, and data scientists need to identify and try to correct bias.
1. Datasets may be limited in size and may not be representative of all data.
2. Or the data capture process may not have considered potential bias.
Bias often becomes apparent after a thorough data analysis, or when analyzing the relationship between model predictions and model inputs.
Note that there is no single solution to the problem of bias, but a critical step in explainability is awareness and detection of potential bias.
1) Deviations in feature engineering
In a virtual resume screening scenario, we segmented the resumes submitted by the interviewers and created a word index table. However, a problem that is easily overlooked is that for the technical department of an IT company, male job applicants often account for The vast majority, so words like "technical resume" are more likely to appear in male job applicants' resumes.
When training a model with these word vectors, a recruiter searching for "technical profile" would keep female resumes at the bottom.
But on the other hand, we also need to pay attention, if this deviation is intentional, then this kind of bais is acceptable. For example, our goal is not to search for resumes of job seekers equally, but to find job seekers with technical skills as much as possible, or our recruitment is more inclined to men, then this word table index encoding method is no problem.
In conclusion, bias in data mining projects needs to be treated carefully, it can be beneficial or harmful to us.
2) Unbalanced distribution of training data samples
For example when you are training an object detection model on a small dataset, often the width of the images is too limited. To avoid models that only fit noisy and unimportant elements in the data, images of various objects in different environments, under different lighting conditions and at different angles are required.
In the non-image field, especially in the field of NLP text detection, the number of samples of different text styles (code styles) is often unevenly distributed. This is very common in the backdoor of WEBSHELL website. Hackers tend to use one or a certain Several WEBSHELL samples, the most obvious problem that this deviation will bring is that the model may ignore some very small black samples.
2. Accounting for the context of the problem
In most problems, the dataset you are working with is only a rough representation of the problem you are trying to solve, and the machine learning model cannot capture the completeness of the real task Complexity. Purely theoretical infinite N samples are very difficult to achieve in actual engineering.
At this point, an interpretable model can help you understand and explain what is and is not included in the model.
3. Improving generalization and performance
Highly interpretable models usually have better generalization capabilities.
Interpretability is not about knowing every detail of the model for all data points. Solid data, models, and problem understanding must be combined for better performing solutions.
0x2: Interpretability is also needed in the field of security offensive and defensive AI.
In the industry and in the technical circle, the author has seen many practical engineering applications with good results in recent years. The deep learning model is based on massive data and large-scale deep structures. The results are quite good, even surpassing the detection ability of international veteran AV engines. Judging from the results, we don't seem to need interpretability that much. After all, the result is the most important thing. There are too many things in the world that cannot be explained, and not everything can be perfectly interpreted.
However, the author believes that there are still some scenarios where we need interpretability:
1. Find the real reason for the decline in model detection performance - track and locate the cause of the abnormality
One afternoon, when the sky was clear and we were writing blogs at our seats, suddenly a bunch of false alarms sounded, and your deep model encountered a batch of errors. report. You log off the model urgently to calm down, and the next thing to do is to find the reason for the false positive.
But now the problem is coming. Your model has 2000w of its own parameters, and the layers are completely nonlinear combinations. How do you know which neurons caused the false positives for this batch of samples? What is the commonality of this batch of samples that lead to false positives? Which of these feature patterns are misidentified by the model as a black pattern?
This series of questions will drive us to explore more deeply the interpretability of our trained deep learning model.
2. Find the fastest direction for iterative optimization.
I believe that everyone will go through a stage in the process of model development. After reading a lot of academic papers on Google and trying various neural network combinations, you will find that there are many neural network structures now. It was proposed, and the experimental results in the paper are also very amazing. At this time, we will try a lot with a "lucky mentality", although the problem scenarios in the paper may not be consistent with our projects.
Finally, through a large number of experiments for several weeks, we "groped" out a model structure, which performed very well on both the training set and the verification set, and both got 99.8/99.9 acc, but there were still a small amount of false positives (the author The industry we work in is sensitive to false positives, and false positives are relatively acceptable). At this time, we will face a problem, how to optimize it next? Continue to increase the sample? Or continue to read the paper to optimize the model structure? Are there some commonalities in principle between different neural network structures?
If you want to solve this problem systematically and engineeringly, you need to conduct interpretability research. For example, we obtained the hotspot weight distribution of the model through interpretable visualization, that is, analyzed which feature dimensions in the input feature space the model is more "fancy" and "relatively underestimated" some other feature dimensions, which may be precisely This is one of the reasons for our false positives.
Relevant Link:
https://www.leiphone.com/news/201805/Ti3mOdeRlWTplIlZ.html
https://cloud.tencent.com/developer/article/1096716
https://zhuanlan.zhihu.com/p/38151985
https://www.jiqizhixin.com/articles/2018-01-10
3. What interpretability methods are available
The purpose of machine learning is to discover knowledge or solve problems from data, so as long as it can provide us with understandable information about data or models in this process, it will help us more fully discover knowledge, understand and solve problems. methods, then they can all be classified as interpretability methods. If it is divided according to the process of interpretability method, it can be roughly divided into three categories:
1. Interpretability method before modeling
2. Build a model that is itself interpretable
3. Use interpretability method to explain the model after modeling 0x1: The category of
interpretability method before modeling
In fact, the method mainly involves some methods of data preprocessing or data display. Machine learning solves the problem of discovering knowledge and laws from data. If we know little about the characteristics of the data we want to deal with, it is unrealistic to expect to have a good understanding of the problem we want to solve. Therefore, the key to the interpretability method before modeling is to help us quickly and comprehensively understand the characteristics of the data distribution, so as to help us consider the problems we may face during the modeling process and choose the most reasonable model to approach the problem best possible solution.
1. Data visualization
Data visualization method is a very important pre-modeling interpretability method. Most of the time, before we really want to study a data problem, it is very necessary to establish our intuitive understanding of the data by establishing a series of visualization methods in all aspects, especially when the amount of data is very large or the data dimension is very high. For example, for some spatio-temporal high-dimensional data, if some interactive visualization methods can be established, it will greatly help us understand the distribution of data from various levels.
For related discussions in this regard, you can refer to my other blog.
2. Sample data exploration
Another important method is exploratory data analysis, which can help us better understand the distribution of data. For example, a method called MMD-critic can help us find some representative or unrepresentative samples in the data.
3. Exploration of feature importance
Feature importance is a basic method to explain the model
0x2: Build a model that is interpretable.
The interpretability of the model itself is the best and most direct type of interpretability method, and it is also a method with high requirements and restrictions (in many cases, deep neural networks are does not have interpretability), models with interpretability can be roughly divided into the following types
1. Rule-based method (Rule-based)
2. Single feature-based method (Per-feature-based)
3. Instance-based method (Case-based)
4. Sparsity method (Sparsity)
5. Monotonicity method ( Monotonicity)
1. Rule-based (Rule-based)
Rule-based methods such as the very classic decision tree model we mentioned. Any decision chain in this type of model can correspond to a logical rule representation. In a decision tree, interpretability is expressed through a series of if-else logical combinations,
However, rule-based methods are sometimes not suitable when the rules represent too much or the original features themselves are not particularly easy to explain.
2. Per-feature-based methods based on a single feature
are mainly some very classic linear models, such as linear regression, logistic regression, generalized linear regression, generalized additive model, etc., such models can be said It is currently the most interpretable method. The w and b in the linear regression equation are very interpretable.
Linear regression can be described as a very classic mathematical model. In econometrics, most books discuss linear models, including papers in economics and related fields. In fact, most of them also use linear regression as a method for research. This very classic model is used about 8 million times per second around the world.
Why do people like this model so much? In addition to the relatively simple structure of the model, what is more important is that the linear regression model and some of its variants have a very solid statistical foundation. Statistics can be said to be the subject that values explainability the most. For hundreds of years, countless mathematicians Statisticians discuss the parameter estimation, parameter correction, hypothesis testing, boundary conditions and other issues of the model in various situations, the purpose is to make the model have a very good explainability in various situations sex.
3. Case-based method (Case-based)
The case-based method is mainly a method to explain the clustering/classification results through some representative samples. By observing representative samples, we can intuitively obtain the macroscopic characteristics of samples of their corresponding categories.
For example, in the Bayesian Case Model (BCM) shown in the figure below, we divide the samples into three groups, and we can find out the representative samples and important subspaces in each group.
For the following first type of clustering:
The green face is a representative sample; and the green and square are representative feature subspaces (including 2 feature dimensions).
From this, we can get a preliminary judgment that the macroscopic feature of the first category is a bunch of squares with green faces.
The author's thinking: the cluster center of community discovery and the cluster centroid of Kmeans clustering can also provide an example-based data visualization perspective in essence.
For a discussion about community discovery, see my other blog.
4. Sparsity method (Sparsity)
The method based on sparsity mainly uses the sparsity characteristic of information to simplify the representation of the model as much as possible.
For example, a graph-sparse LDA method as shown in the figure below forms a hierarchical topic expression based on hierarchical word information, so that some small topics can be summarized by more general topics, which makes it easier for us to understand What a particular theme stands for.
5. Monotonicity
In many machine learning problems, there is a positive/negative correlation between some inputs and outputs. If we can find out this monotonic relationship during model training, we can make the model have Higher interpretability.
For example, doctors' estimates of the probability of suffering from a specific disease are mainly determined by some high-risk factors associated with the disease, and finding out the monotonic relationship can help us identify these high-risk factors.
But at the same time, on the other hand, in many cases, the feature space of the input data is a high-dimensional feature. For example, in the security field where the author is located, the abstracted vector features are often as high as tens of thousands. Faced with such high-dimensional data, it is necessary to find the input-ouput The linear correlation between them is very difficult.
0x3: Use the interpretability method to explain the model after modeling. The
interpretability method after modeling is mainly for deep learning models with black-box properties, and is mainly divided into the following categories of work:
1. Hidden layer analysis method
2. Simulation/proxy model
3. Sensitivity analysis method
Relevant Link:
https://www.leiphone.com/news/201805/Ti3mOdeRlWTplIlZ.html
https://new.qq.com/omn/20180117/20180117A0T4JE.html
4. Lime - Explaining the predictions of any machine learning classifier
Lime can show us in a visual way which "factors" the machine learning decision maker makes a comprehensive decision on.
0x1: The principle of Lime
Intuitively speaking, Lime uses a local linear approximation to simulate the target model. Although the target model itself may be globally complex, we can more easily approximate a local instance of a feature.
We regard the target model as a black box, and learn a low-dimensional sparse linear model through continuous and gradual trials as an explanation.
The figure below is a decision function of a complex model. The blue/pink background represents two decision surfaces. Obviously, this is non-linear.
The bright red cross in the figure is the explanation example obtained by Lime. We sample around the explanation example X, and the weight standard of the sampling is the degree of approximation to X (in this case, size). Then we get a linear model, namely the dotted line.
Clearly, this is a locally approximated linear model, not a global one.
0x2: Explain the decision factors of random forest through Lime
# -*- coding: utf-8 -*-
import lime
import sklearn
import numpy as np
import sklearn
import sklearn.ensemble
import sklearn.metrics
# For this tutorial, we'll be using the 20 newsgroups dataset. In particular, for simplicity, we'll use a 2-class subset: atheism and christianity.
from sklearn.datasets import fetch_20newsgroups
categories = ['alt.atheism', 'soc.religion.christian']
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)
newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)
class_names = ['atheism', 'christian']
# Let's use the tfidf vectorizer, commonly used for text.
vectorizer = sklearn.feature_extraction.text.TfidfVectorizer(lowercase=False)
train_vectors = vectorizer.fit_transform(newsgroups_train.data)
test_vectors = vectorizer.transform(newsgroups_test.data)
# Now, let's say we want to use random forests for classification. It's usually hard to understand what random forests are doing, especially with many trees.
rf = sklearn.ensemble.RandomForestClassifier(n_estimators=500)
rf.fit(train_vectors, newsgroups_train.target)
pred = rf.predict(test_vectors)
res = sklearn.metrics.f1_score(newsgroups_test.target, pred, average='binary')
print res
# Explaining predictions using lime
from lime import lime_text
from sklearn.pipeline import make_pipeline
c = make_pipeline(vectorizer, rf)
print(c.predict_proba([newsgroups_test.data[0]]))
# Now we create an explainer object. We pass the class_names a an argument for prettier display.
from lime.lime_text import LimeTextExplainer
explainer = LimeTextExplainer(class_names=class_names)
# We then generate an explanation with at most 6 features for an arbitrary document in the test set.
idx = 83
exp = explainer.explain_instance(newsgroups_test.data[idx], c.predict_proba, num_features=6)
print('Document id: %d' % idx)
print('Probability(christian) =', c.predict_proba([newsgroups_test.data[idx]])[0,1])
print('True class: %s' % class_names[newsgroups_test.target[idx]])
# The classifier got this example right (it predicted atheism).
# The explanation is presented below as a list of weighted features.
print exp.as_list()
# These weighted features are a linear model, which approximates the behaviour of the random forest classifier in the vicinity of the test example.
# Roughly, if we remove 'Posting' and 'Host' from the document , the prediction should move towards the opposite class (Christianity) by about 0.27 (the sum of the weights for both features).
# Let's see if this is the case.
print('Original prediction:', rf.predict_proba(test_vectors[idx])[0,1])
tmp = test_vectors[idx].copy()
tmp[0,vectorizer.vocabulary_['Posting']] = 0
tmp[0,vectorizer.vocabulary_['Host']] = 0
print('Prediction removing some features:', rf.predict_proba(tmp)[0,1])
print('Difference:', rf.predict_proba(tmp)[0,1] - rf.predict_proba(test_vectors[idx])[0,1])
# The explanations can be returned as a matplotlib barplot:
fig = exp.as_pyplot_figure()
# The explanations can also be exported as an html page (which we can render here in this notebook), using D3.js to render graphs.
exp.show_in_notebook(text=False)
# Alternatively, we can save the fully contained html page to a file:
exp.save_to_file('./oi.html')
# Finally, we can also include a visualization of the original document, with the words in the explanations highlighted. Notice how the words that affect the classifier the most are all in the email header.
exp.show_in_notebook(text=True)
Relevant Link:
https://github.com/marcotcr/lime
https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime
转载于:https://www.cnblogs.com/LittleHann/p/9720503.htmlPart II: The concept and simple implementation of community discovery
In a social network, a user is equivalent to every point, and the structure of the entire network is formed through the relationship between users.
In such a network, some users are relatively closely connected, and some users are relatively sparsely connected. The part that is more closely connected can be regarded as a community, and there is a relatively tight connection between the internal nodes, while the relative connection between the two communities is relatively sparse.
The whole overall structure is called the community structure. As shown in the figure below, the red and black point sets show the structure of the community.
Reference URL: https://www.cnblogs.com/LittleHann/p/9078909.html
—————————————————
Copyright statement: This article is for CSDN blogger "Give another bowl "The original article follows the CC 4.0 BY-SA copyright agreement. For reprinting, please attach the original source link and this statement.
Original link: https://blog.csdn.net/qq_35155934/article/details/109189919