Generative Models@ Machine Learning
转载请标明出处，本篇文章允许转载，禁止抄袭
machine learning model
Machine learning models are divided into generative models and discriminative models
Take the following task as an example:
Task: Generate a model by learning the data in the above picture, and when a new ball appears, it can predict the color of the ball as correctly as possible (blue or black)
Generative and discriminative models
generative model
Through the above data
- Directly learn a decision function f(x)
- Or learn the joint probability density distribution P(X,Y) of the data, generate a conditional probability distribution P(Y|X), and predict future data.
- In short, the generative model can learn the distribution of data during the learning process, thereby generating the output function F(x) - In the
sub-task, the model learns the overall data, obtains the distribution of black balls and basketballs, and then gives a small ball to be judged, calculates the probability of belonging to the black ball and the probability of belonging to the basketball, and makes a judgment.
discriminant model
Through the above data
- Directly learn a decision function f(x)
- Or directly learn the conditional probability distribution P(Y|X) to predict future data.
- In the second task, the model divides the black ball and the basketball through a straight line or a hyperplane, and when a small ball to be judged is given, the probability of belonging to the black ball and the probability of belonging to the basketball are calculated for judgment.
Note:
-
x: is a feature of a data sample, in the form of (feature 1, feature 2, ..., feature n), that is, x=(x1, x2, ..., xn)
-
y: is the label/label of a sample (label in English, translated as label, etc.), in the form of a value, a type, etc. For example,
in this task, a y may be black, that is, y=black -
In the process of supervised learning, a complete training data should include x and y, that is, data=(x, y)
-
Learning purpose: That is, by learning the data on the training set, a function expression (model) can be obtained, and the future x that has not been seen in the training process can be predicted (operated) according to this model, and the predicted value (y_hat = F(x)) is as close as possible to the real value (y).
-
Decision function f(x): For data=(x), input x, calculate a y_hat according to the model, compare y_hat with a previously set threshold (a value, equivalent to a limit, separate the upper and lower parts of this limit), determine the category of this piece of data according to the comparison result, and then output it.
For example, in this task, the threshold is 10, y_hat less than 10 is black, and greater than 10 is blue; input x=(3,2), f(x)=9, then this data belongs to black. -
Conditional probability distribution P(Y|X): For data=(x), input an x, compare its probability of belonging to all classes, and then apply the maximum posterior probability method (MAP) when predicting, that is, compare the class with the highest conditional probability as the category corresponding to x.
For example, in this task, input x, calculate P (category 1 | x) = 0.38, P (category 2 | x) = 0.62, then this x belongs to category 2, which is blue.
Similarities and differences between generative and discriminative
| generative | Discriminant | |
| Model | supervised learning | supervised learning |
| Joint probability density P(x,y) | learn as you learn | not learned |
| focus | similarity between | heterogeneous difference |
| Division method | Distribution function | line or hyperplane |
| shortcoming | Sensitive to outliers; large data set requirements; large amount of calculation | easy to overfit |
| advantage | The probability of overfitting is small | Performs well on small datasets; small amount of computation |
Please indicate the source of the reprint, this article is allowed to be reprinted, plagiarism is prohibited