The core of the linear model is a linear function s= w T x, which is to linearly combine all input variables. For linear regression problems, input x and output w T x; for linear classification problems, discrete discrete For example, output 1 indicates that a sample belongs to category C1, and output 0 indicates that it does not belong to category C1. At this time, it is only necessary to simply add a threshold on the basis of the linear function, and the value obtained when the classification function is executed is greater than or less than this threshold to determine class attribution; for the third problem, logistic regression, the output value is a probability value between 0 and 1.
question
Let's use an example to explain what a logistic regression model is: Suppose we want to infer a person's probability of developing heart disease based on factors such as heart rate, cholesterol level, age and weight, etc. Obviously, we can't tell 100% whether he has a heart attack or not. He has heart disease, but we can infer his probability of having heart disease based on these known conditions. This uses a logistic regression model, and the result is more accurate than the ordinary binary model. The closer the result is to 1, the greater the probability of having a heart attack.
Model
h (x) = θ (w T x)
Where θ is a logistic function or a sigmoid function, and its function form is 
, the corresponding function image is an S-shaped curve with a value between 0 and 1:
decision function
Returning to the question we raised at the beginning of the article, for a given input x (that is, the indicators of the person to be predicted) to predict the probability of his heart disease, we can get an objective function or prediction function:
The value of the h(x) function has a special meaning, it represents the probability that the result takes 1, so for the input x, the probability of the classification result being class 1 (suffering from heart disease) and class-1 (not changing heart disease) are:
We replace h(x) with θ(w T x), and , 
, so we can get P(y|x) = θ(yw T x))
loss function
In the logistic regression model, we use the logarithmic loss function and use the maximum likelihood to estimate the solution parameters. Below I will explain how to use the logarithmic loss function from a mathematical point of view:
Since the data points (x 1 , y 1 ),...,(x N , y N ) in our dataset are independent of each other, the probability of finding all y n for the entire dataset is
(1)
The maximum likelihood estimation method can be used to calculate the parameter w that maximizes the above probability value. For the convenience of calculation, we equivalently convert it into the minimum value of the following formula:
(2)
Because it 
is a monotonically decreasing function, the minimum value of formula (2) is the maximum value of formula (1). We replace P(y|x) with θ(y w T x)) to get the expression related to the parameter w
We find the optimal parameter w by minimizing the above formula, so we can regard the above formula as an error measure, and replace θ(y w T x)) with the sigmoid function, we can get the loss of the logistic regression function function expression
This means that for a data point, the loss function e(h(x n ),y n ) = ln(1+e y n wTx n ), when y n w T x n is a relatively large positive number The smaller the loss function, that is sign(w T x n ) = y n, so we can use this loss function to calculate w. So the expression of the loss function is:
In the next article I will explain how to solve this loss function using gradient descent
参考资料：<< Learning From Data>> Yaser S . Abu-Mostafa California Institute of Technology