Classical algorithm
Logistic regression
question: logistic regression compared to the linear regression, the similarities and differences?
answer: logistic regression was the classification problem, linear regression is a return to the problem, which is the most essential difference between the two. Logistic regression, the dependent variable is a binary value distribution model of learning is derived 
, that is, after the given arguments and hyper-parameters to obtain the desired dependent variable, and based on this classification to predict expect to deal with the problem. The linear regression is actually solved 
, for our hypothetical true relationship 
of an approximation, which 
represents the error term, we use this term to approximate the Regression.
Of course, logistic regression and linear regression there are some similarities, first of all we can think both use maximum likelihood to estimate modeling training samples. Under the assumption of linear regression using the least squares method, in fact, the parameter x determines the super θ the argument, the dependent variable y normally distributed, using a maximum likelihood estimation of simplification; and by logistic regression likelihood the function 
of learning, optimal parameter θ. In addition, both in the process of solving the super parameter, you can use gradient descent method, which is supervised learning in a common similarities.
question: When using logistic regression deal with the multi-label classification problem, how about the relationship between them which is common practice, which are used in the scene,?
answer: Which approach to use to handle the multi-classification problem depends on the definition of specific issues. First of all, if only one sample corresponds to a label, we can assume probability of each sample belonging to different labels subject to the geometric distribution, use multiple logistic regression (Softmax Regression) to classify
Which 
is the parameters of the model, and 
can be seen as the probability of normalization. For convenience, we will 
the k column vectors are arranged in order n × k dimensional matrix form, writing [theta], it represents the entire set of parameters. In general, a number of logistic regression parameters have redundancy features, will soon 
also predicted the same result after a vector addition and subtraction. In particular, when the type of number 2,
Using the parameters redundancy features, we will subtract all the parameters θ1, the above equation becomes
Which 
. The logic behind the formula and return to the same finishing. Therefore, a number of logistic regression binary logistic regression is actually an extension of the multi-label classification.
When the presence of samples may belong to multiple labels, we can train the k binary logistic regression classifier. I category is used to distinguish whether each sample can be classified as class i, training the classifier, you need to rearrange the tag as "tag class i" and "non-class i tag" categories. Through this approach, we have solved the circumstances of each sample may have multiple labels.