Summary of Logistic Regression
Logistic Regression algorithm, as a binary classification algorithm, mainly solves the problem of linear separability. For multi-classification algorithm, Softmax Regression algorithm can be used.
Softmax Regression is a generalized Logistic Regression, and Logistic Regression can be regarded as a special case of Softmax Regression.
So how to choose Softmax Regression and Logistic Regression? Refer to the content of Stanford's article.
Softmax regression vs. k binary classifiers
If you are developing a music classification application and need to identify k types of music, should you choose to use a softmax classifier, or use a logistic regression algorithm to build k independent binary classifiers?
This choice depends on whether your categories are mutually exclusive, for example, if you have four categories of music: classical, country, rock, and jazz, then you can assume that each training sample will only be marked with A label (ie: a song can only belong to one of these four musical genres), at which point you should use softmax regression with k = 4 categories. (If there are songs in your dataset that do not belong to any of the four categories above, you can add an "other category" and set the number of categories k to 5.)
If your four categories are as follows: vocals, dance, soundtrack, pop, then these categories are not mutually exclusive. For example: a song can originate from the soundtrack of a film and television, and also contain vocals. In this case, it is more appropriate to use 4 binary logistic regression classifiers. In this way, for each new musical composition, our algorithm can determine whether it belongs to each category separately.
Now let's look at an example from the field of computational vision, where your task is to classify images into three different classes. (i) Suppose the three categories are: indoor scenes, outdoor urban scenes, and outdoor wilderness scenes. Will you use sofmax regression or 3 logistic regression classifiers? (ii) Now suppose that the three categories are indoor scenes, black and white pictures, and pictures containing people, would you choose softmax regression or multiple logistic regression classifiers?
In the first example, the three classes are mutually exclusive, so a softmax regression classifier is more appropriate to choose. In the second example, it is more appropriate to build three independent logistic regression classifiers.