This is my first original work in more detail. After such a long period of in-depth study, I can now do some small projects, and I have some deep feelings from my heart. The mastery of basic knowledge determines the height of research. When we first came into contact with deep learning, we usually saw other people’s generalizations. This method is very good for us to get started quickly, but it also has a big disadvantage. The knowledge understanding is not thorough. As a result, we are confused about algorithm optimization. I also started my exploration of the essence of deep learning knowledge with the idea of knowledge summary, and I hope to help more people. There are unclear points in the article. I hope fellow researchers (friends who study deep learning) will point it out and I will work hard to improve my article.
Enter the article topic:
SVM loss function (hinge loss):
Targeted problem: classification problem
First, find the product of the input features corresponding to different classification weights, and then use max to calculate the loss based on these scores. The design is calculated back to the total wrong guess score minus the correct score and then added to the set hyperparameter, and the result is maximized compared to zero. The maximum value obtained is the loss value. The weights are updated through derivation to reduce errors.
Softmax loss (also called logistic regression in bisection):
The main focus is on multi-classification and calculating probabilities:
The biggest difference between softmax loss and svm loss is that svm loss only needs to make the correct classification score greater than the incorrect score. However, the softmax loss is to hope that the correct classification will gradually tend to 1, and the incorrect classification probability will tend to zero, which is an infinite gratitude. First find exp(wx) for the wx score, then normalize it to find the probability, and finally use -log() to find the loss function.