1. quadratic loss function Square Error:
$$L(f(x),y)=(f(x)-y)^{2}$$
Then empirical risk function is MSE, such as linear regression occurs
2. The absolute value of the loss function:
$$ L (f (x), y) = \ green f (x) -y \ green $$
Then empirical risk function is MAE
3. 0-1 loss function:
$$L(f(x),y)=1_{\lbrace f(x)\neq y\rbrace}$$
4. Number of loss function (crossentropy)
$$L(P(y\mid x),y)=-\log P(y\mid x)$$
Alignment Model: logistic regression, softmax return
Note that for non-balanced binary classification problem, we can also add the appropriate class of weights $ w (y) $ so called weighted logarithmic loss function:
$$L(P(y\mid x),y)=-w(y)\log P(y\mid x),$$
For example the training set a classification problem: $ D = \ lbrace (x_ {1}, y_ {1}), ..., (x_ {N}, y_ {n}), y_ {i} \ in \ lbrace-1, + 1 \ rbrace \ rbrace $
Number of positive samples $ $ P much less than the number of cases of negative samples {N}, we can choose appropriate $ w (+1) $, $ w (-1) $ such that $ w (+1) P $ and $ W ( -1) N $ closer number.
The exponential loss function Exponential loss Function:
$$L(f(x),y)=\exp(-y\cdot f(x))$$
Alignment Model: AdaBoost
We note that in the logarithmic regression chance binary model, in fact logarithmic loss function can be controlled exponential function loss:
$$\log(1+\exp(-yf(x)))\leq \exp(-yf(x)),$$
Loss function exponentially easier calculation and derivation, so it is suitable loss function as AdaBoost each weak classifier.
6. Hinge loss function
$L(x)=[1-tx]_{+}$
Found in the SVM.