1 The role of loss
In Nanxi's view, the loss function is a formula for measuring the distance between the target and prediction;
2 Design principles of loss function
2.1 Self-immutability
Self-invariance refers to: when prediction = target, the value of the loss function is 0;
3 Remarks
3.1 Before using torch.log(), you need to clamp
When writing the loss function, the torch.log() function is often used, such as when calculating cross entropy;
In PyTorch, before using torch.log(), remember to perform a clamp operation,
Because the log() function has unstable numerical results in the case of 0 and negative numbers,
as the picture shows