1. cross entropy: cross entropy characterizes the distance between two probability distributions, it is the classification used in a broad loss of function. Represented by q p is the cross-entropy:
The Softmax Results obtained in the propagation neural network into a probability distribution, the output of the neural network is used as a raw confidence to generate a new output, the new output probability distribution meet all the requirements.
Cross-entropy function is not symmetric, H (p, q)! = H (q, p), he portrayed the difficulty is to express the probability distribution of the probability distribution p by q. Because the desired result of correct answer is obtained, so that when the cross-entropy function is a neural network losses, P represents the correct answer, q represents the predicted value. Cross entropy is a portrait of the distance between two probability distributions, that cross entropy value is smaller, the closer the two probability distributions.
tensorflow achieve cross-entropy codes:
Which represents the correct result y_, y represents the predicted results. tf.clip_by_value () function means that the number is less than 1e-10 1e-10 replaced the full number greater than 1 into a whole. In tensorflow * mean number of entries corresponding to the same position by not matrix multiplication.
Because usually used softmax return with cross-entropy, so tensorflow these two functions were unified package:
This command can be obtained by using a cross-entropy after Softmax return.
In the classification problem in only one correct answer, tensorflow provided tf.nn.sparse_softmax_cross_entropy_with_logits functions to further speed up the calculation process.
2. Loss Function --------- -------- classic loss function mean squared error (MSE, mean squared error):
Where yi is the correct answer in a batch of i-th data, and yi 'is the predicted value of the neural network analysis. tensorflow implementation code:
3. The loss function custom function ----- ---------
tf.greater (A, B) Returns A> B result, Boolean
tf.select (C, A, B) C is true (True), A return value, the return value B is false (False).
These two functions are carried out at the element level