We can use the tf.GradientTape() function in the with structure to realize the derivation operation of a certain function to the specified parameters.
With the variable function just mentioned, we can realize the derivation operation of the loss function loss to the parameter w
import tensorflow as tf
with tf.GradientTape() as tape:
w = tf.Variable(tf.constant(3.0))
loss = tf.pow(w, 2)
grad = tape.gradient(loss, w)
print(grad)
In this example, the initial value of w is 3.0, the loss function is the square of w, the derivative of the loss function to w is 2w, and the initial value w=3.0 is brought into the result is 6.0
enumerate means enumeration, it can be enumerated Out each element, and match the corresponding index number in front of the element, combined into an index element, often used in a for loop
seq = ['one', 'two', 'three']
for i, element in enumerate(seq):
print(i, element)
In this example, enumerate is the name of the list in parentheses, i accepts the index number, and element accepts the element.
Index number 0 corresponds to the string one
Index number 1 corresponds to the string two
Index number 2 corresponds to the string three
In the classification problem, we often use one-hot codes to represent labels . For example, in the previous iris classification, if the label is 1, it means that the classification result is variegated iris, and it is 0 1 0 in the form of one-hot codes. This can represent the probability of each category, that is, 0% may be 0 Setaria iris, 100% may be 0 variegated iris, 0% may be 2 Virginia iris
TensorFlow provides the one_hot function, you can change The data to be converted is directly converted into one-hot code form
import tensorflow as tf
classes = 3
labels = tf.constant([1, 0, 2]) # 输入的元素值最小为0，最大为2
output = tf.one_hot(labels, depth=classes)
print("result of labels1:", output)
print("\n")
For example, there are 3 categories, a set of labels is 1, 0, and 2, to convert them into one-hot codes, use the tf.one_hot function, the first variable writes the data labels to be converted, and the second variable writes several categories. Assign a value of 3 categories, and convert label 1 label 0 label 2 into a one-hot code, the result is 010, 100, 001
For the classification problem, the neural network completes the forward propagation and calculates the probability of each type of 1.01, 2.01, -0.66. These numbers can only be compared with the one-hot code label after they conform to the probability distribution.
So we use this formula to make the output conform to the probability distribution.
0.256 means that the probability of class 0 iris is 25.6%.
0.695 means that the probability of class 1 iris is 69.5%.
0.048 means 2 The probability of iris-like is 4.8%
. The softmax function is used in TensorFlow. The calculation of this formula
can make the n outputs y0, y1..., yn-1 of n categories conform to the probability distribution, that is, each output value becomes 0 to Probability values between 1, and the sum of these probabilities is 1
import tensorflow as tf
y = tf.constant([1.01, 2.01, -0.66])
y_pro = tf.nn.softmax(y)
print("After softmax, y_pro is:", y_pro) # y_pro 符合概率分布
print("The sum of y_pro:", tf.reduce_sum(y_pro)) # 通过softmax后，所有概率加起来和为1
In the example we give, the neural network forward propagation results 1.01, 2.01, -0.66 form a tensor y, and send it to the softmax function. The output is these values that conform to the probability distribution. The
assign_sub() function is often used for parameter self-update, waiting The self-updating parameter w must first be designated as an updatable training, that is, a variable type, before self-updating can be implemented. For
example, in this example, w must be defined as a variable type first, and the initial value is 4. To decrement w, you can use the w.assign_sub() function. The value in () is 1, which realizes the operation of w=w-1. Operation
result:
The initial value 4 of w is decremented by 1, and w is updated. For 3
tf.argmax() can return the index number of the maximum value of the specified operation axis. For
example, for this two-dimensional tensor argmax, axis=0, the index number of the vertical maximum value is returned, respectively,
1 2 5 8 and the maximum value is 8. , Its index number is 3
2 3 4 7, the maximum value is 7, its index number is 3
3 4 3 2, the maximum value is 4, its index number is 1
argmax, axis=1, it returns the maximum horizontal index Number, respectively
1 2 3, the maximum value is 3, its index number is 2
2 3 4, the maximum value is 4, its index number is 2
5 4 3, the maximum value is 5, and its index number is 0
8 7 2, the maximum value is 8, and its index number is 0