[Softmax] Operation object (use the feature map generated by the last layer of the model to perform the softmax output result verification for verification)

every blog every motto: You can do more than you think.

0. Preface

We know that for classification problems, the last layer will go through softmax, and we use data to verify the final result.

1. Text

softmax formula:

import numpy as np
import os, sys

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
from tensorflow.keras.layers import *
import tensorflow as tf

1.1 One-dimensional

# 一维情况
tensor1 = tf.constant(np.array([1, 0, 0]), dtype=tf.float32)
print('tensor1: ', tensor1)
print('tensor1 shape: ', tensor1.shape)

tensor1_soft = Softmax()(tensor1)
print(tensor1_soft)
print('-' * 50)

Results:

From the above results, we can solve the data in one dimension according to the above formula, and we can verify it with numpy:

np_soft = np.exp(tensor1) / np.sum(np.exp(tensor1))
print(np_soft)
print('-*'*20)

Result:

The result obtained is consistent with the result of softmax.

1.2 Two-dimensional

# 二维情况
tensor2 = tf.constant(np.array([[1, 1], [2, 0]]), dtype=tf.float32)
print('tensor2: ', tensor2)
print('tensor2.shape: ', tensor2.shape)
tensor2_soft = Softmax()(tensor2)
print(tensor2_soft)
print('-' * 50)

result:

From the above results, we can find that only [1,1] and [2,0] have each done softmax, so I won't explain much here.
Specific process:
Take two of them as an example:

1.3 Three-dimensional

1.3.1 The first case

# 三维情况
tensor3 = tf.constant(np.array([[[1, 1], [1, 0], [0, 0]]]), dtype=tf.float32)
print('tensor3: ', tensor3)
print('tensor3.shape', tensor3.shape)
print('-'*29)
tensor3_soft = Softmax()(tensor3)
print(tensor3_soft)

result:

1.3.2 The second case

# 三维情况
tensor3 = tf.constant(np.array([[[1], [0]], [[2], [1]], [[0], [0]]]), dtype=tf.float32)
print('tensor3: ', tensor3)
print('tensor3.shape', tensor3.shape)
print('-' * 29)
tensor3_soft = Softmax()(tensor3)
print(tensor3_soft)

Results:

From the above results, we found that all the values ​​are 1. We then contact the shapes of all tensors and we can find such a rule:
Softmax solves the data in the last dimension.

1.4 ReLu (attached)

Here we only list one column, take 3D as an example: the
result (conclusion) is the same as Softmax

# 三维情况
tensor3 = tf.constant(np.array([[[1], [-1]], [[2], [1]], [[0], [0]]]), dtype=tf.float32)
print('tensor3: ', tensor3)
print('tensor3.shape', tensor3.shape)
print('-' * 29)
tensor3_soft = ReLU()(tensor3)
print(tensor3_soft)

result:

1.5 Summary

1. So far, we can find from the above, and get the following conclusion: the
   activation function has a positive effect on the feature mapLast dimension, To solve.

2. This is why regarding classification issues, the last dimension in the shape of the feature map output by the model is equal to the number of classifications (this sentence may be a bit winding)

3. We can think of the previous semantic segmentation explanation, the final input shape of the model is (208*208, 2), that is, (43264, 2), and its last dimension is 2, and the two numbers are softmax to obtain the probability .

references

[1] https://blog.csdn.net/weixin_39190382/article/details/106301781
[2] https://zhuanlan.zhihu.com/p/105722023
[3] https://blog.csdn.net/weixin_39190382/article/details/106516370
[4] https://blog.csdn.net/siyue0211/article/details/81017728