First, I recommend a website for learning tensorflow and keras. Basically, you can basically get it by going through the document:
https://tensorflow.google.cn/api_docs/python/tf/keras/Model
Here are a few pitfalls encountered by beginners in deep learning from the shallower to the deeper:
About Python:
- When using the
form from package import item, the corresponding item can be either a submodule (subpackage) in the package, or other names defined in the package, such as functions, classes, or variables. - The import syntax will first treat item as the name of a package definition, and if it is not found, it will try to import it according to a module. If it has not been found, congratulations, an :exc:ImportError exception has been thrown.
- Conversely, if you use an import form like import.tem.subitem.subsubitem, all except the last item must be a package, and the last item can be a module or a package, but not a class, function, or variable. first name.
About numpy:
1. Distinguish three points, no colon, single colon, double colon:
- a[...,2]:...means to traverse each row, 2 means the column whose index is 2.
- a[...,:2]:...means to traverse each row, and :2 means the column where the index is 0,1 where the index is <2.
- a[…,::2]:… means to traverse each row, 2 means step length, select the column where the multi-index is 0, 2, 4.
- a[::-1]: Inverted, the fixed dimension of the list to be output is reversed.
import numpy
>>> a = numpy.array([[1,2,3,4,5],[6,7,8,9,10],[1,2,3,4,5],[6,7,8,9,10]])
>>> a
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10]])
>>> a[...,2]
array([3, 8, 3, 8])
>>> a[...,:2]
array([[1, 2],
[6, 7],
[1, 2],
[6, 7]])
>>> a[...,::2]
array([[ 1, 3, 5],
[ 6, 8, 10],
[ 1, 3, 5],
[ 6, 8, 10]])
2. About the use of .numpy() in tensorflow 2.x version.
When we use TensorFlow for deep learning training, we often greet Numpy data, such as our csv or photo data. But we all know that TensorFlow uses Tensor to store variables during training.And the result of the network output is also Tensor. Under normal circumstances, we will not feel the difference between Numpy and Tensor, because the TensorFlow network will automatically convert to Tensor for processing when inputting Numpy data.Since the session mechanism is cancelled in the 2.x version, developers can directly execute the .numpy() method to convert tensor:
data_numpy = data_tensor.numpy()
The print output before conversion is tf.Tensor(1.0, shape=(), dtype=float32), and the converted output is 1.0
About CNN
1. Use a 7×7 convolution kernel and a 1×1 convolution kernel to downsample with the same step size. What is the difference between the two?
The receptive field is different, the 7×7 sliding window is large, and the extraction area is also large.
2. What is the difference between pooling and downsampling with convolution kernel?
The most direct thing is that the calculation of pooling and convolution is not the same. Both will perform effective information extraction on the feature map and perform dimensionality reduction. The advantage of pooling is that the amount of calculation is small, either max or average, while the amount of calculation for conv Large, but the advantage of conv is that it can exchange information between channels, which is more conducive to feature extraction.
3. Regarding the size requirements of the CNN input image,
make a simple summary:
1. Because there is a fully connected layer, if you do not use the pre-training model, you must ensure that your training image size is the same as the predicted image size;
2. If you use pre-training Model, the input image size of your own test set must be the same as the original model training size;
3. If migration learning, the weights of the first several layers are fixed, and only part of the network layer that contains the fully connected layer is trained, that is, it is not an overall migration. It needs to be the same size as the original model training requirement.
About tensorflow
1. The data type
tensor in tensorflow has three attributes: data, dimension, data type, first of all about dimension:The dimension depends on how many left square brackets there are on the leftmost side of the tensor. There are n, then this tensor is an n-dimensional tensor. As follows:
1 #维度为0的标量
[1,2,3] #维度为1,一维向量
[[1,2],[3,4]] #维度为2, 二维矩阵
[[[1,2],[3,4]],[[1,2],[3,4]]] #维度为3,3维空间矩阵
The number of numbers in the brackets of shape() indicates how many dimensions the tensor is.
2. About tf.keras.metrics
When we compile the model in tensorflow 2.x, we will see that one of the parameters is metrics.It is used to monitor some performance indicators during training, And what this performance index is can be specified by us.The performance evaluation function is similar to the objective function, but its evaluation results will not be used for training.
The first thing you need to know is that if it is a class under metrics, then its instantiated object has the following three attributes:
- reset_states() Clear the previous calculation results, which is equivalent to resetting and restarting the calculation
- result() calculate and return the result
- update_state() regards each updated data as a set of data, so that the results of each set will be calculated during the actual calculation, and then the average of the results of multiple sets will be calculated. But it will not be calculated directly. The calculation is still done when result() is called
-in m.result(), a tensor data will be obtained, and then use .numpy to convert it to numpy data.
Here is an eg:
m = tf.keras.metrics.Accuracy() #新建一个实例化对象
m.update_state([1, 2, 3, 4], [0, 2, 3, 4]) #向对象中添加数据
m.result().numpy()
#输出：0.75
m.reset_states()
m = tf.keras.metrics.CategoricalAccuracy()
_ = m.update_state([[0, 0, 1], [0, 1, 0]], [[0.1, 0.9, 0.8],
[0.05, 0.95, 0]])
m.result().numpy()
#输出：0.5
m = tf.keras.metrics.SparseCategoricalAccuracy()
_ = m.update_state([[2], [1]], [[0.1, 0.9, 0.8], [0.05, 0.95, 0]])
m.result().numpy()
#输出：0.5
3. About self.conv1 = Conv2D(32, 3, activation='relu') and x = self.conv1(x)
The former is the instantiation object of the class, and the latter is the convolution operation. Why is there (x) at the end?This is a specific way of writing tensorflowJust remember, the purpose is: add data to the convolutional layer, and then return an output , the above two forms can also be written like this:
- x = Conv2D (32, 3, activation = 'reread') (x)
Note: This is different from the input of our function. Under normal circumstances, the input of the function is placed inside the parentheses, and there are two parentheses here, which combine the instantiation of the class and the addition of data, and belong to one A fixed way of writing. There are detailed instructions on the tensorflow official website:
About model training
If there is only a CPU, it is a serial process . The CPU first loads the image and performs preprocessing, and then performs training, repeating continuously. If using the GPU, the CPU is only responsible for the image reading and preprocessing process, and the GPU concentrates on the network. For training, every time the CPU reads a batch, the GPU trains a batch, which significantly reduces the time consumption.
Why use the tf.data.Dataset.from_tensor_slices() method to read data in parallel?
If you use the keras data generator, it is impossible to use multi-threading in the current version, which will lead to very low efficiency. It takes a long time for the CPU to read and process, but the GPU operation only takes a short time Time, in this case, the GPU will be in an idle state many times, and GPU resources are not used well. If the multi-threading method is used, the time for CPU reading and preprocessing will be greatly shortened, the occupancy rate of the GPU will be greatly improved, and the entire training process of the network will also be improved.
