Further understanding of the neural network convolution

Recently re-learned some convolution neural network, with some deeper understanding

1, the installation tool

Installation tensorflow and keras spent a lot of effort to find a good installation blog at the following link:
installation Anaconda Tensorflow and Keras installation tutorial
now use more
worth noting that
we use the anaconda installation, but also because keras the need to support these libraries, then install the above link to that blog post installation.
install numpy PIP
PIP install matplotlib
PIP install SciPy
PIP install tensorflow
PIP install keras

2, understanding and perception

(1) Optimizer Optimizer

Wherein, following optimizers
deep learning - Detailed optimization algorithm Optimizer ( BGD , the SGD , MBGD , the Momentum , NAG , Adagrad , Adadelta , RMSprop , Adam )
course, the SGD is commonly used, i.e., stochastic gradient descent algorithm.
B stochastic gradient descent station tutorial

(2) Alex classic cnn network

AlexNet whole network structure is composed of five layers and three convolution fully connected layers, the total depth of the layer 8

# 导包
import keras
from keras.models import Sequential
from keras.layers import Dense,Activation,Dropout,Flatten,Conv2D,MaxPool2D
from keras.layers.normalization import BatchNormalization
import numpy as np
np.random.seed(1000)

# 获取数据
import tflearn.datasets.oxflower17 as oxflower17
x,y = oxflower17.load_data(one_hot=True)

# 定义AlexNet模型
model = Sequential()
# 1block
model.add(Conv2D(filters = 97,
                 kernel_size = (11,11),
                 strides = (4,4),
                 padding = "valid",
                 input_shape = (224,224,3)))
model.add(Activation("relu"))
model.add(MaxPool2D(pool_size = (2,2),
                    strides = (2,2),
                    padding = "valid"))
model.add(BatchNormalization())
# 2block
model.add(Conv2D(filters = 256,
                 kernel_size = (11,11),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(MaxPool2D(pool_size = (2,2),
                    strides = (2,2),
                    padding = "valid"))
model.add(BatchNormalization())
# 3 block
model.add(Conv2D(filters = 384,
                 kernel_size = (3,3),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(BatchNormalization())
# 4 block
model.add(Conv2D(filters = 384,
                 kernel_size = (3,3),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(BatchNormalization())
# 5 block
model.add(Conv2D(filters = 256,
                 kernel_size = (3,3),
                 strides = (1,1),
                 padding = "valid"))
model.add(Activation("relu"))
model.add(MaxPool2D(pool_size = (2,2),
                    strides = (2,2),
                    padding = "valid"))
model.add(BatchNormalization())
# 6 dense
model.add(Flatten())
model.add(Dense(4096, input_shape=(224*224*3,)))
model.add(Activation("relu"))
model.add(Dropout(0.4))
model.add(BatchNormalization())
# 7 dense
model.add(Dense(4096))
model.add(Activation("relu"))
model.add(Dropout(0.4))
model.add(BatchNormalization())
# 8 dense
model.add(Dense(17))
model.add(Activation("softmax"))

model.summary()

# compile
model.compile(loss = "categorical_crossentropy",
              optimizer = "adam",
              metrics = ["accuracy"])
# train
model.fit(x,
          y,
          batch_size = 32,
          epochs = 8,
          verbose = 1,
          validation_split = 0.3,
          shuffle = True)

(3) Residual network

ResNet network for solving the problem underlying core structure disappears gradient residuals network

residuals adds a network identity mapping (identity map), the current output is directly transmitted to the next layer network (all 1: 1 transmission, without increasing the additional parameters), equivalent to walking up a shortcut, skip this operation layer, this direct connection is named "skip connection". While the back-propagation process, but also the lower gradient layer of the network layer is directly transmitted to the network, this would resolve the problem of the disappearance of the Deep Web gradient.

import keras
from keras.layers import Conv2D,Input

x = Input(shape=(224,224,3))
y = Conv2D(3,(3,3),padding="same")(x)

z = keras.layers.add([x,y])

from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np

model = ResNet50(weights="imagenet")

img_path = "elephant.jpg"
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

preds = model.predict(x)
# decode the results into a list of tuples (class, description, probability)
# (one such list for each sample in the batch)
print("Predicted:", decode_predictions(preds, top=3)[0])

b stop learning video click here