Deep learning notes - neural network (ANN) building process + python code

Table of contents

1. Multidimensional array operations

(1) Multidimensional array

(2) Matrix multiplication

(3) Inner product of neural network

2.3-Layer Neural Network Implementation

 (1) The weighted sum of the first layer

 (2) Signal transmission from the input layer to the first layer

(3) Signal transmission from layer 1 to layer 2

(4) Complete code

---

1. Multidimensional array operations

(1) Multidimensional array

The dimensions of multidimensional arrays can be obtained with the np.ndim() function. In addition, the shape of the array can be obtained through the instance variable shape.

A two-dimensional array is also called a matrix , and the horizontal arrangement of the array is called a row (row) , and the vertical arrangement is called a column (column)

code:

import numpy as np
A=np.array([1,2,3,4])
print("一维数组A的维数、形状、第一维度形状：")
print(A)
print(np.ndim(A))
print(A.shape)
print(A.shape[0])
B=np.array([[1,2],[3,4],[5,6]])
print("二维数组B的维数、形状、第一维度形状：")
print(B)
print(np.ndim(B))
print(B.shape)
print(B.shape[0])

operation result:

(2) Matrix multiplication

The calculation method of the matrix product is shown in the figure below

 The product of matrices can be calculated by NumPy's np.dot() function (the product is also called the dot product)

code:

(2*2 matrix dot product calculation)

A = np.array([[1,2],[3,4]])
print(A.shape)
B = np.array([[5,6],[7,8]])
print(B.shape)
print(np.dot(A,B))

operation result:

(2*3 matrix and 3*2 matrix dot product operation)

A = np.array([[1,2,3],[4,5,6]])
print(A.shape)
B = np.array([[1,2],[3,4],[5,6]])
print(B.shape)
print(np.dot(A,B))

operation result:

 (2*3 and 2*2 matrix for dot product operation)

A = np.array([[1,2,3],[4,5,6]])
print(A.shape)
B = np.array([[1,2],[3,4],[5,6]])
C = np.array([[1,2],[3,4]])
print(C.shape)
print(np.dot(A,C))

operation result:

 An error message "ValueError: shapes (2,3) and (2,2) not aligned: 3 (dim 1) != 2 (dim 0)" occurs, because in multidimensional array operations, two dot product operations The number of corresponding dimensions in the matrix must be the same .

(3) Inner product of neural network

Implement a simple neural network using numpy matrices (bias and activation functions omitted, only weights)

The following figure shows the structure of the simple neural network

 code:

X = np.array([1,2])
print(X.shape)
W = np.array([[1,3,5],[2,4,6]])
print(W)
print(W.shape)
Y = np.dot(X,W)
print(Y)

operation result:

2.3-Layer Neural Network Implementation

The 3-layer neural network structure is as follows:

 Among them, the neural network: the input layer (layer 0) has 2 neurons, the first hidden layer (layer 1) has 3 neurons, and the second hidden layer (layer 2) has 2 neurons , the output layer (layer 3) has 2 neurons

Increase the neuron "1" representing the bias, as shown in the following figure:

 (1) The weighted sum of the first layer

Using matrix multiplication, the weighted sum of the first layer can be expressed as:

Code implementation (here set the input signal, weight, and bias to any value)

X = np.array([1.0,0.5])
W1 = np.array([[0.1,0.3,0.5],[0.2,0.4,0.6]])
B1 = np.array([0.1,0.2,0.3])

A1 = np.dot(X,W1) + B1

After using the activation function sigmoid:

def sigmoid(x):
    return 1 / (1 + np.exp(-x))
X = np.array([1.0,0.5])
W1 = np.array([[0.1,0.3,0.5],[0.2,0.4,0.6]])
B1 = np.array([0.1,0.2,0.3])

A1 = np.dot(X,W1) + B1
Z1 = sigmoid(A1)

 (2) Signal transmission from the input layer to the first layer

    code:

W2 = np.array([[0.1,0.4],[0.2,0.5],[0.3,0.6]])
B2 = np.array([0.1,0.2])

A2 = np.dot(Z1,W2) + B2
Z2 = sigmoid(A2)

(3) Signal transmission from layer 1 to layer 2

   Here we define the identity_function() function (identity function) and use it as the activation function of the output layer.

(Note: The activation function used in the output layer should be determined according to the nature of the problem to be solved. Generally, the identity function can be used for regression problems, the sigmoid function can be used for binary classification problems, and the softmax function can be used for multi-class classification problems )

def identity_function(x):
    return x
W3 = np.array([[0.1,0.3],[0.2,0.4]])
B3 = np.array([0.1,0.2])

A3 = np.dot(Z2,W3) + B3
Y = identity_function(A3) 

(4) Complete code

def init_network():
    network = {}
    network['W1'] = np.array([[0.1,0.3,0.5],[0.2,0.4,0.6]])
    network['b1'] = np.array([0.1,0.2,0.3])
    network['W2'] = np.array([[0.1,0.4],[0.2,0.5],[0.3,0.6]])
    network['b2'] = np.array([0.1,0.2])
    network['W3'] = np.array([[0.1,0.3],[0.2,0.4]])
    network['b3'] = np.array([0.1,0.2])

    return network

def sigmoid(x):
     return 1 / (1 + np.exp(-x))

def identity_function(x):
     return x

import numpy as np

def forward(network, x):
    W1,W2,W3 = network['W1'],network['W2'],network['W3']
    b1,b2,b3 = network['b1'],network['b2'],network['b3']

    a1 = np.dot(x, W1) + b1
    z1 = sigmoid(a1)
    a2 = np.dot(z1, W2) + b2
    z2 = sigmoid(a2)
    a3 = np.dot(z2, W3) + b3
    y = identity_function(a3)

    return y

if __name__ == '__main__':
    network = init_network()
    x = np.array([1.0,0.5])
    y = forward(network,x)
    print(y)

Running result (output y):