[Study Notes] Introduction to Deep Learning: Theory and Implementation Based on Python-Python Introduction and Perceptron

1. Introduction to Python

1.1 NumPy

In the implementation of deep learning, the calculation of arrays and matrices often occurs. NumPyMany convenience methods are provided in the array class (numpy.array) of , which we will use when implementing deep learning. For details on how to install various third-party libraries, see: Tutorial on installing third-party modules related to Python machine learning and data analysis in VS Code .

Import NumPy:

import numpy as np

Generate NumPy arrays:

x = np.array([1.0, 2.0, 3.0])
print(x)  # [1. 2. 3.]

NumPy array arithmetic operations:

x = np.array([1.0, 2.0, 3.0])
y = np.array([2.0, 4.0, 6.0])
x + y  # [3. 6. 9.]
x - y  # [-1. -2. -3.]
x * y  # [2. 8. 18.]
x / y  # [0.5 0.5 0.5]
x / 2.0  # [0.5 1. 1.5]，广播功能

Multidimensional NumPy arrays:

A = np.array([[1, 2], [3, 4]])
A.shape  # (2, 2)，查看A的形状
A.dtype  # dtype('int64')，查看矩阵元素的数据类型

The broadcast function is shown in the figure below:

Ways to access matrix elements:

X = np.array([[51, 55], [14, 19], [0, 4]])
X[0]  # array([51, 55])，第0行
X[0][1]  # 55，(0, 1)位置的元素
X = X.flatten()  # 将X转换为一维数组
print(X)  # [51 55 14 19 0 4]
X[np.array([0, 2, 4])]  # array([51, 14, 0])，获取索引为0、2、4的元素

from XXExtraction in$X$$1\; of5$ elements:

X > 15  # array([True, True, False, True, False, False], dtype=bool)
X[X > 15]  # array([51, 55, 19])

1.2 Matplotlib

In deep learning experiments, graph drawing and data visualization are very important. MatplotlibIt is a library for drawing graphics. Matplotlib can be used to easily draw graphics and realize data visualization.

draw $sin$ function image:

import numpy as np
import matplotlib.pyplot as plt

# 生成数据
x = np.arange(0, 6, 0.1)  # 以0.1为步长，生成[0, 6)的数据
y = np.sin(x)

# 绘制图形
plt.plot(x, y)
plt.show()

The drawing result is shown in the figure below:

add $cos$ function, and add title and$Other\; functions\; such\; as\; x-$ axis label names:

import numpy as np
import matplotlib.pyplot as plt

# 生成数据
x = np.arange(0, 6, 0.1)  # 以0.1为步长，生成[0, 6)的数据
y1 = np.sin(x)
y2 = np.cos(x)

# 绘制图形
plt.plot(x, y1, label="sin")
plt.plot(x, y2, linestyle="--", label="cos")  # 用虚线绘制
plt.xlabel("x")  # x轴标签
plt.ylabel("y")  # y轴标签
plt.title('sin & cos')  # 标题
plt.legend()
plt.show()

The drawing result is shown in the figure below:

pyplotMethods for displaying images are also provided in imshow(). Alternatively, images can be read in using matplotlib.imagethe module's methods:imread()

import matplotlib.pyplot as plt
from matplotlib.image import imread

img = imread('lena.png')  # 读入图像（设定合适的路径！这里假定图像lena.png在当前目录下）
plt.imshow(img)
plt.show()

After running the above code, the image shown below will be displayed:

2. Perceptron

2.1 What is a perceptron

A perceptron receives multiple input signals and outputs a single signal. The figure below is an example of a perceptron that receives two input signals, $x_1,x_2$is the input signal, $y$ is the output signal,$w_1,w_2$is the weight ( $w$是$weight$ ). The circles in the diagram are called "neurons" or "nodes". When the input signal is sent to the neuron, it will be multiplied by fixed weights ($w_1{x}_1,w_2{x}_2$). The neuron will calculate the sum of the transmitted signals, and only when the sum exceeds a certain threshold value, it will output $1$ . This is also known as "the neuron is activated". This boundary value is calledthe threshold, with the symbol$\theta$ represents.

The above content is expressed in a mathematical formula as shown in the following formula:

The multiple input signals to a perceptron have inherent weights that act to control the importance of each signal. That is, the larger the weight, the higher the importance of the signal corresponding to that weight.

2.2 Simple logic circuit

Now consider using a perceptron to implement $ANDgate$ , its truth table is shown in the figure below:

There are countless ways to select parameters that meet the conditions in the above figure. When $(w_1,w_2,i)=(0.5,0.5,0.7)$ , the above conditions can be met. After setting such parameters, only when$x_1$and $x_2$Simultaneously $1$ , the weighted sum of the signals will exceed the given threshold$i$ .

2.3 Realization of Perceptron

Use Python to realize the above logic circuit:

def AND(x1, x2):
	w1, w2, theta = 0.5, 0.5, 0.7
	tmp = x1 * w1 + x2 * w2
	if tmp <= theta:
		return 0
	elif tmp > theta:
		return 1

We put the θ \theta in the previous mathematical formula$\theta$ is replaced by$-b$ , the perceptron can be represented by the following formula:

Here, $b$ is calledbias,$w_1$Sum $w_2$called weight . The perception machine calculates the product of the input signal and the weight, and then adds a bias, if this value is greater than $0$ then output$1$ , otherwise output$0$。

Using weights and biases, an AND gate can be implemented like this:

def AND(x1, x2):
	x = np.array([x1, x2])
	w = np.array([0.5, 0.5])
	b = -0.7
	tmp = np.sum(w * x) + b
	if tmp <= 0:
		return 0
	else:
		return 1

Note that the biases and weights $w_1,w_2$function is different. Specifically, $w_1$Sum $w_2$is the parameter that controls the importance of the input signal, and the bias is to adjust how easily the neuron is activated (the output signal is $1$ degree) parameters.

2.4 Limitations of Perceptron

$The\; truth\; table\; of\; XORgate$ is shown in the figure below:

This XOR gate cannot be realized with the perceptron introduced earlier. Take the OR gate as an example, when the weight parameter $(b,w_1,w_2)=(-0.5,1.0,1.0)$ , the truth table condition can be satisfied, and the perceptron can be expressed by the following formula:

At this time, the perception opportunity is generated by the straight line $-0.5+x_1+x_2=0$ to separate the two spaces. One of the spaces outputs$1$ , another space outputs$0$ , as shown in the figure below:

OR gate at $(x_1,x_2)=(0,0)$ output$0$ , at$(x_1,x_2)$ is$(0,1),(1,0),(1,1)$ output$1$ . In the figure above, the circle represents$0$ , the triangle means$1$。

The output of the XOR gate is shown in the figure below:

Trying to separate the circles and triangles in the image above with a straight line is impossible anyway.

The above conditions can be achieved by using the nonlinear space divided by the curve :

2.5 Multilayer Perceptron

The cool thing about the perceptron is that it can "stack layers". XOR gates can be implemented by combining AND gates, NAND gates and OR gates, as shown in the following figure:

Assuming that all three gates have been implemented, the code to implement the XOR gate in Python is as follows:

def XOR(x1, x2):
	s1 = NAND(x1, x2)
	s2 = OR(x1, x2)
	y = AND(s1, s2)
	return y

Next, we try to express this XOR gate with the representation of a perceptron (explicitly displaying neurons). As shown in the figure below, the XOR gate is a multi-layer neural network. Here, the leftmost column is referred to as 0 $Layer\; 0$ , the middle column is called 1$1$ layer, the rightmost column is called 2nd$2$ floors.

Next section: [Study Notes] Introduction to Deep Learning: Theory and Implementation Based on Python - Neural Network .