Machine learning----perceptron (the basis of neural network)

What is a perceptron?

There are two major problems in machine learning: classification and regression:

Regression predicts linear problems such as house prices and rainfall per year.
Classify the object, and then output the output to determine whether it is a banana or an apple.

感知机是神经网络的基础

introduce problems

Insert image description here

The redder the color and rounder the shape are,
the yellower the apple is, and the longer the shape is the banana.

This is a scatter plot of data distribution. How can we find a line to classify bananas and apples?

Insert image description here

decision boundary

y=mx + b is the data predicted by linear regression are all points distributed on the line
The classification problem generates a decision line, above which is an output (apple), and below the decision line is another output (banana).

决策边界线公式： $m_1x_1 + m_2x_2 + b = 0$

This line is the decision boundary. By bringing the coordinate points into the decision boundary line, we can get a result : $score = m_1x_1 + m_2x_2 + b$
If the calculated result is greater than 0, it means it is above the decision boundary, otherwise it is below :
$=\begin{cases}score >0:Apple\ \ score <0:banana\end{cases}$
Change to matrix operation : $\begin{bmatrix} w_1\\ w_2 \\ b \end{bmatrix}$

$\begin{bmatrix} x_1 & x_2 & 1 \end{bmatrix}$

$\\ Feature * Weight < 0 => Banana$

$\overline {f(x_1,x_2)}=\begin{cases}1\left( FeatureWeight >0\right) \\ 0\left( FeatureWeight <0\right) \end{cases}$

Calculation of decision boundaries in three-dimensional space :

Insert image description here
$\begin{bmatrix} w_1 \\ w_2 \\ w_3 \\ b \end{bmatrix}$

$\overline {f(x_1,x_2,x_3)}=\begin{cases}1\left( FeatureWeight >0\right) \\ 0\left( FeatureWeight <0\right) \end{cases}$

1. Activation function

When calculating whether it is above or below the decision boundary, the output cannot be completely 1 or 0, but may be positive infinity or negative infinity.
We only need to output 1 (apple) or 0 (banana). 激活函数The function is 结果缩小范围到1（苹果）或0（香蕉）

to correctly classify the perceptron? (Introduced through code fake data)

import matplotlib.pyplot as plt
import numpy as np
# 定义正确的输入和输出 x 表示颜色(黄--->红), y 表示形状(方形-->圆形)
test_inputs = [(0,0),(0,1),(1,0),(1,1)]
# 定义正确的结果  1 表示苹果, 0 表示香蕉
correct_outputs = [0,0,0,1]

# 随机指定3个数
weight1 = 0.8
weight2 = 0.8
bias = -0.5
# 方程 :weight1 * x1 + weight2 * x2 + bias = 0
X = np.linspace(-5,5,10)
Y = -(weight1*X + bias)/weight2

# 画图
fig = plt.figure(figsize=(5,5),dpi=100)
plt.xlim(-1,4)
plt.xlabel("yellow--->red")
plt.ylim(-1,4)
plt.ylabel("rect--->circle")

# 绘制一条直线
plt.plot(X,Y)
for index,item in enumerate(test_inputs):
    # 为了方便观察数据,当它为苹果的时候,图标显示红色
    if correct_outputs[index] == 1:
        plt.scatter(item[0],item[1],c="red")
    else:
        plt.scatter(item[0],item[1],c="yellow")

plt.show()

No trained decision boundaries:
Insert image description here

# 激活函数
def activateFunction(value):
    if value > 0:
        return 1
    else:
        return 0
for index,item in enumerate(test_inputs):
	# 根据公式测试预测值
    result = weight1 * item[0] + weight2 * item[1] + bias
    # 调用激活函数  如果跟真实结果一样就是分类成功
    if(activateFunction(result) == correct_outputs[index]):
        print("分类成功，预测结果和真实结果一样")
    else:
        print("分类失败")

The two points (0, 1) and (1, 0) were not classified successfully:

Insert image description here

2. Obtain the moving process of the perceptron decision boundary line:

Insert image description here

$0.8x_1 + 0.8x_2 - 0.5 = 0$ straight line needs to be close to the coordinates (0,1) and (1,0)

原函数先减去(0,1,1) 让直线靠近(0,1)坐标点，第三个数字1是为了给b补位

$0.8 - 0)x_1 + (0.8 - 1)x_2 + (-0.5 -1) = 0$

$0.8x_1 - 0.2x_2 - 1.5 = 0$

The process of the perceptron is to keep looping this operation so that the values of m1, m2 and b find the optimal solution.

Moving decision boundaries

If point (p,q) is classified correctly, do nothing
If point (p,q) is not classified correctly
The classification result is banana, but it is actually an apple. Point below the line and the line moves upward. Learning rate $\alpha$ , $w_1,w_2,b)$ minus ( $\alpha p$ , $\alpha q$ , $\alpha*1$ )
( $w_{1}$ = $w_{1}$ - $\alpha p$ , $w_{2}$ = $w_{2}$ - $\alpha q$ ,b=b- $\alpha$ *1)
Classified as an apple, it is actually a banana. Point above the line and the line moves downward. Learning rate $\alpha$ ,
( $w_1$ , $w_2$ ,b) plus ( $\alpha p$ , $\alpha q$ , $\alpha$ *1)
( $w_{1}$ = $w_{1}$ + $\alpha p$ , $w_{2}$ = $w_{2}$ + $\alpha q$ ,b=b+ $\alpha*1$ )
Update the values of w1, w2 and b
Guess the middle value is (0.5,0.5)
Move down according to the formula line
w1' = w1 + α * 0.5
w2' = w2 + α * 0.5
b' = b + α * 1

# 实现感知机
import numpy as np
import matplotlib.pyplot as plt
data = np.loadtxt("data_ganziji.csv",delimiter=",")
# 数据中香蕉和苹果点的x坐标和y坐标
X = data[:,0:2]
# 数据第三列为分类 香蕉or苹果
Y = data[:,2:3]
# 创建自定义 规定大小
fig = plt.figure(figsize=(5,5),dpi=80)
# 遍历数据的长度
for i in range(len(Y)):
    # 如果第三列是0那么就是香蕉
    if Y[i] == 0 :# 香蕉
        # 根据坐标画出黄色点
        plt.scatter(X[i][0],X[i][1],c="yellow")
    else:
        # 反之画出红色点
        plt.scatter(X[i][0],X[i][1],c="red")

# 随机w1和w2的值
np.random.seed(42)
W = np.random.rand(2,1)
b = -1
w1 = W[0,0]
w2 = W[1,0]
# 创建线的x坐标
XX = np.linspace(-2,2,10)
# 根据公式求y坐标  m1 * x2 + m2 * x2 + b = 0
YY = -(w1*XX + b)/w2
# 画出散点图和决策边界线
plt.plot(XX,YY)
plt.xlim(0,1)
plt.ylim(0,1)

# 定义学习率
learning_rate = 0.01
# 更新w1、w2和b的值 猜测中间值为(0.5,0.5) 根据公式线往下移 w1' = w1 + α * p 
w1 = w1 + learning_rate * 0.5
w2 = w2 + learning_rate * 0.5
b = b + learning_rate * 1
YY2 = -(w1 * XX +b) / w2
plt.plot(XX,YY2)
plt.show()

Insert image description here

At this time, you can see that the decision boundary line is indeed moving towards the middle. You just need to keep looping until the decision boundary line fits to a perfect position.

Perceptron implementation

Implementation steps:

Randomly generate w1, w2 variables and custom b variables
Traverse all data and apply formulas using the current w1, w2 and b variables to calculate
1. Put the calculated result into the activation function
2. If it is greater than 0, it is directly 1; if it is less than 0, it is 0.
3. A value of 1 is a category and a value of 0 is a category.
Determine whether the classification calculated using the current w1, w2 and b variables is the same as the true value
1. If it's the same, there's nothing to do
2. If they are different, determine whether the decision boundary line should move up or down.
3. To move up, use the subtraction formula. To move down, use the addition formula.
Update the value of each variable
Repeat steps 2 and 3 until the decision boundary line can make the correct classification
You can draw scatter plots and line plots to see whether the decision boundary converges to a suitable position.

# 实现感知机
import numpy as np
import matplotlib.pyplot as plt
# 加载数据
data = np.loadtxt("data_ganziji.csv",delimiter=",")

# 数据中香蕉和苹果点的x坐标和y坐标
X = data[:,0:2]
# 数据第三列为分类 香蕉or苹果
Y = data[:,2:3]

# 创建自定义画图窗口 规定大小
fig = plt.figure(figsize=(5,5),dpi=80)

# 遍历数据的长度
for i in range(len(Y)):
    # 如果第三列是0那么就是香蕉
    if Y[i] == 0 :# 香蕉
        # 根据坐标画出黄色点
        plt.scatter(X[i][0],X[i][1],c="yellow")
    else:
        # 反之画出红色点
        plt.scatter(X[i][0],X[i][1],c="red")

# 随机w1和w2的值
np.random.seed(42)
W = np.random.rand(2,1)
b = -1
# 创建线的x坐标
XX = np.linspace(-2,2,10)

plt.xlim(0,1)
plt.ylim(0,1)



def step_func(value):
    """
    激活函数  如果大于0就输出1 反之输出0 （0为香蕉，1为苹果）
    @param value: 用w1、w2和b的值算出来的分类 如果大于0就是在决策边界线上面，反之在决策边界下面，用来判断是香蕉还是苹果
    @return: 返回1或0（0为香蕉，1为苹果）
    """
    if value >= 0:
        return 1
    else:
        return 0
        
def perception(X,W,b):
	"""
	检测函数
	@param X: 数据中每一个点的x坐标和y坐标
	@param W: 随机出来的w1变量和w2变量
	@param b: 自定义的b变量
	"""
	# 矩阵相乘  y分类 = w1 * x1 + w2 * x2 + b
    value = (np.matmul(X,W) + b)[0]
    # 调用激活函数  把激活函数的返回值返回
    return step_func(value)

def perceptronStep(X,Y,W,b,learning_rate):
	"""
	修改参数函数
	@param X: 数据中每一个点的x坐标和y坐标的二维数组
	@param Y: 数据的分类 （0为香蕉，1为苹果）
	@param W: 随机出来的w1变量和w2变量
	@param b: 自定义的b变量
	@param learning_rate: 学习率
	"""
	# 有多少条数据遍历多少次
    for i in range(len(X)):
    	# 调用检测函数 使用目前的w1、w2和b的值进行分类
        y_pred = perception(X[i],W,b)
        # 获取到真实值
        y_real = Y[i]
        # 如果真实值为1 预测值为0
        # 检测的为香蕉  真实为苹果  线在当前点的上方  往下移动  使用加法
        if y_real - y_pred == 1:
        	# 使用公式修改w1、w2和b的值
        	# w1' = w1 + x1 * learning_rate
            W[0] += X[i,0] * learning_rate
            # w2' = w2 + x2 * learning_rate
            W[1] += X[i,1] * learning_rate
            # b' = b + learning_rate * 1
            b += learning_rate *1
        # 反之  线在当前点的下方  往上移动  使用减法
        elif y_real - y_pred == -1:
            W[0] -= X[i,0] * learning_rate
            W[1] -= X[i,1] * learning_rate
            b -= learning_rate *1
        else: # 如果真实值和预测值相等 什么都不用干
            pass
    # 把更新后的W变量和b变量返回
    return W,b

def trainPerceptron(X,Y,W,b,learning_rate,num_epochs):
	"""
	训练函数
	@param X: 数据中每一个点的x坐标和y坐标的二维数组
	@param Y: 数据的分类 （0为香蕉，1为苹果）
	@param W: 随机出来的w1变量和w2变量
	@param b: 自定义的b变量
	@param learning_rate: 学习率
	@param num_epochs: 学习次数 纪元
	"""
	# 用于存放训练完成之后的变量
    w1,w2,f_b = 0,0,0
    # 遍历学习次数
    for i in range(num_epochs):
    	# 调用修改参数的函数
        W,b = perceptronStep(X,Y,W,b,learning_rate)
        w1 = W[0]
        w2 = W[1]
        f_b = b
    return w1,w2,f_b
# 获取到训练完的参数
w1,w2,f_b = trainPerceptron(X,Y,W,b,learning_rate=0.01,num_epochs=30)
# 使用参数预测分类
a = w1 * 1 + w2 * 0.4 + b
print(a)
if a[0]<0:
    print('香蕉')
elif a[0] > 0:
    print('苹果')

Insert image description here
Dynamic display of the movement of the decision boundary line:

Insert image description here

Why Neural Network Basics?

Implementing a perceptron is implementing a single neuron
Receive input ➡ Calculate within the neuron ➡ Modify the variable and output the result
A neural network is a combination of multiple perceptrons

Single neuron:
Insert image description here
Multiple perceptrons combined together form a neural network: