1. What is deep learning?
In the field of artificial intelligence, there is a method called machine learning. There is a class of algorithms in machine learning methods called neural networks:
As shown below:
Figure (1)
(Neurons) 
(connections between neurons)
Layer L1: Input layer Layer L2: Hidden layer Layer L3: Output layer
Each circle in Figure (1) is a neuron, and the connection between each neuron is the connection between neurons. The neurons above are divided into 3 layers. Neurons between layers are connected, but neurons in the same layer are not connected. The leftmost (Layer L1) is the input layer, which is responsible for receiving input data; the middle layer (Layer L2) is the hidden layer ; the rightmost layer (Layer L3) is the output layer , from which the neural network can be obtained Output Data.
There are hidden layers between the input layer and the output layer. A neural network with a number of layers (greater than 2) is called a deep neural network. Deep learning is a machine learning method using deep architecture (deep neural network).
The connection between the deep network and the shallow network :
The expression of the deep network should be intuitive and clear. A neural network with only one hidden layer can also fit a function, but in this hidden layer, a lot of neurons are needed to display what it wants to express. The deep network can fit the same function with very few neurons (relatively a single hidden layer). Therefore, when fitting a function, there are two methods to fit a single hidden layer (shallow and wide) and multiple hidden layers (deep and narrow ). But relatively speaking, the deep network saves more resources and is not easy to train. For the deep network, a large amount of data and a lot of skills are required to train a better deep network.
2. Perceptron (neuron)
Perceptron -the building blocks of neural networks ( neurons )
Perceptron definition:
(The formula can't be played, so I took a screenshot directly, which is very embarrassing.
Example: Implement and function with perceptron
Design a perceptron, let it realize and operation. and is a binary function (with two parameters X1 and X2), the truth table is as follows:
| X1 | X2 | Y |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
0 means false and 1 means true.
Let W1 = 0.5; W2 = 0.5; b = -0.8, and the activation function f is the step function written above , and the perceptron at this time is equivalent to the and function.
Substituting the above truth table into formula (1), the output is calculated, and the first line can be verified:
y = f(w*X + b)
= f(W1X1 + W2X2 + b)
= f(0.5*0+0.5*0-0.8)
=f(-0.8) =0
That is, when both X1 and X2 are 0, y is 0.
Example: implement or function with perceptron
Design a perceptron and let it implement or operation. or is a binary function (with two parameters X1 and X2), the truth table is as follows:
| X1 | X2 | Y |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
0 means false and 1 means true.
Let W1 = 0.5; W2=0.5; b = -0.3 to verify the second line:
y = f(w*X + b)
= f(W1X1 + W2X2 + b)
= f(0.5*0+0.5*1-0.8)
=f(0.2) =1
That is, when X1=0, X2=1, y is 1.
3. Other applications of perceptron
The perceptron can not only implement simple non-dual operations, it can also fit linear functions, and any linear classification or linear regression problem can be solved by the perceptron. The previous Boolean operation can be regarded as a binary classification problem, that is, given an input, output 0 (belonging to the category 0) or output 1 (belonging to the category 1). As shown below, the andoperation is a linear classification problem, that is, a straight line can be used to separate classification 0 (false, indicated by a red cross) and classification 1 (true, indicated by a green dot). However, XOR operation cannot be performed.
Four, perceptron training
Obtain the weight and bias terms:
Use the perceptron to train the algorithm: Initialize the weight and bias terms to 0, and then use the perceptron rules to iteratively update wi and b until the training is completed.
among them 
wi is the weight term corresponding to the input xi, b is the bias term (b can be regarded as the weight corresponding to the input xb whose value is always 1), t is the actual value of the sample , generally label, and y is the perceptron The output value of he is calculated by formula 1. yita is the learning rate , which controls the update range (or step length) of the weight of each step.
Each time the input vector X of one sample is taken from the training data, the perceptron is used to calculate its output y, and multiple iterations are performed according to the perceptron rules to obtain the final weight.
Finally, the code:
Perceptron class:
# -*- coding: utf-8 -*-
# !/usr/bin/env python
# @Time : 2019/4/25 15:57
# @Author : xhh
# @Desc : 机器学习入门--感知器（cell）
# @File : test_perceptorn.py
# @Software: PyCharm
from __future__ import print_function
from functools import reduce
class VectorOp(object):
"""
实现向量计算操作
"""
@staticmethod
def dot(x, y):
"""
计算两个向量的内积
1、首先把x[x1, x2, x3...] 和y[y1, y2, y3...]按元素相乘
2、变成[x1*y1, x2*y2,x3*y3]
3、然后利用reduce求和
:param y:
:return:
"""
return reduce(lambda a, b: a + b,VectorOp.element_multiply(x, y), 0.0)
@staticmethod
def element_multiply(x, y):
"""
将两个向量x和y按元素相乘
1、首先把x[x1, x2, x3...]和y[y1, y2, y3...]打包在一起
2、变成[(x1,y1),(x2, y2),(x3,y3),...]
3、然后利用map 函数计算[x1*y1, x2*y2, x3*y3]
:param y:
:return: 返回列表集合
"""
return list(map(lambda x_y: x_y[0] * x_y[1], zip(x, y)))
@staticmethod
def element_add(x, y):
"""
将两个向量x和y按元素相加
1、首先把x[x1, x2, x3...]和y[y1, y2, y3...]打包在一起
2、变成[(x1,y1),(x2, y2),(x3,y3),...]
3、然后利用map 函数计算[x1+y1, x2+y2, x3+y3]
:param y:
:return: 返回一个列表
"""
return list(map(lambda x_y: x_y[0] + x_y[1], zip(x, y)))
@staticmethod
def scala_multiply(v, s):
"""
将向量v中的每个元素和标量s相乘
:param s: 标量s
:return: 返回map
"""
return map(lambda e: e * s, v)
class Perceptron(object):
def __init__(self, input_num, activator):
"""
初始化感知器，设置输入参数的个数，以及激活函数。
激活函数的类型为double —> double
:param input_num:
:param activator:
"""
self.activator = activator
# 设置权重向量初始化为0
self.weights = [0.0]*input_num
# 偏置项初始化为0
self.bias = 0.0
def __str__(self):
"""
打印学习到的权重、偏置项
:return:
"""
return 'weight\t:%s\nbias\t:%f\n'%(self.weights, self.bias)
def predict(self, input_vec):
"""
输入向量，输出感知器的计算结果
1、计算输入的向量input_vec[x1, x2, x3...]和权重weights[w1, w2, w3,...]的内积
2、然后加上bias
:param input_vec:
:return:
"""
return self.activator(
VectorOp.dot(input_vec, self.weights)+self.bias)
def train(self, input_vecs, labels, iteration, rate):
"""
输入训练数据：一组向量、以及每个向量对应的label；以及训练数和学习率
:param input_vecs: 输入的向量
:param labels: 每个向量的标签
:param iteration: 迭代次数
:param rate: 学习率
:return:
"""
for i in range(iteration):
self._one_iteration(input_vecs, labels, rate)
def _one_iteration(self, input_vecs, labels, rate):
"""
每迭代一次，把所有的训练数据过一遍
1、把输入和输出打包在一起，生成样本列表[(input, label),...]
2、而每个训练样本(samples)都是（input_vec, label）
:param input_vecs: 输入向量
:param labels: 向量对应的标签
:param rate: 学习率
:return:
"""
samples = zip(input_vecs, labels)
# 对每个样本，感知感知器的规则更新权重
for (input_vec,label) in samples:
# 计算感知器在当前权重下的输出
output = self.predict(input_vec)
# 更新权重
self._update_weights(input_vec, output, label, rate)
def _update_weights(self, input_vec, output, label, rate):
"""
按照感知器规则更新权重
1、首先计算本次更新的delta
2、然后把input_vec[x1, x2, x3,...]向量中的每个值乘上delta
3、最后再把权重更新按元素加到原先的weights[w1, w2, w3,...]上
:param output: 输入向量
:param label: 输出向量
:param rate: 学习率
:return:
"""
delta = label - output
self.weights = VectorOp.element_add(
self.weights, VectorOp.scala_multiply(input_vec, rate * delta))
# 更新bias
self.bias += rate * delta
Implementation of the and function:
def f(x):
"""
定义激活函数
:param x:
:return:
"""
return 1 if x > 0 else 0
def get_training_dataset():
"""
基于and真值表构建训练数据
:return:
"""
# 构建训练数据， 输入向量的列表
input_vecs = [[1, 1],[0, 0], [1, 0], [0, 1]]
# 期望的输出列表
# [1, 1]—> 1, [0, 0] —>0, [1, 0] —>0, [0, 1] —>0
labels = [1, 0, 0, 0]
return input_vecs, labels
def train_and_perceptron():
"""
使用and真值表训练感知器
:return:
"""
# 创建感知器，输入参数个数为2（and是一个二元函数），激活函数为f
p = Perceptron(2, f)
# 训练，迭代10轮， 学习速率为0.1
input_vecs, labels =get_training_dataset()
p.train(input_vecs, labels, 10, 0.1)
# 返回训练好的感知器
return pTest the written perceptron:
if __name__ == '__main__':
# 训练感知器
and_perception = train_and_perceptron()
# 打印训练获得权重
print(and_perception)
# 测试
print('1 and 1 = %d' % and_perception.predict([1, 1]))
print('0 and 0 = %d' % and_perception.predict([0, 0]))
print('1 and 0 = %d' % and_perception.predict([1, 0]))
print('0 and 1 = %d' % and_perception.predict([0, 1]))operation result:
At this point, the perceptron is finished. It is for understanding and learning with reference to others. Write down your own learning process~~~~
You can pay attention to the official account of my friend and me~~~ Here are some python technical information that my friend and I update from time to time! ! You can also leave a message to discuss technical issues. I hope you can support and pay attention to it. Thank you~~
