Make Your Own Neural Network
Build your own neural network
Author: lz0499
statement:
1) Make Your Own Neural Network translated from the Neural Network Introductory book written by Tariq Rashid . The author's goal is to use as little jargon and advanced mathematics as possible to explain how neural networks work in a graphic and textual way. Anyone with a high school math level can understand how neural networks work. This book is highly recommended for beginners as an introductory book on neural networks.
2) This article is for academic exchange only, not for commercial use. The original intention of translation is to deepen the understanding of neural networks while translating.
3) Due to the new knowledge of neural networks, some mistakes are inevitable in the translation process. If you find any mistakes, please correct me. Thanks!
4) Due to work reasons, I will selectively update the translation according to the original chapters from time to time.
5) This belongs to the first version. If there are errors, it is necessary to continue to correct and add or delete.
content:
Part 1: How Neural Networks Work
A single classifier does not seem to be enough
Neurons, the computers of nature
Signals through the neural network
Using a matrix to calculate the output of a three-layer neural network
Update weights from multiple nodes
Backpropagating errors from multiple nodes
Multilayer Neural Network Layer Backpropagation Error
Calculating Backward Propagation Error Using Matrix Multiplication
How to actually update the weights (1)
How to actually update the weights (2)
Update weights from multiple nodes
Previously, we obtained a simple linear classifier by changing the slope of the linear function: by using the error between the value calculated by the linear function and the training sample value, adjust the size of the slope, so that the value output by the linear function is getting closer and closer to the training sample value. It is actually proved that there is a relatively simple relationship between the error and the slope, and it is easy to calculate the slope close to the sample training value through the error.
When the output and its error are affected by multiple nodes, how do we update the weight value between each node and the output node? The image below reveals the problem:
果输出节点的输出结果只收一个输入节点的影响，那计算很简单。但是，当输出节点的输出结果受到不止一个输入节点的影响时，我们如何利用输出结果的误差调整对应的权重呢？
用所有误差值用于更新一个权重值显然是不合适的。因为此时忽视了其他节点对输出节点的影响。输出节点的误差是由多个输入节点影响的，而不止是受一个节点的影响。
输出结果的误差只受到一个节点的影响这种情况概率很小。
一种想法是把误差平分给受那些起作用的节点。如下图所示：
另一种想法是我们把误差给那些作用比较大的相连节点，而不是把误差平分给那些起作用的节点。为什么呢？因为这些节点对误差的影响更大。下图表示的这种思考方法：
上图中，输出节点受到两个输入节点的影响。两个输入节点分别以3.0和1.0的权重与输出节点相连。如果我们按照相连输入节点的权重按比例分割误差，我们可以知道3/4的误差应该用于更新第一个较大的权重；1/4的误差大小应该用于更新第二个比较小的权重值。
对于众多节点的情况，我们可以延拓这种思维方式。假设我们有100个节点与输出节点相连，
那么，其误差应该使用每一个节点占100个节点权重的比例对其权重进行更新。
可以看到，我们用了两种方式使用权重。首先，我们利用权重把输出信号传输至神经网络中，我们在之前的小节中已经计算过；其次，我们利用权重把误差从输出后向传输至神经网络中。因此，这种方法也被称之为反向传输发。
如果输出层有两个节点，我们只需要对每一个节点进行相同的操作即可。输出层的第二个节点将有自己的误差，同样是按照相连节点的权重比例进行分割反向传输到神经网络中。
下一小节中，我们将详细介绍如何把误差反向传输至神经网络中。