Make Your Own Neural Network
Build your own neural network
Author: lz0499
statement:
1) Make Your Own Neural Network translated from the Neural Network Introductory book written by Tariq Rashid . The author's goal is to use as little jargon and advanced mathematics as possible to explain how neural networks work in a graphic and textual way. Anyone with a high school math level can understand how neural networks work. This book is highly recommended for beginners as an introductory book on neural networks.
2) This article is for academic exchange only, not for commercial use. The original intention of translation is to deepen the understanding of neural networks while translating.
3) Due to the new knowledge of neural networks, some mistakes are inevitable in the translation process. If you find any mistakes, please correct me. Thanks!
4) Due to work reasons, I will selectively update the translation according to the original chapters from time to time.
5) This belongs to the first version. If there are errors, it is necessary to continue to correct and add or delete.
content:
Part 1: How Neural Networks Work
A single classifier does not seem to be enough
Neurons, the computers of nature
Signals through the neural network
Using a matrix to calculate the output of a three-layer neural network
Update weights from multiple nodes
Backpropagating errors from multiple nodes
Multilayer Neural Network Layer Backpropagation Error
Calculating Backward Propagation Error Using Matrix Multiplication
How to actually update the weights (1)
How to actually update the weights (2)
Part 1 How Neural Networks Work
A simple prediction machine
Imagine a primitive machine that takes in a question, "thinks" about it, and outputs an answer. As in the previous example, we input corresponding visual signals through our eyes, and then analyze the acquired visual information through our brains, and finally draw corresponding conclusions. For example, what objects are in the visual scene. The diagram below illustrates this thought process:
Computers don't think like humans, so we describe the "thinking" process of the above computer in more appropriate terms as shown in the following diagram:
计算机获取一些输入信号，经过一些计算最后输出一个答案。如下图所示，假设计算机输入是一个“3*4”的命令，计算机可能把相应的乘法操作转换为比较简单的一系列加法操作，最后输出答案12.
“就这么简单”，你可能会纳闷。是的。我们先通过这种简单而熟悉的例子引入一系列后续我们将要学习的神经网络知识概念。
那我们把问题稍稍复杂化！
想象一台机器能够把千米换算成米，如下图所示：
假设我们并不知道千米转换成米的公式。我们仅仅知道千米和米之间存在线性关系。这意味着，当我们成倍改变以米为单位的一个距离数值时，那么同样的距离，以千米为单位时也将成倍的改变。这个很好理解。这千米和米之间线性关系给我们的直觉感受。
千米和米两者之间的这种线性关系，给了我们如何对两者进行转换的一个线索。即“米=千米*c”，c是一个常量。现在我们并不知道这个常量值是多少。
另外一条线索是，我们有两组千米和米相对应数值的表格。如下所示：
测试例子 |
千米 |
米 |
1 |
0 |
0 |
2 |
100 |
62.137 |
利用上面的一些线索，我们该如何计算出常数c的值呢？让我们随机取一个值。就假设c=0.5,然后我们看看会发生什么！
因此，我们有关系式米=千米*c。假设千米为100，c为0.5，则输出50米。
Ok.对于随机取的C=0.5来说，结果还不算坏。但是，我们知道这不是一个正确的数值，因为在上述表格第二个例子告诉我们答案应该是62.137.
我们差了12.137.这是误差，是由计算出来的数值和真实数值之间的差值。即
误差=实际值-计算值
=62.137-50
=12.137
有误差，我们该如何是好呢？我们知道我们计算出来的值是错误的，而且还错的比较大。与其瞪着这个误差露出绝望的神情，我们还不如利用这个误差做点什么。我们可以利用这个误差作为指导再重新计算c值.
我们再次观察这个误差值，由于千米和米之间存在线性的关系：米= 千米*c，因此，当我们增加常量c时，输出也应该同样增加。
让我们把c值从0.5增加到0.6，然后看看会发生什么。
现在c设置为0.6，所以米 = 千米*0.6= 60。嗯，这个答案比第一次的答案更加接近实际值。我们显然进步了一点。
现在的误差2.137已经小很多了.是否我们还可以再进一步减小误差呢？
从上述推到过程中我们使用了误差对c值进行调整。我们想把输出50再提高一点，所以我们提高了一点c值。
让我们放下用代数的思考方式计算c的确切值，转而继续使用上一步中的方式改变c值。我们在按照上一步的做法再做一遍。输出值60仍然比实际值要小。那么，我们继续改变c的值，由0.6改变到0.7，看看会发生什么！
噢，糟糕！我们走的太远，计算出来得值已经超过了正确的值。之前的误差是2.137，而现在的误差是-7.863。负号表示的是我们越过了真实值，记住误差的计算公式是：误差=真实值-计算值。
好吧！既然c = 0.6要好于c = 0.7，我们应该在c取0.6的时候感到高兴，毕竟误差已经比较小了，并且准备结束计算。但是，我们再想想，0.6到0.7之前还有许多数，最佳的答案会不会介于它们两者之间呢？那为什么我们不再小一点改变c值呢？比如所从0.6改变为0.61。很好，那我们继续尝试之前的方法，看看结果如何？
喔！结果很不错呀！比之前的结果更加好。计算出来的值61，与实际值62.137仅仅相差1.137。
因此最后一次尝试告诉我们，我们应该缓慢的改变c值，已达到较好的结果。如果输出的计算值已经接近实际值，即误差将会越来越小时，我们不应该较大的增加c值，这样我们就避免了计算出来得值超过了实际值，就像我们之前计算的那样。
We calculated a good value of c without much effort. Through the above calculations, we should form this awareness: little by little, continuously change the value, so that we will get a more accurate answer.
What we just did, you might find it too simple. But it is actually very close to the core computing method of neural network learning.
We should summarize: the problem we just solved is not like solving a math or science problem in one step at school. Instead, we took a different approach, which we call iterative, that is, iterating over and over again, getting closer to the right answer bit by bit.