Pytorch Notes (1) an initial neural network

First, the artificial neurons

Above artificial neurons:

1. * Enter weight -> equivalent to the human neuron dendrites function
2. Each of the inputs are summed, then a non-linear variation f -> functions as soma
3. The result of nonlinear variations in output -> equivalent axon

 

In the case of a fixed non-linear function f, different choices weights, individual neurons can perform different operations

But not all, such as: "OR"

The certification process can be expressed as: y = f (w [0] * x [0] + w [1] * w [1] + w [2])

At x [0] x [1] the case where the two inputs are 00100111, the final output y = 0 1 1 1 [OR]

Use required to prove

 

 

Second, the artificial neural network

Refers to a plurality of neurons -> Network composition

[Wherein some of the output of the neuron as will some other input neurons]

 

 For example, with two weights built of different neuronal or neural network operation

Neural network can simulate all possible operations

Proof: (., 0) of the neural network is a nonlinear function f () = max of neurons can simulate any continuous piecewise linear function on the closed interval

If the input and output is not directly related to a piecewise linear function, it can not be modeled using neural networks? ---- of course can

 

Reason: any input / output relationship can be approximated by piecewise linear function as long as enough segment points, can function very accurately approximated by piecewise function. The piecewise linear function, can be set up by the artificial neural network to obtain ----> If the number of neurons in an artificial neural network enough, the relationship between neurons sufficiently complex, can be very accurately simulate any input / output relationship [i.e., universal approximation Theorem ]

 

 

 Third, the design and the right to re-learning neural network 

1. 神经网络结构的确定：    神经元个数越多，链接越复杂，能便是的 输入/输出 关系越多，对特定 输入/输出关系的表达 就 越精确，但也就越难找到 最合适的权重------最优的输入/输出关系   【中间 存在 折中关系】
2. 神经网络中神经元权重的确定：         权重的选取  可以看作  是一个 优化问题 。  对于每一组确定的权重值，我们可以确定出优化问题的 收益或损失。当权重不合适时，优化问题的收益笑，损失大；权重合适时，优化问题的收益大，损失小   【通过调节权重，最大化收益，最小化损失，就可以得到合适的权重】