Q1 cost function
(1) neural network training samples assuming there are m each contains a set of inputs and a set of output signals x y, L denotes the number of layers of the neural network, S L is the number of neurons in each layer, S L representatives the last layer number of the unit process.
The cost function (not the same [theta] 0 regularization):
Q2 back-propagation algorithm
Error formula, noted that the first column as the input is not an error.
Forward-propagation algorithm:
Represents an error by δ, then [delta] (. 4) = A (. 4) -Y
Is the error of the previous layer:
Error of the previous layer then:
。
There is no error in the input layer.
After each layer have errors, respectively, the partial derivative can then update θ.
Q3 intuitive understanding backpropagation algorithm
slightly
Q4 Implementation Note: Expand the parameters
slightly
Q5 gradient test
With two connection points of the slope of a field point as the estimated value of the point, then the value of the calculated value of the neural network is compared.
Q6 random initialization
Initialization parameters should be random, if the same value, the unit will activate all of the second layer have the same value, similarly hereinafter.
Q7 together
Steps when using neural networks:
(1) network architecture: The first thing to do is to select the network structure, i.e., number of layers and the decision decided how much each respective unit.
The number of units is the number of the first layer characterized in our training set.
Finally, the number of units is the number one type of results we train set.
(2) to train the neural network:
1. The random initialization parameters;
2. Calculate all forward propagation method using h [theta] (X);
3. Write code calculates a cost function J;
4. All the partial derivatives calculated using a back propagation method;
The test method for testing these numerical partial derivatives;
6. The use of an optimization algorithm to minimize the cost function.
Q8 autopilot
slightly.