table of Contents
1.1.1 Defects of a single layer of neurons
2 Logistic regression and cross entropy
1 Multilayer perceptron
1.1 Theoretical knowledge
1.1.1 Defects of a single layer of neurons
The problem of XOR, just look at the notes, here is the main explanation
1.1.2 Neuron inspiration
Multilayer perceptron
When the activation function is added, it is for the output to reach a certain submitted output expected value
1.2.3 Activation function:
Relu
simod and tanh can easily cause saturation problems
Leak relu is used in the deep network
1.3 code implementation
Model analysis:
analysis:
We set the output to have ten hidden units Ouput Shape (None, 10)
(3 variables multiplied by their weights) x1 w1 x2 w2 x3 w3 plus one param * 10 = 40
As shown:
Dense_1 10 hidden units plus a bias parameter 11
Next train the model, next we need to configure his optimizer
test
2 Logistic regression and cross entropy
2.1 Output binary value
Sigmoid function
Neural network is a mapping network
If you use the square difference loss function to describe the loss , the loss is generally at the same level as the original data set
Problem: There are many iterations and the training speed is slow
Therefore, the classification problem uses cross entropy (much like the information entropy of Naive Bayes)
2.2 Theoretical basis
Characterize probability
Close to 0 is very large, magnifying the loss of the probability distribution
We use strings to represent binary cross entropy
Look at the data
Because the support vector machine uses -1.1 and we use the second type of logistic regression problem here, we should use 0, -1 and add layers to it.
4 hidden units, for the first layer, the first 15 columns of the data shape activate relu
2 hidden layers + one output layer
