Depth study current and future have good prospects for development. During a time when the epidemic, enrolled in the hands-on science learning pytorch depth version of the open class, hope in the future of learning in life can utilize this knowledge learned.
The first class consists of three main parts: linear regression, softmax and classification model, MLP, which mainly is an entry-depth study of understanding, simple, brief overview of what the future programming content specific pytorch again write.
Linear Regression
With housing prices and housing area and the Age of example, the basic linear regression model can be expressed as
the loss function used to measure the error of the predicted value and the true value, commonly used is the mean square error function, as follows:
After obtaining loss function, we desired loss function reaches a minimum, the approach is to first normal equation method, this method is simple derivation of function, the derivation process described ominous, this method is not very recommended. Another method is the most widely used is the gradient descent method, gradient descent function means decreases along the advancing direction of the fastest current point, the gradient descent method variety of types, SGD, Adam, RMSprop, these methods the core idea is the same, but have different iterations ways, to use for different situations.
Softmax and classification model
This section is mainly about the concept Softmax algorithm and cross-entropy loss function.
Softmax algorithm
In the multi-classification problems, on the one hand, due to the range of the output value of the output layer of uncertainty, we intuitively difficult to judge the significance of these values. On the other hand, because the real label is a discrete value, the error between the output value of the discrete value and uncertainty range of difficult to measure, so we use Softmax algorithm for multi-class classification algorithm is assumed that the output of the three categories, the specific formula is as follows:
Cross-entropy loss function
Used in the calculation of the loss before the function is the mean square error function, this method is more useful when dealing with back problems, but in classification, it is difficult to calculate the error in the desired value using this function, for proposed cross entropy loss function:
Multilayer Perceptron
MLP is essentially containing hidden layer neural network, in an example of FIG MLP is the hidden layer contains five hidden units,
calculated with a neural network algorithm for MLP and no difference, the main comprising the step of:
(1) initialize random weights w and bias B
(2) to calculate the predicted value before propagation
(3) and calculation of the loss function using a gradient descent method to optimize the most some w and b. a
Difficulties after-school title
An after-school programming question wrong question, it seems that programming still have to practice more.
effect y.view (a, b) is the y dimension becomes a * b, automatically determine when a or b is -1, when only one parameter is changed to a dimension dimensional
shape y_hat is [10,1 ]; y is the shape [10], which is one-dimensional, y_hat size and y are not the same, so there is a problem b options
So far, the first part is over, the next part is the basis of NLP.
