Reference: https://developers.google.com/machine-learning/crash-course/reducing-loss/an-iterative-approach
iterative approach
Machine learning algorithms for training model of iterative trial and error process:
We will use the same method throughout the iterative machine learning crash course in detail a variety of complex situations, especially in the blue region of the storm cloud. Application of Iterative strategy in machine learning is very popular, mainly because they can scale well to large data sets.
"Model" section, one or more features as input, and returns a prediction (y ') as an output. For simplicity, consider a feature of using one kind of a prediction model and returns:
What we should be set to an initial value of b and w1? For linear regression problem, the fact that the initial value is not important. We can randomly select a value, but we chose to adopt the following values irrelevant:
- b = 0
- w1 = 0
Assume that the first characteristic value is 10. The characteristic values are substituted into the prediction function will get the following results:
Figure "Calculated loss" section is to be used to model loss function . Suppose we use the quadratic loss function. Loss of function takes two input values:
- Model predictive feature of x: y '
- y: x corresponding to the correct label feature
Finally, we look at the chart "Calculation parameter update" section. Machine learning system in this section is to check the value of the loss function, and generate new values for b and w1. Now, assume that the mysterious green block produces a new value, then the machine learning system will re-evaluate all features from all labels, generates a new value for the loss function, which in turn produces a new value parameter. This learning process will continue iterated until the algorithm find the lowest possible loss model parameters. Usually, you can continue to iterate until the total loss will not change, or at least change very slowly so far. At this time, we can say that the model has been convergence .
Summary
When training machine learning models, first of all weights and biases the initial guess, and then repeatedly adjust these speculations, until the weight loss may be the lowest weights and biases so far.