Table of contents
- introduction
- Solver's process:
- train test model
- parameter
introduction
The solver is the core of the core of caffe, which coordinates the operation of the entire model. One of the parameters that must be carried by the caffe program is the solver configuration file. The running code is generally
# caffe train --solver=*_slover.prototxt
In Deep Learning, the loss function is often non-convex, and there is no analytical solution. We need to solve it through optimization methods. The main function of the solver is to alternately call the forward (forward) algorithm and the backward (backward) algorithm to update the parameters, so as to minimize the loss, which is actually an iterative optimization algorithm.
Up to the current version, caffe provides six optimization algorithms to solve the optimal parameters, which can be selected by setting the type in the solver configuration file.
- Stochastic Gradient Descent (
type: "SGD"), - AdaDelta (
type: "AdaDelta"), - Adaptive Gradient (
type: "AdaGrad"), - Adam (
type: "Adam"), - Nesterov’s Accelerated Gradient (
type: "Nesterov") and - RMS plug (
type: "RMSProp")
For the specific introduction of each method, please see the next article in this series. This article focuses on the preparation of solver configuration files.