tf.train.exponential_decay achieve exponential decay rate. With this function, you can use a larger learning rate to quickly get a more optimal solution, and then continue to gradually decrease as the iterative learning rate, making the model more stable in the latter part of training.
tf.train.exponential_decay：
tf.train.exponential_decay(learning_rate, global_, decay_steps, decay_rate, staircase=True/False)
Parameter Description:
- learning_rate: initial learning rate
- global_step: The current number of iterations
- decay_steps: decay rate
- decay_rate: attenuation coefficient, typically between 0-1.
- staircase = False: attenuation mode (the default is False, when is True, (global_step / decay_steps) were converted to an integer) of formula.
