http://machinelearningmastery.com/using-learning-rate-schedules-deep-learning-models-python-keras/
https://stackoverflow.com/questions/39779710/setting-up-a-learningratescheduler-in-keras (打印出每一个周期的学习率lr的代码)
https://github.com/fchollet/keras/issues/898
直接使用命令设置
import keras.backend as K
sgd = SGD(lr=0.1, decay=0, momentum=0.9, nesterov=True)
K.set_value(sgd.lr, 0.5 * K.get_value(sgd.lr))
或者编写回调类
class decay_lr(Callback): ''' n_epoch = no. of epochs after decay should happen. decay = decay value ''' def __init__(self, n_epoch, decay): super(decay_lr, self).__init__() self.n_epoch=n_epoch self.decay=decay def on_epoch_begin(self, epoch, logs={}): old_lr = self.model.optimizer.lr.get_value() if epoch > 1 and epoch%self.n_epoch == 0 : new_lr= self.decay*old_lr k.set_value(self.model.optimizer.lr, new_lr) else: k.set_value(self.model.optimizer.lr, old_lr) decaySchedule=decay_lr(10, 0.95)
With TF backend, I did this (for inception-V3)
from keras.callbacks import LearningRateScheduler
def scheduler(epoch):
if epoch%2==0 and epoch!=0:
lr = K.get_value(model.optimizer.lr)
K.set_value(model.optimizer.lr, lr*.9)
print("lr changed to {}".format(lr*.9))
return K.get_value(model.optimizer.lr)
lr_decay = LearningRateScheduler(scheduler)
model.fit_generator(train_gen, (nb_train_samples//batch_size)*batch_size,
nb_epoch=100, verbose=1,
validation_data=valid_gen, nb_val_samples=val_size,
callbacks=[lr_decay])
以上都是以epoch为周期的,其实每一次minibatch就算一次update(例如model.train_on_batch()), iteration的状态+1, 这就是学习率中decay作用时的iteration数值,并不等于epoch. 一个epoch有多少minibatch,就有多少iteration