目标函数就是我们常说的损失函数，优化函数就是我们常说的反调参数的函数，包括：梯度下降函数、随机梯度下降函数等。
这些我相信大家都很清楚了，下面我就简单的介绍一下keras提供的常见的目标函数和优化函数，本人能力有限，平时用到的酒多说几句，用不到的就把名字写出来，欢迎大家补充和指正。
目标函数:
keras文档: http://keras.io/objectives/
- mean_squared_error / mse 均方误差，常用的目标函数，公式为((y_pred-y_true)**2).mean()
- mean_absolute_error / mae 绝对值均差，公式为(|y_pred-y_true|).mean()
- mean_absolute_percentage_error / mape公式为：(|(y_true - y_pred) / clip((|y_true|),epsilon, infinite)|).mean(axis=-1)
* 100，和mae的区别就是，累加的是（预测值与实际值的差）除以（剔除不介于epsilon和infinite之间的实际值)，然后求均值。 - mean_squared_logarithmic_error / msle公式为： (log(clip(y_pred, epsilon, infinite)+1)- log(clip(y_true, epsilon,infinite)+1.))^2.mean(axis=-1)，这个就是加入了log对数，剔除不介于epsilon和infinite之间的预测值与实际值之后，然后取对数，作差，平方，累加求均值。
- squared_hinge 公式为：(max(1-y_true*y_pred,0))^2.mean(axis=-1)，取1减去预测值与实际值乘积的结果与0比相对大的值的平方的累加均值。
- hinge 公式为：(max(1-y_true*y_pred,0)).mean(axis=-1)，取1减去预测值与实际值乘积的结果与0比相对大的值的的累加均值。
- binary_crossentropy: 常说的逻辑回归, 就是常用的交叉熵函数
- categorical_crossentropy: 多分类的逻辑， 交叉熵函数的一种变形吧，没看太明白
详细请参照theano的官方文档，
http://deeplearning.net/software/theano/library/tensor/nnet/nnet.html#tensor.nnet.categorical_crossentropy
优化函数：
keras文档：
下面是一个自定义的优化器的一个demo：
- model = Sequential()
- model.add(Dense(64, init=‘uniform’, input_dim=10))
- model.add(Activation(’tanh’))
- model.add(Activation(’softmax’))
- sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
- model.compile(loss=’mean_squared_error’, optimizer=sgd)
model = Sequential()
model.add(Dense(64, init='uniform', input_dim=10))
model.add(Activation('tanh'))
model.add(Activation('softmax'))
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mean_squared_error', optimizer=sgd) - model.compile(loss=‘mean_squared_error’, optimizer=‘sgd’)
model.compile(loss='mean_squared_error', optimizer='sgd')- keras.optimizers.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False)
- Stochastic gradient descent, with support for momentum, learning rate decay, and Nesterov momentum.
- <span style=”color:#ff0000;”>随机梯度下降算法， 看到这个名字是不是很熟了</span>
- Arguments
- lr: float >= 0. Learning rate.
- momentum: float >= 0. Parameter updates momentum.（<span class=“s1” style=“font-family: Arial, Helvetica, sans-serif;”>参数更新的动量， 我也不知道啥意思</span><span style=“font-family: Arial, Helvetica, sans-serif;”>）</span>
- decay: float >= 0. Learning rate decay over each update.（学习率衰减量，具体是每次都衰减）
- nesterov: boolean. Whether to apply Nesterov momentum.（是否启用nesterov动量）
keras.optimizers.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False)
Stochastic gradient descent, with support for momentum, learning rate decay, and Nesterov momentum.
<span style="color:#ff0000;">随机梯度下降算法， 看到这个名字是不是很熟了</span>
Arguments
lr: float >= 0. Learning rate.
momentum: float >= 0. Parameter updates momentum.（<span class="s1" style="font-family: Arial, Helvetica, sans-serif;">参数更新的动量， 我也不知道啥意思</span><span style="font-family: Arial, Helvetica, sans-serif;">）</span>
decay: float >= 0. Learning rate decay over each update.（学习率衰减量，具体是每次都衰减）
nesterov: boolean. Whether to apply Nesterov momentum.（是否启用nesterov动量）上面好多名词都不明白什么意思，应该把实现的源码贴出来大家看一下
- class SGD(Optimizer):
- ””’Stochastic gradient descent, with support for momentum,
- learning rate decay, and Nesterov momentum.
- # Arguments
- lr: float >= 0. Learning rate.
- momentum: float >= 0. Parameter updates momentum.
- decay: float >= 0. Learning rate decay over each update.
- nesterov: boolean. Whether to apply Nesterov momentum.
- ”’
- def __init__(self, lr=0.01, momentum=0., decay=0., nesterov=False,
- *args, **kwargs):
- super(SGD, self).__init__(**kwargs)
- self.__dict__.update(locals())
- self.iterations = K.variable(0.)
- self.lr = K.variable(lr)
- self.momentum = K.variable(momentum)
- self.decay = K.variable(decay)
- def get_updates(self, params, constraints, loss):
- grads = self.get_gradients(loss, params)
- lr = self.lr * (1. / (1. + self.decay * self.iterations))
- self.updates = [(self.iterations, self.iterations + 1.)]
- # momentum
- self.weights = [K.variable(np.zeros(K.get_value(p).shape)) for p in params]
- for p, g, m in zip(params, grads, self.weights):
- v = self.momentum * m - lr * g # velocity
- self.updates.append((m, v))
- if self.nesterov:
- new_p = p + self.momentum * v - lr * g
- else:
- new_p = p + v
- # apply constraints
- if p in constraints:
- c = constraints[p]
- new_p = c(new_p)
- self.updates.append((p, new_p))
- return self.updates
- def get_config(self):
- return {“name”: self.__class__.__name__,
- ”lr”: float(K.get_value(self.lr)),
- ”momentum”: float(K.get_value(self.momentum)),
- ”decay”: float(K.get_value(self.decay)),
- ”nesterov”: self.nesterov}
class SGD(Optimizer):
'''Stochastic gradient descent, with support for momentum,
learning rate decay, and Nesterov momentum.
# Arguments
lr: float >= 0. Learning rate.
momentum: float >= 0. Parameter updates momentum.
decay: float >= 0. Learning rate decay over each update.
nesterov: boolean. Whether to apply Nesterov momentum.
'''
def __init__(self, lr=0.01, momentum=0., decay=0., nesterov=False,
*args, **kwargs):
super(SGD, self).__init__(**kwargs)
self.__dict__.update(locals())
self.iterations = K.variable(0.)
self.lr = K.variable(lr)
self.momentum = K.variable(momentum)
self.decay = K.variable(decay)
def get_updates(self, params, constraints, loss):
grads = self.get_gradients(loss, params)
lr = self.lr * (1. / (1. + self.decay * self.iterations))
self.updates = [(self.iterations, self.iterations + 1.)]
# momentum
self.weights = [K.variable(np.zeros(K.get_value(p).shape)) for p in params]
for p, g, m in zip(params, grads, self.weights):
v = self.momentum * m - lr * g # velocity
self.updates.append((m, v))
if self.nesterov:
new_p = p + self.momentum * v - lr * g
else:
new_p = p + v
# apply constraints
if p in constraints:
c = constraints[p]
new_p = c(new_p)
self.updates.append((p, new_p))
return self.updates
def get_config(self):
return {"name": self.__class__.__name__,
"lr": float(K.get_value(self.lr)),
"momentum": float(K.get_value(self.momentum)),
"decay": float(K.get_value(self.decay)),
"nesterov": self.nesterov}
这篇文章写的已经很好了： http://blog.csdn.net/luo123n/article/details/48239963
时间紧迫，以后在学习