解读Depth Map Prediction from a Single Image using a Multi-Scale Deep Network (4)
上次把CNN卷积网络的结构作了介绍,这次仍然分析这篇博文,点击打开链接
来学习卷积神经网络的训练
训练原理:利用链式求导,计算损失函数对每一个权重的偏导数,然后根据梯度下降公式更新权重,依然是反向
传播算法
反向传播算法三个步骤:
1,前向计算每个神经元的输出值
2,反向计算每个神经元的误差项(也叫误差传递),即网络损失函数对该神经元加权输入的偏导数
3,计算每个神经元连接权重的梯度
卷积网络的训练:
1,卷积层训练:
用反向传播的原理计算filter每个权值的梯度
步长为2的情况,通过补零,转化为步长为1的情况
输入深度为D的误差传递
filter数量为N的误差传递
filter权重梯度的计算
2,Pooling层的训练:
Pooling层没有需要学习的参数,它需要做的仅仅是讲误差项传递到上一层,而没有梯度计算
写到这里,我不得不感谢这篇博客,教会我CNN的基础架构和反向传播算法
理论上大体明白了CNN网络,可以看一下CNN网络的Python实现
卷积层的初始化:
class ConvLayer(object): def __init__(self, input_width, input_height, channel_number, filter_width, filter_height, filter_number, zero_padding, stride, activator, learning_rate): self.input_width = input_width self.input_height = input_height self.channel_number = channel_number self.filter_width = filter_width self.filter_height = filter_height self.filter_number = filter_number self.zero_padding = zero_padding self.stride = stride self.output_width = \ ConvLayer.calculate_output_size( self.input_width, filter_width, zero_padding, stride) self.output_height = \ ConvLayer.calculate_output_size( self.input_height, filter_height, zero_padding, stride) self.output_array = np.zeros((self.filter_number, self.output_height, self.output_width)) self.filters = [] for i in range(filter_number): self.filters.append(Filter(filter_width, filter_height, self.channel_number)) self.activator = activator self.learning_rate = learning_rate
这段代码可以了解到:
1,参数input,channel,filter,zero_padding,stride,output的设置
2,设置filter的activator以及learning_rate
来看一下calculate_output_size()函数,
@staticmethod def calculate_output_size(input_size, filter_size, zero_padding, stride): return (input_size - filter_size + 2 * zero_padding) / stride + 1
其实就是按照计算公式实现的
来看一下Filter类,
class Filter(object): def __init__(self, width, height, depth): self.weights = np.random.uniform(-1e-4, 1e-4, (depth, height, width)) self.bias = 0 self.weights_grad = np.zeros( self.weights.shape) self.bias_grad = 0 def __repr__(self): return 'filter weights:\n%s\nbias:\n%s' % ( repr(self.weights), repr(self.bias)) def get_weights(self): return self.weights def get_bias(self): return self.bias def update(self, learning_rate): self.weights -= learning_rate * self.weights_grad self.bias -= learning_rate * self.bias_grad
从Filter类代码可以了解到:
1,filter的基本参数
2,用梯度下降法更新filter权重
明日接着分析CNN的Python代码