mobilenet v1
Reading papers
Papers Address: https://arxiv.org/abs/1704.04861
The core idea is to replace ordinary conv by depthwise conv. About depthwise conv can refer https://www.cnblogs.com/sdu20112013/p/11759928.html
Model structure: similar to the structure of such a stack vgg.
Each layer of computation can be seen, the amount of computation is not absolutely proportional to the number of parameters , of course, the overall trend, the parameters of the model will be smaller amount of computation faster.
Code
https://github.com/marvis/pytorch-mobilenet
Network architecture:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
def conv_bn(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, oup, 3, stride, 1, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True)
)
def conv_dw(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, inp, 3, stride, 1, groups=inp, bias=False),
nn.BatchNorm2d(inp),
nn.ReLU(inplace=True),
nn.Conv2d(inp, oup, 1, 1, 0, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True),
)
self.model = nn.Sequential(
conv_bn( 3, 32, 2),
conv_dw( 32, 64, 1),
conv_dw( 64, 128, 2),
conv_dw(128, 128, 1),
conv_dw(128, 256, 2),
conv_dw(256, 256, 1),
conv_dw(256, 512, 2),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 1024, 2),
conv_dw(1024, 1024, 1),
nn.AvgPool2d(7),
)
self.fc = nn.Linear(1024, 1000)
def forward(self, x):
x = self.model(x)
x = x.view(-1, 1024)
x = self.fc(x)
return x
Dissertation reference structure, the first layer is a common layer of convolution, convolution back contact are separable.
Here noted that the usage parameter groups when the number of groups = input channel, i.e. for each channel to do the convolution. Default groups =. 1, this case is the ordinary convolution.
Training pseudo-code
# create model
model = Net()
# define loss function (criterion) and optimizer
criterion = nn.CrossEntropyLoss().cuda()
optimizer = torch.optim.SGD(model.parameters(), args.lr,
momentum=args.momentum,
weight_decay=args.weight_decay)
# load data
train_loader = torch.utils.data.DataLoader()
# train
for every epoch:
input,target=get_from_data
#前向传播得到预测值
output = model(input_var)
#计算loss
loss = criterion(output, target_var)
#反向传播更新网络参数
optimizer.zero_grad()
loss.backward()
optimizer.step()