ShuffleNetV2:设计轻量化卷积神经网络的理论准则和应用实现
参考论文：ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
作者：Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, Jian Sun;
上一篇我们复现过ShuffleNetV1架构，今天来复现ShuffleNetV2架构
1、论文摘要
目前，神经网络架构设计主要由计算复杂度的间接度量，即 FLOPs 指导。然而，直接指标（例如速度）还取决于其他因素，例如内存访问成本和平台特性。因此，这项工作建议评估目标平台上的直接指标，而不仅仅是考虑 FLOP。基于一系列受控实验，这项工作得出了几个有效网络设计的实用指南。因此，提出了一种新的架构，称为 ShuffleNet V2。全面的消融实验验证了我们的模型在速度和准确性的权衡方面是最先进的。
作者在摘要和综述部分主要想说明不能仅仅只是用FLOPs来衡量网络性能，因该使用直接指标(例如速度)，而不是间接指标(FLOPs)，还应该在真实真实硬件上测试真实延迟。
如图 2 所示。我们注意到 FLOPs 指标仅考虑卷积部分。虽然这部分消耗的时间最多，但其他操作包括数据 I/O、数据 shuffle 和元素操作（AddTensor、ReLU 等）也占用了相当多的时间。因此，FLOPs 对实际运行时间的估计不够准确。
2、设计轻量化网络的四条准则
- 输入输出通道相同时，内存访问量MAC最小。
- 分组数过大的分组卷积会增加MAC
- 碎片化操作对并行加速不友好，例如多分支、多路
- 逐元素操作(Element-wise)带来的内存和耗时不可忽略，例如残差中的逐元素加法
3、ShuffleNetV2的骨干网络
Channel Split：在每个单元格的开始，c个特征通道的输入被分成两个分支，分别具有 c − c ′ c-c' c−c′和 c ′ c' c′个通道。(论文后面说为了方便直接设计成 c ′ = 1 2 c c'=\frac{1}{2} c c′=21c,即通道对半分，一部分用于残差连接，一部分用于特征提取)
ChannelShuffle：将堆叠的特征图通道重排，实现各分组之间的特征融合。
注意，ShuffleNetV2没有分组卷积操作，和ShuffleNetV1是不同的，说来也有意思，分组卷积还是ShuffleNetV1的创新点，V2直接删了，哈哈
在ShuffleNetV2基本模块中，特征图大小不变，通道数不变(因为刚开始通道数就被Channel Split操作分办，后面又用Concat操作在通道方向堆叠)
在ShuffleNetV2下采样模块中，特征图长宽减半，通道数加倍。
4、ShuffleNetV2网络架构设计
和ShuffleNetV1没有啥大的区别，就是多了个Conv5,即在全局平均池化前添加一个额外的
1*1卷积来混合特征。
#of Weights表示权重个数
从表中可以看出，每个stage模块的第一个block的stride=2,即第一个block是ShuffleNetV2的下采样模块(d图)，其他的stride=1,即ShuffleNetV2的基本模块(c图)。
Stage模块 由 一个ShuffleNetv2下采样模块 和 若干个ShuffleNetv2基本模块 组成。常用的模块是outchannel列的宽度超参数为1的那一列。
5、ShuffleNet-V2网络结构复现
import tensorflow as tf
from tensorflow.keras.layers import concatenate, Conv2D, Activation, BatchNormalization, DepthwiseConv2D
from tensorflow.keras.layers import add, AvgPool2D, MaxPooling2D, GlobalAveragePooling2D, Dense
from tensorflow.keras.layers import ReLU, Concatenate,Input
from tensorflow.keras.models import Model
from plot_model import plot_model
5.1 标准卷积块
标准卷积块=卷积+批标准化+ReLU
# 标准卷积快：卷积+批标准化+ReLU
# 普通卷积：卷积+批标准化+ReLU激活
def conv_block(inputs, filters, kernel_size, stride=1):
x = Conv2D(filters, kernel_size, stride, padding='same', use_bias=False)(inputs)
x = BatchNormalization()(x)
x = ReLU()(x)
return x
5.2 深度可分离卷积块
论文中DWconv的kernel_size都是3*3，只是下采样模块中的stride=2
# 深度可分离卷积模块
def depthwise_conv_block(inputs, kernel_size, stride=1):
x = DepthwiseConv2D(kernel_size,
strides=stride,
padding='same',
use_bias=False # 有BN就不要用偏置
)(inputs)
x = BatchNormalization()(x)
return x
5.3 Channel Shuffle
# Channel Shuffle模块
# 通道重排,跨组信息交互
# num_groups=2:论文中默认就是2组特征，对应左分支的shortcut和右边经过卷积之后的
def channel_shuffle(inputs, num_groups=2):
# 先得到输入特征图的shape，b:batch size，h,w:一张图的size，c:通道数
b, h, w, c = inputs.shape
# 确定shape = [b, h, w, num_groups, c//num_groups]。通道维度原来是一个长为c的一维tensor，变成num_groups行n列的矩阵
# 在通道维度上将特征图reshape为num_groups行n列的矩阵
x_reshaped = tf.reshape(inputs, [-1, h, w, num_groups, c // num_groups])
# 确定转置的矩形的shape = [b, h, w, c//num_groups, num_groups]
# 矩阵转置，最后两个维度从num_groups行n列变成n行num_groups列
x_transposed = tf.transpose(x_reshaped, [0, 1, 2, 4, 3])
# 重新排列，shotcut和x的通道像素交叉排列，通道维度重新变成一维tensor
output = tf.reshape(x_transposed, [-1, h, w, c])
# 返回通道维度交叉排序后的tensor
return output
5.4 Channel Split操作
这里原论文中直接对半分了。
def channel_split(inputs, num_splits=2):
b1, b2 = tf.split(inputs, num_splits, axis=-1)
return b1, b2
5.5 ShuffleNetV2基本模块
# ShuffleNetV2基本模块
# 长宽不变，通道数不变
def shuffle_block_s1(inputs, out_channels):
# 论文中直接将特征图在通道维度分成两半，分别经过左右分支
x1, x2 = channel_split(inputs) # 此时左右分支的通道数都只有原来的一半
# 右分支：1*1Conv+3*3DWConv+1*1Conv
x2 = conv_block(x2, filters=out_channels // 2, kernel_size=(1, 1), stride=1)
x2 = depthwise_conv_block(x2, kernel_size=(3, 3), stride=1)
x2 = conv_block(x2, filters=out_channels // 2, kernel_size=(1, 1), stride=1)
# 左右分支在通道方向上堆叠(concat)
x = Concatenate(axis=-1)([x1, x2])
# Channel Shuffle
x = channel_shuffle(x)
return x
5.6 ShueffleNetV2下采样模块
# ShuffleNetV2下采样模块
# 下采样模块没有进行Channel Split操作，最后还是用了Concat
# 所以长宽减半，通道数加倍
# 左分支输出特征图数量+右分支输出特征图数量=下采样模块输出特征图数量
def shuffle_block_s2(inputs, out_channels):
shortcut=inputs
in_channels = inputs.shape[-1]
# 左分支：3*3 DWConv(stride=2)+1*1Conv
shortcut = depthwise_conv_block(shortcut, kernel_size=(3, 3), stride=2) # 特征图size减半
shortcut = conv_block(shortcut,in_channels, kernel_size=(1, 1), stride=1)
# 右分支：1*1Conv+3*3DWConv(stride=2)+1*1Conv
x=conv_block(inputs,in_channels//2,kernel_size=(1,1),stride=1)
x=depthwise_conv_block(x,kernel_size=(3,3),stride=2)
# 右分支的通道数和左分支的通道数叠加 == 输出特征图的通道数out_channel
x=conv_block(x,out_channels-in_channels,kernel_size=(1,1),stride=1)
# 左右分支的特征在通道维度上堆叠，out.shape[-1]==out_channel
out=Concatenate(axis=-1)([shortcut,x])
out=channel_shuffle(out,2)
return out
5.7 stage设计
每个stage中的第一个block的stride = 2(即下采样模块)，其他block的stride = 1(即基本模块)
都是按照论文搭建的，要去看论文原文，要不你绝对不理解为什么这样搭建。
def stage(inputs,out_channels,n):
# 每个stage中的第一个block的stride = 2(即下采样模块)，其他block的stride = 1(即基本模块)
# 下采样单元
x=shuffle_block_s2(inputs,out_channels)
for _ in range(n):
x=shuffle_block_s1(x,out_channels)
return x
5.8 完整网络搭建
def ShuffleNet(input_shape,num_classes):
# 构建输入Tensor
inputs=Input(shape=input_shape)
x=Conv2D(filters=24,kernel_size=(3,3),strides=2,padding='same')(inputs)
x=MaxPooling2D(pool_size=(3,3),strides=2,padding='same')(x)
x=stage(x,out_channels=116,n=3)
x=stage(x,out_channels=232,n=7)
x=stage(x,out_channels=464,n=3)
x=Conv2D(filters=1024,kernel_size=(1,1),strides=1,padding='same')(x)
x=GlobalAveragePooling2D()(x)
x=Dense(num_classes,activation='softmax')(x)
model=Model(inputs,x)
return model
model=ShuffleNet(input_shape=(224,224,3),num_classes=1000)
model.summary()
5.9 模型摘要
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 112, 112, 24) 672 input_1[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 56, 56, 24) 0 conv2d[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 56, 56, 12) 288 max_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 56, 56, 12) 48 conv2d_2[0][0]
__________________________________________________________________________________________________
re_lu_1 (ReLU) (None, 56, 56, 12) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
depthwise_conv2d (DepthwiseConv (None, 28, 28, 24) 216 max_pooling2d[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_1 (DepthwiseCo (None, 28, 28, 12) 108 re_lu_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 28, 28, 24) 96 depthwise_conv2d[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 28, 28, 12) 48 depthwise_conv2d_1[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 28, 28, 24) 576 batch_normalization[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 28, 28, 92) 1104 batch_normalization_3[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 28, 28, 24) 96 conv2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 28, 28, 92) 368 conv2d_3[0][0]
__________________________________________________________________________________________________
re_lu (ReLU) (None, 28, 28, 24) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
re_lu_2 (ReLU) (None, 28, 28, 92) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 28, 28, 116) 0 re_lu[0][0]
re_lu_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape (TensorFlow [(None, 28, 28, 2, 5 0 concatenate[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose (TensorFl [(None, 28, 28, 58, 0 tf_op_layer_Reshape[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_1 (TensorFl [(None, 28, 28, 116) 0 tf_op_layer_transpose[0][0]
__________________________________________________________________________________________________
tf_op_layer_split (TensorFlowOp [(None, 28, 28, 58), 0 tf_op_layer_Reshape_1[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 28, 28, 58) 3364 tf_op_layer_split[0][1]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 28, 28, 58) 232 conv2d_4[0][0]
__________________________________________________________________________________________________
re_lu_3 (ReLU) (None, 28, 28, 58) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 58) 522 re_lu_3[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 28, 28, 58) 232 depthwise_conv2d_2[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 28, 28, 58) 3364 batch_normalization_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 28, 28, 58) 232 conv2d_5[0][0]
__________________________________________________________________________________________________
re_lu_4 (ReLU) (None, 28, 28, 58) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 28, 28, 116) 0 tf_op_layer_split[0][0]
re_lu_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_2 (TensorFl [(None, 28, 28, 2, 5 0 concatenate_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_1 (Tensor [(None, 28, 28, 58, 0 tf_op_layer_Reshape_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_3 (TensorFl [(None, 28, 28, 116) 0 tf_op_layer_transpose_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_1 (TensorFlow [(None, 28, 28, 58), 0 tf_op_layer_Reshape_3[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 28, 28, 58) 3364 tf_op_layer_split_1[0][1]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 28, 28, 58) 232 conv2d_6[0][0]
__________________________________________________________________________________________________
re_lu_5 (ReLU) (None, 28, 28, 58) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 58) 522 re_lu_5[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 28, 28, 58) 232 depthwise_conv2d_3[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 28, 28, 58) 3364 batch_normalization_9[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 28, 28, 58) 232 conv2d_7[0][0]
__________________________________________________________________________________________________
re_lu_6 (ReLU) (None, 28, 28, 58) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 28, 28, 116) 0 tf_op_layer_split_1[0][0]
re_lu_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_4 (TensorFl [(None, 28, 28, 2, 5 0 concatenate_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_2 (Tensor [(None, 28, 28, 58, 0 tf_op_layer_Reshape_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_5 (TensorFl [(None, 28, 28, 116) 0 tf_op_layer_transpose_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_2 (TensorFlow [(None, 28, 28, 58), 0 tf_op_layer_Reshape_5[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 28, 28, 58) 3364 tf_op_layer_split_2[0][1]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 28, 28, 58) 232 conv2d_8[0][0]
__________________________________________________________________________________________________
re_lu_7 (ReLU) (None, 28, 28, 58) 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_4 (DepthwiseCo (None, 28, 28, 58) 522 re_lu_7[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 28, 28, 58) 232 depthwise_conv2d_4[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 28, 28, 58) 3364 batch_normalization_12[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 28, 28, 58) 232 conv2d_9[0][0]
__________________________________________________________________________________________________
re_lu_8 (ReLU) (None, 28, 28, 58) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 28, 28, 116) 0 tf_op_layer_split_2[0][0]
re_lu_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_6 (TensorFl [(None, 28, 28, 2, 5 0 concatenate_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_3 (Tensor [(None, 28, 28, 58, 0 tf_op_layer_Reshape_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_7 (TensorFl [(None, 28, 28, 116) 0 tf_op_layer_transpose_3[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 28, 28, 58) 6728 tf_op_layer_Reshape_7[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 28, 28, 58) 232 conv2d_11[0][0]
__________________________________________________________________________________________________
re_lu_10 (ReLU) (None, 28, 28, 58) 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_5 (DepthwiseCo (None, 14, 14, 116) 1044 tf_op_layer_Reshape_7[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 58) 522 re_lu_10[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_5[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 14, 14, 58) 232 depthwise_conv2d_6[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_14[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 14, 14, 116) 6728 batch_normalization_17[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 14, 14, 116) 464 conv2d_10[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 14, 14, 116) 464 conv2d_12[0][0]
__________________________________________________________________________________________________
re_lu_9 (ReLU) (None, 14, 14, 116) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
re_lu_11 (ReLU) (None, 14, 14, 116) 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 14, 14, 232) 0 re_lu_9[0][0]
re_lu_11[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_8 (TensorFl [(None, 14, 14, 2, 1 0 concatenate_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_4 (Tensor [(None, 14, 14, 116, 0 tf_op_layer_Reshape_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_9 (TensorFl [(None, 14, 14, 232) 0 tf_op_layer_transpose_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_3 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_9[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_3[0][1]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 14, 14, 116) 464 conv2d_13[0][0]
__________________________________________________________________________________________________
re_lu_12 (ReLU) (None, 14, 14, 116) 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 116) 1044 re_lu_12[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_7[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_20[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 14, 14, 116) 464 conv2d_14[0][0]
__________________________________________________________________________________________________
re_lu_13 (ReLU) (None, 14, 14, 116) 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_3[0][0]
re_lu_13[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_10 (TensorF [(None, 14, 14, 2, 1 0 concatenate_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_5 (Tensor [(None, 14, 14, 116, 0 tf_op_layer_Reshape_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_11 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_4 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_11[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_4[0][1]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 14, 14, 116) 464 conv2d_15[0][0]
__________________________________________________________________________________________________
re_lu_14 (ReLU) (None, 14, 14, 116) 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 116) 1044 re_lu_14[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_8[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_23[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 14, 14, 116) 464 conv2d_16[0][0]
__________________________________________________________________________________________________
re_lu_15 (ReLU) (None, 14, 14, 116) 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_4[0][0]
re_lu_15[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_12 (TensorF [(None, 14, 14, 2, 1 0 concatenate_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_6 (Tensor [(None, 14, 14, 116, 0 tf_op_layer_Reshape_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_13 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_5 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_13[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_5[0][1]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 14, 14, 116) 464 conv2d_17[0][0]
__________________________________________________________________________________________________
re_lu_16 (ReLU) (None, 14, 14, 116) 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 116) 1044 re_lu_16[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_9[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_26[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 14, 14, 116) 464 conv2d_18[0][0]
__________________________________________________________________________________________________
re_lu_17 (ReLU) (None, 14, 14, 116) 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_5[0][0]
re_lu_17[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_14 (TensorF [(None, 14, 14, 2, 1 0 concatenate_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_7 (Tensor [(None, 14, 14, 116, 0 tf_op_layer_Reshape_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_15 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_6 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_15[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_6[0][1]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 116) 464 conv2d_19[0][0]
__________________________________________________________________________________________________
re_lu_18 (ReLU) (None, 14, 14, 116) 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 116) 1044 re_lu_18[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_10[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_29[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 14, 14, 116) 464 conv2d_20[0][0]
__________________________________________________________________________________________________
re_lu_19 (ReLU) (None, 14, 14, 116) 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_6[0][0]
re_lu_19[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_16 (TensorF [(None, 14, 14, 2, 1 0 concatenate_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_8 (Tensor [(None, 14, 14, 116, 0 tf_op_layer_Reshape_16[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_17 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_7 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_17[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_7[0][1]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 116) 464 conv2d_21[0][0]
__________________________________________________________________________________________________
re_lu_20 (ReLU) (None, 14, 14, 116) 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 116) 1044 re_lu_20[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_11[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_32[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 116) 464 conv2d_22[0][0]
__________________________________________________________________________________________________
re_lu_21 (ReLU) (None, 14, 14, 116) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
concatenate_9 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_7[0][0]
re_lu_21[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_18 (TensorF [(None, 14, 14, 2, 1 0 concatenate_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_9 (Tensor [(None, 14, 14, 116, 0 tf_op_layer_Reshape_18[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_19 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_8 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_19[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_8[0][1]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 116) 464 conv2d_23[0][0]
__________________________________________________________________________________________________
re_lu_22 (ReLU) (None, 14, 14, 116) 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_12 (DepthwiseC (None, 14, 14, 116) 1044 re_lu_22[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_12[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_35[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 116) 464 conv2d_24[0][0]
__________________________________________________________________________________________________
re_lu_23 (ReLU) (None, 14, 14, 116) 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
concatenate_10 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_8[0][0]
re_lu_23[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_20 (TensorF [(None, 14, 14, 2, 1 0 concatenate_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_10 (Tenso [(None, 14, 14, 116, 0 tf_op_layer_Reshape_20[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_21 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_9 (TensorFlow [(None, 14, 14, 116) 0 tf_op_layer_Reshape_21[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 14, 14, 116) 13456 tf_op_layer_split_9[0][1]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 116) 464 conv2d_25[0][0]
__________________________________________________________________________________________________
re_lu_24 (ReLU) (None, 14, 14, 116) 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_13 (DepthwiseC (None, 14, 14, 116) 1044 re_lu_24[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 116) 464 depthwise_conv2d_13[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 14, 14, 116) 13456 batch_normalization_38[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 116) 464 conv2d_26[0][0]
__________________________________________________________________________________________________
re_lu_25 (ReLU) (None, 14, 14, 116) 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
concatenate_11 (Concatenate) (None, 14, 14, 232) 0 tf_op_layer_split_9[0][0]
re_lu_25[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_22 (TensorF [(None, 14, 14, 2, 1 0 concatenate_11[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_11 (Tenso [(None, 14, 14, 116, 0 tf_op_layer_Reshape_22[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_23 (TensorF [(None, 14, 14, 232) 0 tf_op_layer_transpose_11[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 14, 14, 116) 26912 tf_op_layer_Reshape_23[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 14, 14, 116) 464 conv2d_28[0][0]
__________________________________________________________________________________________________
re_lu_27 (ReLU) (None, 14, 14, 116) 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 232) 2088 tf_op_layer_Reshape_23[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_15 (DepthwiseC (None, 7, 7, 116) 1044 re_lu_27[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 7, 7, 232) 928 depthwise_conv2d_14[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 116) 464 depthwise_conv2d_15[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 7, 7, 232) 53824 batch_normalization_40[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 7, 7, 232) 26912 batch_normalization_43[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 7, 7, 232) 928 conv2d_27[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 232) 928 conv2d_29[0][0]
__________________________________________________________________________________________________
re_lu_26 (ReLU) (None, 7, 7, 232) 0 batch_normalization_41[0][0]
__________________________________________________________________________________________________
re_lu_28 (ReLU) (None, 7, 7, 232) 0 batch_normalization_44[0][0]
__________________________________________________________________________________________________
concatenate_12 (Concatenate) (None, 7, 7, 464) 0 re_lu_26[0][0]
re_lu_28[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_24 (TensorF [(None, 7, 7, 2, 232 0 concatenate_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_12 (Tenso [(None, 7, 7, 232, 2 0 tf_op_layer_Reshape_24[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_25 (TensorF [(None, 7, 7, 464)] 0 tf_op_layer_transpose_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_10 (TensorFlo [(None, 7, 7, 232), 0 tf_op_layer_Reshape_25[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 7, 7, 232) 53824 tf_op_layer_split_10[0][1]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 232) 928 conv2d_30[0][0]
__________________________________________________________________________________________________
re_lu_29 (ReLU) (None, 7, 7, 232) 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_16 (DepthwiseC (None, 7, 7, 232) 2088 re_lu_29[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 7, 7, 232) 928 depthwise_conv2d_16[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 7, 7, 232) 53824 batch_normalization_46[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 7, 7, 232) 928 conv2d_31[0][0]
__________________________________________________________________________________________________
re_lu_30 (ReLU) (None, 7, 7, 232) 0 batch_normalization_47[0][0]
__________________________________________________________________________________________________
concatenate_13 (Concatenate) (None, 7, 7, 464) 0 tf_op_layer_split_10[0][0]
re_lu_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_26 (TensorF [(None, 7, 7, 2, 232 0 concatenate_13[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_13 (Tenso [(None, 7, 7, 232, 2 0 tf_op_layer_Reshape_26[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_27 (TensorF [(None, 7, 7, 464)] 0 tf_op_layer_transpose_13[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_11 (TensorFlo [(None, 7, 7, 232), 0 tf_op_layer_Reshape_27[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 7, 7, 232) 53824 tf_op_layer_split_11[0][1]
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 7, 7, 232) 928 conv2d_32[0][0]
__________________________________________________________________________________________________
re_lu_31 (ReLU) (None, 7, 7, 232) 0 batch_normalization_48[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_17 (DepthwiseC (None, 7, 7, 232) 2088 re_lu_31[0][0]
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, 7, 7, 232) 928 depthwise_conv2d_17[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 7, 7, 232) 53824 batch_normalization_49[0][0]
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, 7, 7, 232) 928 conv2d_33[0][0]
__________________________________________________________________________________________________
re_lu_32 (ReLU) (None, 7, 7, 232) 0 batch_normalization_50[0][0]
__________________________________________________________________________________________________
concatenate_14 (Concatenate) (None, 7, 7, 464) 0 tf_op_layer_split_11[0][0]
re_lu_32[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_28 (TensorF [(None, 7, 7, 2, 232 0 concatenate_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_14 (Tenso [(None, 7, 7, 232, 2 0 tf_op_layer_Reshape_28[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_29 (TensorF [(None, 7, 7, 464)] 0 tf_op_layer_transpose_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_12 (TensorFlo [(None, 7, 7, 232), 0 tf_op_layer_Reshape_29[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 7, 7, 232) 53824 tf_op_layer_split_12[0][1]
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, 7, 7, 232) 928 conv2d_34[0][0]
__________________________________________________________________________________________________
re_lu_33 (ReLU) (None, 7, 7, 232) 0 batch_normalization_51[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_18 (DepthwiseC (None, 7, 7, 232) 2088 re_lu_33[0][0]
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, 7, 7, 232) 928 depthwise_conv2d_18[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 7, 7, 232) 53824 batch_normalization_52[0][0]
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, 7, 7, 232) 928 conv2d_35[0][0]
__________________________________________________________________________________________________
re_lu_34 (ReLU) (None, 7, 7, 232) 0 batch_normalization_53[0][0]
__________________________________________________________________________________________________
concatenate_15 (Concatenate) (None, 7, 7, 464) 0 tf_op_layer_split_12[0][0]
re_lu_34[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_30 (TensorF [(None, 7, 7, 2, 232 0 concatenate_15[0][0]
__________________________________________________________________________________________________
tf_op_layer_transpose_15 (Tenso [(None, 7, 7, 232, 2 0 tf_op_layer_Reshape_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_31 (TensorF [(None, 7, 7, 464)] 0 tf_op_layer_transpose_15[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 7, 7, 1024) 476160 tf_op_layer_Reshape_31[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 1024) 0 conv2d_36[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 1000) 1025000 global_average_pooling2d[0][0]
==================================================================================================
Total params: 2,216,440
Trainable params: 2,203,236
Non-trainable params: 13,204
5.10 模型结构大图
图太长了，需要原图或者pdf，留下邮箱我私发你。
References
Ma N, Zhang X, Zheng H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 116-131.
8.3 使用Tensorflow2搭建ShuffleNetv2
ShuffleNetV1:极致轻量化卷积神经网络(分组卷积+通道重排)