EfficientNet代码复现--ICML2019

参考论文：EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

作者：Mingxing Tan, Quoc Le

前置知识：SENet架构-通道注意力机制

MobileNetV3架构解析与代码复现

1、EfficientNet简介

  在本文中，我们系统地研究了模型缩放，并确定仔细平衡网络深度、宽度和分辨率可以带来更好的性能。基于这一观察，我们提出了一种新的缩放方法，该方法使用简单但高效的复合系数统一缩放深度/宽度/分辨率的所有维度。我们证明了这种方法在扩展 MobileNets 和 ResNet 方面的有效性。

  我们使用神经架构搜索来设计一个新的基线网络并将其扩展以获得一系列模型，称为 EfficientNets，它们比以前的 ConvNets 实现了更好的准确性和效率。特别是，我们的 EfficientNet-B7 在 ImageNet 上实现了最先进的 84.3% 的 top-1 准确率，同时比现有的最佳 ConvNet 小 8.4 倍，推理速度快 6.1 倍。我们的 EfficientNets 在 CIFAR-100 (91.7%)、Flowers (98.8%) 和其他 3 个迁移学习数据集上也能很好地迁移并实现最先进的准确度，参数少一个数量级。

2、各种缩放方法

  图 2. 模型缩放。 (a) 是一个基准网络示例； (b)-(d) 是常规缩放，仅增加网络宽度、深度或分辨率的一维。 (e) 是我们提出的复合缩放方法，它以固定比例均匀缩放所有三个维度。

3、Compound Model Scaling

  图 3. 放大具有不同网络宽度 (w)、深度 (d) 和分辨率 ® 系数的基准模型。具有更大宽度、深度或分辨率的更大网络往往会获得更高的准确度，但准确度增益在达到 80% 后会迅速饱和，这表明了单维缩放的局限性。基准网络在表 1 中进行了描述。

3.1 Depth(d):网络深度

  缩放网络深度是许多 ConvNet 最常用的方式.直觉是，更深的 ConvNet 可以捕获更丰富、更复杂的特征，并且可以很好地概括新任务。然而，由于梯度消失问题，更深层次的网络也更难训练。尽管skip connections和batch normailzation等多种技术缓解了训练问题，但非常深的网络的准确度增益降低了。

3.2 Width(d)：网络宽度

  缩放网络宽度通常用于小尺寸模型，更宽的网络往往能够捕获更细粒度的特征并且更容易训练。然而，极宽但很浅的网络往往难以捕捉更高层次的特征。我们在图 3（左）中的经验结果表明，当网络变得更宽且 w 越大时，准确度会迅速饱和。

3.3 Resolution®：分辨率

  使用更高分辨率的输入图像，ConvNets 可以潜在地捕获更细粒度的模式。图 3（右）显示了缩放网络分辨率的结果，其中确实更高的分辨率提高了精度，但精度增益对于非常高的分辨率会降低（r = 1.0 表示分辨率 224x224，r = 2.5 表示分辨率 560x560）。

  作者通过实验得出下面两个Observation

• 扩大网络宽度、深度或分辨率的任何维度都会提高准确性，但对于更大的模型，准确性增益会降低。
• 为了追求更好的准确性和效率，在 ConvNet 缩放期间平衡网络宽度、深度和分辨率的所有维度至关重要。

3.4 Compound Scaling(负荷缩放)

  图 4. 不同基准网络的缩放网络宽度。一行中的每个点表示具有不同宽度系数 (w) 的模型。所有基准网络均来自表 1。第一个基准网络（d=1.0，r=1.0）有 18 个卷积层，分辨率为 224x224，而最后一个基线（d=2.0，r=1.3）有 36 层，分辨率为 299x299。

  为了验证我们的直觉，我们比较了不同网络深度和分辨率下的宽度缩放，如图 4 所示。如果我们只缩放网络宽度 w 而不改变深度 (d=1.0) 和分辨率 (r=1.0)，精度很快就会饱和。随着更深（d=2.0）和更高的分辨率（r=2.0），宽度缩放在相同的 FLOPS 成本下实现了更好的精度。这些结果使我们得出第二个观察结果：

  在本文中，我们提出了一种新的复合缩放方法，它使用复合系数 φ 以有原则的方式统一缩放网络宽度、深度和分辨率：

  其中 α、β、γ 是可以通过小网格搜索确定的常数。直观地说，φ 是用户指定的系数，它控制有多少资源可用于模型缩放，而 α、β、γ 分别指定如何将这些额外资源分配给网络宽度、深度和分辨率。

  在本文中，我们约束 $\alpha \cdot {\beta }^2\cdot {\gamma }^2\approx 2$，这样对于任何新的 φ，总 FLOPS 将大约 3 增加 $2^{\varphi }$。

4、MBConv结构

  第一个升维的1*1卷积层，它的卷积核个数是输入特征矩阵channel的n倍。

  当n=1时，是没有第一个升维的1*1卷积层的，即Stage2中的MBConv结构都没有第一个升维的1*1卷积层(这个和MobileNetV3网络类似)。

  关于shortcut连接，仅当输入MBConv结构的特征矩阵与输出的特征矩阵shape相同时才存在。

5、SeNet结构

具体实现如下：

• 对输入进来的特征层进行全局平均池化。
• 然后进行两次全连接(这两个全连接可用1*1卷积代替)，第一次全连接神经元个数较少，第二次全连接神经元个数和输入特征层个数相同。
• 在完成两次全连接之后，再取一次sigmoid讲值固定到0-1之间，此时我们获得了输入特征层每一个通道的权值(0-1之间)。
• 在获得这个权值之后，讲这个权值与原输入特征层相乘即可。

  上面是经典的SENet,这里用的激活函数时swish

  本文SE模块如图所示， 由一个全局平均池化， 两个全连接层组成。 第一个全连接层的节点个数是输入该MBConv特征矩阵channels的1/4， 且使用Swish激活函数。 第二个全连接层的节点个数等于Depthwise Conv层输出的特征矩阵channels， 且使用Sigmoid激活函数。

6、EfficientNet Architecture

  EfficientNet-B0 基准网络。每行描述一个阶段i,$\hat{L_i} $表示每个block的重复次数。

  Resolution为图像分辨率

  Channels为输出通道数。

从上图可以分析出步长如下：

  只有每个stage中第一个Operator中的步长为右边的stride，其他默认为1。

7、各种EfficientNet结构

  图片来源：https://www.bilibili.com/video/BV1XK4y1U7PX/?spm_id_from=pageDriver&vd_source=2ce79abf5d0d27bf7f3f2c38406c8eb6

• width_coefficient代表channel维度上的倍率因子， 比如在 EfficientNetB0中Stage1的3x3卷积层所使用的卷积核个数是32， 那么在B6中就是 32 × 1.8 = 57.6接着取整到离它最近的8的整数倍即56， 其它Stage同理。
• depth_coefficient代表depth维度上的倍率因子（仅针对Stage2到Stage8） ， 比如在EfficientNetB0中Stage7的L = 4 ， 那么在B6中就是 4 × 2.6 = 10.4， 接着向上取整即11.

8、EfficientNetB0代码复现(自己搭建)

  这里我仿照github源码搭建的。

import tensorflow as tf
import math
from tensorflow.keras import layers
from tensorflow.keras.models import Model
from typing import Union
from tensorflow.keras import backend

8.1、Block参数设置

DEFAULT_BLOCKS_ARGS = [{
    
    
    'kernel_size': 3,
    'repeats': 1,
    'filters_in': 32,
    'filters_out': 16,
    'expand_ratio': 1,
    'id_skip': True,
    'strides': 1,
    'se_ratio': 0.25
}, {
    
    
    'kernel_size': 3,
    'repeats': 2,
    'filters_in': 16,
    'filters_out': 24,
    'expand_ratio': 6,
    'id_skip': True,
    'strides': 2,
    'se_ratio': 0.25
}, {
    
    
    'kernel_size': 5,
    'repeats': 2,
    'filters_in': 24,
    'filters_out': 40,
    'expand_ratio': 6,
    'id_skip': True,
    'strides': 2,
    'se_ratio': 0.25
}, {
    
    
    'kernel_size': 3,
    'repeats': 3,
    'filters_in': 40,
    'filters_out': 80,
    'expand_ratio': 6,
    'id_skip': True,
    'strides': 2,
    'se_ratio': 0.25
}, {
    
    
    'kernel_size': 5,
    'repeats': 3,
    'filters_in': 80,
    'filters_out': 112,
    'expand_ratio': 6,
    'id_skip': True,
    'strides': 1,
    'se_ratio': 0.25
}, {
    
    
    'kernel_size': 5,
    'repeats': 4,
    'filters_in': 112,
    'filters_out': 192,
    'expand_ratio': 6,
    'id_skip': True,
    'strides': 2,
    'se_ratio': 0.25
}, {
    
    
    'kernel_size': 3,
    'repeats': 1,
    'filters_in': 192,
    'filters_out': 320,
    'expand_ratio': 6,
    'id_skip': True,
    'strides': 1,
    'se_ratio': 0.25
}]
CONV_KERNEL_INITIALIZER = {
    
    
    'class_name': 'VarianceScaling',
    'config': {
    
    
        'scale': 2.0,
        'mode': 'fan_out',
        'distribution': 'truncated_normal'
    }
}

DENSE_KERNEL_INITIALIZER = {
    
    
    'class_name': 'VarianceScaling',
    'config': {
    
    
        'scale': 1. / 3.,
        'mode': 'fan_out',
        'distribution': 'uniform'
    }
}

8.2 correct_pad

def correct_pad(inputs, kernel_size):
    """Returns a tuple for zero-padding for 2D convolution with downsampling.
  Args:
    inputs: Input tensor.
    kernel_size: An integer or tuple/list of 2 integers.
  Returns:
    A tuple.
  """
    img_dim = 2 if backend.image_data_format() == 'channels_first' else 1
    input_size = backend.int_shape(inputs)[img_dim:(img_dim + 2)]
    if isinstance(kernel_size, int):
        kernel_size = (kernel_size, kernel_size)
    if input_size[0] is None:
        adjust = (1, 1)
    else:
        adjust = (1 - input_size[0] % 2, 1 - input_size[1] % 2)
    correct = (kernel_size[0] // 2, kernel_size[1] // 2)
    return ((correct[0] - adjust[0], correct[0]),
            (correct[1] - adjust[1], correct[1]))

8.3 MBConv block

def block(inputs,  # 输入张量
          activation='swish',  # 激活函数
          drop_rate=0.,
          name='',
          filters_in=32,  # 输入通道数
          filters_out=16,  # 输出通道数
          kernel_size=3,  # 卷积核大小
          strides=1,  # 卷积的步长
          expand_ratio=1,  # 缩放系数
          se_ratio=0.,  # 压缩SE模块中的通道数
          id_skip=True):
    # Expansion phase
    filters = filters_in * expand_ratio
    if expand_ratio != 1:
        x = layers.Conv2D(filters=filters,
                          kernel_size=(1, 1),
                          padding='same',
                          use_bias=False,
                          kernel_initializer=CONV_KERNEL_INITIALIZER,
                          name=name + 'expand_conv')(inputs)
        x = layers.BatchNormalization(name=name + 'expand_bn')(x)
        x = layers.Activation(activation, name=name + 'expand_activation')(x)
    else:
        x = inputs

    # Depthwise Convolution
    if strides == 2:
        x = layers.ZeroPadding2D(padding=correct_pad(x, kernel_size),
                                 name=name + 'dwconv_pad')(x)
    x = layers.DepthwiseConv2D(kernel_size,
                               strides=strides,
                               padding='same' if strides == 1 else 'valid',
                               use_bias=False,
                               depthwise_initializer=CONV_KERNEL_INITIALIZER,
                               name=name + 'dwconv')(x)
    x = layers.BatchNormalization(name=name + 'bn')(x)
    x = layers.Activation(activation, name=name + 'activation')(x)

    # SE注意力机制模块
    if 0 < se_ratio <= 1:
        filters_se = max(1, int(filters_in * se_ratio))
        se = layers.GlobalAveragePooling2D(name=name + "se_squeeze")(x)
        se = layers.Reshape((1, 1, filters), name=name + "se_reshape")(se)
        se = layers.Conv2D(filters=filters_se,
                           kernel_size=1,
                           padding="same",
                           activation=activation,
                           kernel_initializer=CONV_KERNEL_INITIALIZER,
                           name=name + "se_reduce")(se)
        se = layers.Conv2D(filters=filters,
                           kernel_size=1,
                           padding="same",
                           activation="sigmoid",
                           kernel_initializer=CONV_KERNEL_INITIALIZER,
                           name=name + "se_expand")(se)
        x = layers.multiply([x, se], name=name + "se_excite")

    # OutPut phase
    # 1*1卷积降维
    x = layers.Conv2D(filters=filters_out,
                      kernel_size=(1, 1),
                      padding='same',
                      use_bias=False,
                      kernel_initializer=CONV_KERNEL_INITIALIZER,
                      name=name + 'project_conv')(x)
    x = layers.BatchNormalization(name=name + 'project_bn')(x)
    # shape相同得时候才能Add,stride==1并且输入通道数和输出通道数相等
    if id_skip and strides == 1 and filters_in == filters_out:
        # 只有使用shortcut连接且drop_rate>0的时候才会有DropOut层
        if drop_rate > 0:
            x = layers.Dropout(drop_rate, noise_shape=(None, 1, 1, 1), name=name + 'drop')(x)
        x = layers.add([x, inputs], name=name + 'add')
    return x

8.4 EfficientNet NetWok

def EfficientNet(
        width_coefficient=1.0,  # 网络宽度的缩放系数
        depth_coefficient=1.0,  # 网络深度的缩放系数
        default_size=224,  # 默认输入图像大小
        dropout_rate=0.2,  # 最终分类层之前的dropout
        drop_connect_rate=0.2,  # skip connection时的dropout
        depth_divisor=8,  #
        activation='swish',  # 激活函数
        blocks_args='default',  # 字典列表，构造MBConv模块的参数
        model_name='efficientnet',  # 模型名称
        include_top=True,  # 是否包含位于网络顶部的全连接层
        weights='imagenet',
        input_tensor=None,  # 可选的keras张量，用作模型的输入
        input_shape=None,  # 输入shape
        pooling=None,  # 用于特征提取的可选池化模式
        classes=1000,  # 用于图像的分类个数
        classifier_activation='softmax'):  # 分类器的激活函数
    if blocks_args == 'default':
        blocks_args = DEFAULT_BLOCKS_ARGS
    img_input = layers.Input(shape=input_shape)

    # 将filters调整到离它最近的8的整数倍
    def round_filters(filters, divisor=depth_divisor):
        # 基于深度乘数的卷积核的整数
        filters *= width_coefficient
        new_filters = max(divisor, int(filters + divisor / 2) // divisor * divisor)
        # 确保向下舍入不超过10%
        if new_filters < 0.9 * filters:
            new_filters += divisor
        return int(new_filters)

    # 重复MBConv模块的次数
    def round_repeats(repeats):
        # 基于深度乘数的重复循环次数
        return int(math.ceil(depth_coefficient * repeats))

    # Build stem
    x = img_input
    # 这里源码中有如下预处理
#     x = layers.Rescaling(1. / 255.)(x)
#     x = layers.Normalization(axis=bn_axis)(x)
    x = layers.ZeroPadding2D(padding=correct_pad(x, 3), name='stem_conv_pad')(x)
    x = layers.Conv2D(
        round_filters(32),
        kernel_size=(3, 3),
        strides=2,
        padding='valid',
        use_bias=False,
        kernel_initializer=CONV_KERNEL_INITIALIZER,
        name='stem_conv')(x)
    x = layers.BatchNormalization(name='stem_bn')(x)
    x = layers.Activation(activation, name='stem_activation')(x)

    # Build blocks
    b = 0
    blocks = float(sum(round_repeats(args['repeats']) for args in blocks_args))
    for (i, args) in enumerate(blocks_args):
        assert args['repeats'] > 0
        # 根据深度乘数更新块输入和输出的卷积核个数
        args['filters_in'] = round_filters(args['filters_in'])
        args['filters_out'] = round_filters(args['filters_out'])

        for j in range(round_repeats(args.pop('repeats'))):
            # 第一个块需要处理步幅和卷积核大小的增加
            if j > 0:
                args['strides'] = 1
                args['filters_in'] = args['filters_out']
            x = block(
                x,
                activation,
                drop_connect_rate * b / blocks,
                name='block{}{}_'.format(i + 1, chr(j + 97)),
                **args)
            b += 1
    # Build top
    x = layers.Conv2D(
        round_filters(1280),
        1,
        padding='same',
        use_bias=False,
        kernel_initializer=CONV_KERNEL_INITIALIZER,
        name='top_conv')(x)
    x = layers.BatchNormalization(name='top_bn')(x)
    x = layers.Activation(activation, name='top_activation')(x)

    if include_top:
        x = layers.GlobalAveragePooling2D(name='avg_pool')(x)
        if dropout_rate > 0:
            x = layers.Dropout(dropout_rate, name='top_dropout')(x)
        x = layers.Dense(classes,
                         activation=classifier_activation,
                         kernel_initializer=DENSE_KERNEL_INITIALIZER,
                         name='predictions')(x)
    else:
        if pooling == 'avg':
            x = layers.GlobalAveragePooling2D(name='avg_pool')(x)
        elif pooling == 'max':
            x = layers.GlobalMaxPooling2D(name='max_pool')(x)
    # Create model
    model = Model(img_input, x, name=model_name)

    return model

  源码中还有如下预处理，我这里由于tensorflow版本较低，就先注释掉了。

 x = layers.Rescaling(1. / 255.)(x)
 x = layers.Normalization(axis=bn_axis)(x)

8.5 EfficientNetB0

def EfficientNetB0(num_classes=1000,
                   include_top=True,
                   input_shape=(224, 224, 3)):
    return EfficientNet(width_coefficient=1.0,
                        depth_coefficient=1.0,
                        include_top=include_top,
                        input_shape=input_shape
                        )

if __name__ == '__main__':
    efficientNetb0 = EfficientNetB0()
    efficientNetb0.summary()

Model: "efficientnet"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
stem_conv_pad (ZeroPadding2D)   (None, 225, 225, 3)  0           input_1[0][0]                    
__________________________________________________________________________________________________
stem_conv (Conv2D)              (None, 112, 112, 32) 864         stem_conv_pad[0][0]              
__________________________________________________________________________________________________
stem_bn (BatchNormalization)    (None, 112, 112, 32) 128         stem_conv[0][0]                  
__________________________________________________________________________________________________
stem_activation (Activation)    (None, 112, 112, 32) 0           stem_bn[0][0]                    
__________________________________________________________________________________________________
block1a_dwconv (DepthwiseConv2D (None, 112, 112, 32) 288         stem_activation[0][0]            
__________________________________________________________________________________________________
block1a_bn (BatchNormalization) (None, 112, 112, 32) 128         block1a_dwconv[0][0]             
__________________________________________________________________________________________________
block1a_activation (Activation) (None, 112, 112, 32) 0           block1a_bn[0][0]                 
__________________________________________________________________________________________________
block1a_se_squeeze (GlobalAvera (None, 32)           0           block1a_activation[0][0]         
__________________________________________________________________________________________________
block1a_se_reshape (Reshape)    (None, 1, 1, 32)     0           block1a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block1a_se_reduce (Conv2D)      (None, 1, 1, 8)      264         block1a_se_reshape[0][0]         
__________________________________________________________________________________________________
block1a_se_expand (Conv2D)      (None, 1, 1, 32)     288         block1a_se_reduce[0][0]          
__________________________________________________________________________________________________
block1a_se_excite (Multiply)    (None, 112, 112, 32) 0           block1a_activation[0][0]         
                                                                 block1a_se_expand[0][0]          
__________________________________________________________________________________________________
block1a_project_conv (Conv2D)   (None, 112, 112, 16) 512         block1a_se_excite[0][0]          
__________________________________________________________________________________________________
block1a_project_bn (BatchNormal (None, 112, 112, 16) 64          block1a_project_conv[0][0]       
__________________________________________________________________________________________________
block2a_expand_conv (Conv2D)    (None, 112, 112, 96) 1536        block1a_project_bn[0][0]         
__________________________________________________________________________________________________
block2a_expand_bn (BatchNormali (None, 112, 112, 96) 384         block2a_expand_conv[0][0]        
__________________________________________________________________________________________________
block2a_expand_activation (Acti (None, 112, 112, 96) 0           block2a_expand_bn[0][0]          
__________________________________________________________________________________________________
block2a_dwconv_pad (ZeroPadding (None, 113, 113, 96) 0           block2a_expand_activation[0][0]  
__________________________________________________________________________________________________
block2a_dwconv (DepthwiseConv2D (None, 56, 56, 96)   864         block2a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block2a_bn (BatchNormalization) (None, 56, 56, 96)   384         block2a_dwconv[0][0]             
__________________________________________________________________________________________________
block2a_activation (Activation) (None, 56, 56, 96)   0           block2a_bn[0][0]                 
__________________________________________________________________________________________________
block2a_se_squeeze (GlobalAvera (None, 96)           0           block2a_activation[0][0]         
__________________________________________________________________________________________________
block2a_se_reshape (Reshape)    (None, 1, 1, 96)     0           block2a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block2a_se_reduce (Conv2D)      (None, 1, 1, 4)      388         block2a_se_reshape[0][0]         
__________________________________________________________________________________________________
block2a_se_expand (Conv2D)      (None, 1, 1, 96)     480         block2a_se_reduce[0][0]          
__________________________________________________________________________________________________
block2a_se_excite (Multiply)    (None, 56, 56, 96)   0           block2a_activation[0][0]         
                                                                 block2a_se_expand[0][0]          
__________________________________________________________________________________________________
block2a_project_conv (Conv2D)   (None, 56, 56, 24)   2304        block2a_se_excite[0][0]          
__________________________________________________________________________________________________
block2a_project_bn (BatchNormal (None, 56, 56, 24)   96          block2a_project_conv[0][0]       
__________________________________________________________________________________________________
block2b_expand_conv (Conv2D)    (None, 56, 56, 144)  3456        block2a_project_bn[0][0]         
__________________________________________________________________________________________________
block2b_expand_bn (BatchNormali (None, 56, 56, 144)  576         block2b_expand_conv[0][0]        
__________________________________________________________________________________________________
block2b_expand_activation (Acti (None, 56, 56, 144)  0           block2b_expand_bn[0][0]          
__________________________________________________________________________________________________
block2b_dwconv (DepthwiseConv2D (None, 56, 56, 144)  1296        block2b_expand_activation[0][0]  
__________________________________________________________________________________________________
block2b_bn (BatchNormalization) (None, 56, 56, 144)  576         block2b_dwconv[0][0]             
__________________________________________________________________________________________________
block2b_activation (Activation) (None, 56, 56, 144)  0           block2b_bn[0][0]                 
__________________________________________________________________________________________________
block2b_se_squeeze (GlobalAvera (None, 144)          0           block2b_activation[0][0]         
__________________________________________________________________________________________________
block2b_se_reshape (Reshape)    (None, 1, 1, 144)    0           block2b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block2b_se_reduce (Conv2D)      (None, 1, 1, 6)      870         block2b_se_reshape[0][0]         
__________________________________________________________________________________________________
block2b_se_expand (Conv2D)      (None, 1, 1, 144)    1008        block2b_se_reduce[0][0]          
__________________________________________________________________________________________________
block2b_se_excite (Multiply)    (None, 56, 56, 144)  0           block2b_activation[0][0]         
                                                                 block2b_se_expand[0][0]          
__________________________________________________________________________________________________
block2b_project_conv (Conv2D)   (None, 56, 56, 24)   3456        block2b_se_excite[0][0]          
__________________________________________________________________________________________________
block2b_project_bn (BatchNormal (None, 56, 56, 24)   96          block2b_project_conv[0][0]       
__________________________________________________________________________________________________
block2b_drop (Dropout)          (None, 56, 56, 24)   0           block2b_project_bn[0][0]         
__________________________________________________________________________________________________
block2b_add (Add)               (None, 56, 56, 24)   0           block2b_drop[0][0]               
                                                                 block2a_project_bn[0][0]         
__________________________________________________________________________________________________
block3a_expand_conv (Conv2D)    (None, 56, 56, 144)  3456        block2b_add[0][0]                
__________________________________________________________________________________________________
block3a_expand_bn (BatchNormali (None, 56, 56, 144)  576         block3a_expand_conv[0][0]        
__________________________________________________________________________________________________
block3a_expand_activation (Acti (None, 56, 56, 144)  0           block3a_expand_bn[0][0]          
__________________________________________________________________________________________________
block3a_dwconv_pad (ZeroPadding (None, 59, 59, 144)  0           block3a_expand_activation[0][0]  
__________________________________________________________________________________________________
block3a_dwconv (DepthwiseConv2D (None, 28, 28, 144)  3600        block3a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block3a_bn (BatchNormalization) (None, 28, 28, 144)  576         block3a_dwconv[0][0]             
__________________________________________________________________________________________________
block3a_activation (Activation) (None, 28, 28, 144)  0           block3a_bn[0][0]                 
__________________________________________________________________________________________________
block3a_se_squeeze (GlobalAvera (None, 144)          0           block3a_activation[0][0]         
__________________________________________________________________________________________________
block3a_se_reshape (Reshape)    (None, 1, 1, 144)    0           block3a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block3a_se_reduce (Conv2D)      (None, 1, 1, 6)      870         block3a_se_reshape[0][0]         
__________________________________________________________________________________________________
block3a_se_expand (Conv2D)      (None, 1, 1, 144)    1008        block3a_se_reduce[0][0]          
__________________________________________________________________________________________________
block3a_se_excite (Multiply)    (None, 28, 28, 144)  0           block3a_activation[0][0]         
                                                                 block3a_se_expand[0][0]          
__________________________________________________________________________________________________
block3a_project_conv (Conv2D)   (None, 28, 28, 40)   5760        block3a_se_excite[0][0]          
__________________________________________________________________________________________________
block3a_project_bn (BatchNormal (None, 28, 28, 40)   160         block3a_project_conv[0][0]       
__________________________________________________________________________________________________
block3b_expand_conv (Conv2D)    (None, 28, 28, 240)  9600        block3a_project_bn[0][0]         
__________________________________________________________________________________________________
block3b_expand_bn (BatchNormali (None, 28, 28, 240)  960         block3b_expand_conv[0][0]        
__________________________________________________________________________________________________
block3b_expand_activation (Acti (None, 28, 28, 240)  0           block3b_expand_bn[0][0]          
__________________________________________________________________________________________________
block3b_dwconv (DepthwiseConv2D (None, 28, 28, 240)  6000        block3b_expand_activation[0][0]  
__________________________________________________________________________________________________
block3b_bn (BatchNormalization) (None, 28, 28, 240)  960         block3b_dwconv[0][0]             
__________________________________________________________________________________________________
block3b_activation (Activation) (None, 28, 28, 240)  0           block3b_bn[0][0]                 
__________________________________________________________________________________________________
block3b_se_squeeze (GlobalAvera (None, 240)          0           block3b_activation[0][0]         
__________________________________________________________________________________________________
block3b_se_reshape (Reshape)    (None, 1, 1, 240)    0           block3b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block3b_se_reduce (Conv2D)      (None, 1, 1, 10)     2410        block3b_se_reshape[0][0]         
__________________________________________________________________________________________________
block3b_se_expand (Conv2D)      (None, 1, 1, 240)    2640        block3b_se_reduce[0][0]          
__________________________________________________________________________________________________
block3b_se_excite (Multiply)    (None, 28, 28, 240)  0           block3b_activation[0][0]         
                                                                 block3b_se_expand[0][0]          
__________________________________________________________________________________________________
block3b_project_conv (Conv2D)   (None, 28, 28, 40)   9600        block3b_se_excite[0][0]          
__________________________________________________________________________________________________
block3b_project_bn (BatchNormal (None, 28, 28, 40)   160         block3b_project_conv[0][0]       
__________________________________________________________________________________________________
block3b_drop (Dropout)          (None, 28, 28, 40)   0           block3b_project_bn[0][0]         
__________________________________________________________________________________________________
block3b_add (Add)               (None, 28, 28, 40)   0           block3b_drop[0][0]               
                                                                 block3a_project_bn[0][0]         
__________________________________________________________________________________________________
block4a_expand_conv (Conv2D)    (None, 28, 28, 240)  9600        block3b_add[0][0]                
__________________________________________________________________________________________________
block4a_expand_bn (BatchNormali (None, 28, 28, 240)  960         block4a_expand_conv[0][0]        
__________________________________________________________________________________________________
block4a_expand_activation (Acti (None, 28, 28, 240)  0           block4a_expand_bn[0][0]          
__________________________________________________________________________________________________
block4a_dwconv_pad (ZeroPadding (None, 29, 29, 240)  0           block4a_expand_activation[0][0]  
__________________________________________________________________________________________________
block4a_dwconv (DepthwiseConv2D (None, 14, 14, 240)  2160        block4a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block4a_bn (BatchNormalization) (None, 14, 14, 240)  960         block4a_dwconv[0][0]             
__________________________________________________________________________________________________
block4a_activation (Activation) (None, 14, 14, 240)  0           block4a_bn[0][0]                 
__________________________________________________________________________________________________
block4a_se_squeeze (GlobalAvera (None, 240)          0           block4a_activation[0][0]         
__________________________________________________________________________________________________
block4a_se_reshape (Reshape)    (None, 1, 1, 240)    0           block4a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block4a_se_reduce (Conv2D)      (None, 1, 1, 10)     2410        block4a_se_reshape[0][0]         
__________________________________________________________________________________________________
block4a_se_expand (Conv2D)      (None, 1, 1, 240)    2640        block4a_se_reduce[0][0]          
__________________________________________________________________________________________________
block4a_se_excite (Multiply)    (None, 14, 14, 240)  0           block4a_activation[0][0]         
                                                                 block4a_se_expand[0][0]          
__________________________________________________________________________________________________
block4a_project_conv (Conv2D)   (None, 14, 14, 80)   19200       block4a_se_excite[0][0]          
__________________________________________________________________________________________________
block4a_project_bn (BatchNormal (None, 14, 14, 80)   320         block4a_project_conv[0][0]       
__________________________________________________________________________________________________
block4b_expand_conv (Conv2D)    (None, 14, 14, 480)  38400       block4a_project_bn[0][0]         
__________________________________________________________________________________________________
block4b_expand_bn (BatchNormali (None, 14, 14, 480)  1920        block4b_expand_conv[0][0]        
__________________________________________________________________________________________________
block4b_expand_activation (Acti (None, 14, 14, 480)  0           block4b_expand_bn[0][0]          
__________________________________________________________________________________________________
block4b_dwconv (DepthwiseConv2D (None, 14, 14, 480)  4320        block4b_expand_activation[0][0]  
__________________________________________________________________________________________________
block4b_bn (BatchNormalization) (None, 14, 14, 480)  1920        block4b_dwconv[0][0]             
__________________________________________________________________________________________________
block4b_activation (Activation) (None, 14, 14, 480)  0           block4b_bn[0][0]                 
__________________________________________________________________________________________________
block4b_se_squeeze (GlobalAvera (None, 480)          0           block4b_activation[0][0]         
__________________________________________________________________________________________________
block4b_se_reshape (Reshape)    (None, 1, 1, 480)    0           block4b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block4b_se_reduce (Conv2D)      (None, 1, 1, 20)     9620        block4b_se_reshape[0][0]         
__________________________________________________________________________________________________
block4b_se_expand (Conv2D)      (None, 1, 1, 480)    10080       block4b_se_reduce[0][0]          
__________________________________________________________________________________________________
block4b_se_excite (Multiply)    (None, 14, 14, 480)  0           block4b_activation[0][0]         
                                                                 block4b_se_expand[0][0]          
__________________________________________________________________________________________________
block4b_project_conv (Conv2D)   (None, 14, 14, 80)   38400       block4b_se_excite[0][0]          
__________________________________________________________________________________________________
block4b_project_bn (BatchNormal (None, 14, 14, 80)   320         block4b_project_conv[0][0]       
__________________________________________________________________________________________________
block4b_drop (Dropout)          (None, 14, 14, 80)   0           block4b_project_bn[0][0]         
__________________________________________________________________________________________________
block4b_add (Add)               (None, 14, 14, 80)   0           block4b_drop[0][0]               
                                                                 block4a_project_bn[0][0]         
__________________________________________________________________________________________________
block4c_expand_conv (Conv2D)    (None, 14, 14, 480)  38400       block4b_add[0][0]                
__________________________________________________________________________________________________
block4c_expand_bn (BatchNormali (None, 14, 14, 480)  1920        block4c_expand_conv[0][0]        
__________________________________________________________________________________________________
block4c_expand_activation (Acti (None, 14, 14, 480)  0           block4c_expand_bn[0][0]          
__________________________________________________________________________________________________
block4c_dwconv (DepthwiseConv2D (None, 14, 14, 480)  4320        block4c_expand_activation[0][0]  
__________________________________________________________________________________________________
block4c_bn (BatchNormalization) (None, 14, 14, 480)  1920        block4c_dwconv[0][0]             
__________________________________________________________________________________________________
block4c_activation (Activation) (None, 14, 14, 480)  0           block4c_bn[0][0]                 
__________________________________________________________________________________________________
block4c_se_squeeze (GlobalAvera (None, 480)          0           block4c_activation[0][0]         
__________________________________________________________________________________________________
block4c_se_reshape (Reshape)    (None, 1, 1, 480)    0           block4c_se_squeeze[0][0]         
__________________________________________________________________________________________________
block4c_se_reduce (Conv2D)      (None, 1, 1, 20)     9620        block4c_se_reshape[0][0]         
__________________________________________________________________________________________________
block4c_se_expand (Conv2D)      (None, 1, 1, 480)    10080       block4c_se_reduce[0][0]          
__________________________________________________________________________________________________
block4c_se_excite (Multiply)    (None, 14, 14, 480)  0           block4c_activation[0][0]         
                                                                 block4c_se_expand[0][0]          
__________________________________________________________________________________________________
block4c_project_conv (Conv2D)   (None, 14, 14, 80)   38400       block4c_se_excite[0][0]          
__________________________________________________________________________________________________
block4c_project_bn (BatchNormal (None, 14, 14, 80)   320         block4c_project_conv[0][0]       
__________________________________________________________________________________________________
block4c_drop (Dropout)          (None, 14, 14, 80)   0           block4c_project_bn[0][0]         
__________________________________________________________________________________________________
block4c_add (Add)               (None, 14, 14, 80)   0           block4c_drop[0][0]               
                                                                 block4b_add[0][0]                
__________________________________________________________________________________________________
block5a_expand_conv (Conv2D)    (None, 14, 14, 480)  38400       block4c_add[0][0]                
__________________________________________________________________________________________________
block5a_expand_bn (BatchNormali (None, 14, 14, 480)  1920        block5a_expand_conv[0][0]        
__________________________________________________________________________________________________
block5a_expand_activation (Acti (None, 14, 14, 480)  0           block5a_expand_bn[0][0]          
__________________________________________________________________________________________________
block5a_dwconv (DepthwiseConv2D (None, 14, 14, 480)  12000       block5a_expand_activation[0][0]  
__________________________________________________________________________________________________
block5a_bn (BatchNormalization) (None, 14, 14, 480)  1920        block5a_dwconv[0][0]             
__________________________________________________________________________________________________
block5a_activation (Activation) (None, 14, 14, 480)  0           block5a_bn[0][0]                 
__________________________________________________________________________________________________
block5a_se_squeeze (GlobalAvera (None, 480)          0           block5a_activation[0][0]         
__________________________________________________________________________________________________
block5a_se_reshape (Reshape)    (None, 1, 1, 480)    0           block5a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block5a_se_reduce (Conv2D)      (None, 1, 1, 20)     9620        block5a_se_reshape[0][0]         
__________________________________________________________________________________________________
block5a_se_expand (Conv2D)      (None, 1, 1, 480)    10080       block5a_se_reduce[0][0]          
__________________________________________________________________________________________________
block5a_se_excite (Multiply)    (None, 14, 14, 480)  0           block5a_activation[0][0]         
                                                                 block5a_se_expand[0][0]          
__________________________________________________________________________________________________
block5a_project_conv (Conv2D)   (None, 14, 14, 112)  53760       block5a_se_excite[0][0]          
__________________________________________________________________________________________________
block5a_project_bn (BatchNormal (None, 14, 14, 112)  448         block5a_project_conv[0][0]       
__________________________________________________________________________________________________
block5b_expand_conv (Conv2D)    (None, 14, 14, 672)  75264       block5a_project_bn[0][0]         
__________________________________________________________________________________________________
block5b_expand_bn (BatchNormali (None, 14, 14, 672)  2688        block5b_expand_conv[0][0]        
__________________________________________________________________________________________________
block5b_expand_activation (Acti (None, 14, 14, 672)  0           block5b_expand_bn[0][0]          
__________________________________________________________________________________________________
block5b_dwconv (DepthwiseConv2D (None, 14, 14, 672)  16800       block5b_expand_activation[0][0]  
__________________________________________________________________________________________________
block5b_bn (BatchNormalization) (None, 14, 14, 672)  2688        block5b_dwconv[0][0]             
__________________________________________________________________________________________________
block5b_activation (Activation) (None, 14, 14, 672)  0           block5b_bn[0][0]                 
__________________________________________________________________________________________________
block5b_se_squeeze (GlobalAvera (None, 672)          0           block5b_activation[0][0]         
__________________________________________________________________________________________________
block5b_se_reshape (Reshape)    (None, 1, 1, 672)    0           block5b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block5b_se_reduce (Conv2D)      (None, 1, 1, 28)     18844       block5b_se_reshape[0][0]         
__________________________________________________________________________________________________
block5b_se_expand (Conv2D)      (None, 1, 1, 672)    19488       block5b_se_reduce[0][0]          
__________________________________________________________________________________________________
block5b_se_excite (Multiply)    (None, 14, 14, 672)  0           block5b_activation[0][0]         
                                                                 block5b_se_expand[0][0]          
__________________________________________________________________________________________________
block5b_project_conv (Conv2D)   (None, 14, 14, 112)  75264       block5b_se_excite[0][0]          
__________________________________________________________________________________________________
block5b_project_bn (BatchNormal (None, 14, 14, 112)  448         block5b_project_conv[0][0]       
__________________________________________________________________________________________________
block5b_drop (Dropout)          (None, 14, 14, 112)  0           block5b_project_bn[0][0]         
__________________________________________________________________________________________________
block5b_add (Add)               (None, 14, 14, 112)  0           block5b_drop[0][0]               
                                                                 block5a_project_bn[0][0]         
__________________________________________________________________________________________________
block5c_expand_conv (Conv2D)    (None, 14, 14, 672)  75264       block5b_add[0][0]                
__________________________________________________________________________________________________
block5c_expand_bn (BatchNormali (None, 14, 14, 672)  2688        block5c_expand_conv[0][0]        
__________________________________________________________________________________________________
block5c_expand_activation (Acti (None, 14, 14, 672)  0           block5c_expand_bn[0][0]          
__________________________________________________________________________________________________
block5c_dwconv (DepthwiseConv2D (None, 14, 14, 672)  16800       block5c_expand_activation[0][0]  
__________________________________________________________________________________________________
block5c_bn (BatchNormalization) (None, 14, 14, 672)  2688        block5c_dwconv[0][0]             
__________________________________________________________________________________________________
block5c_activation (Activation) (None, 14, 14, 672)  0           block5c_bn[0][0]                 
__________________________________________________________________________________________________
block5c_se_squeeze (GlobalAvera (None, 672)          0           block5c_activation[0][0]         
__________________________________________________________________________________________________
block5c_se_reshape (Reshape)    (None, 1, 1, 672)    0           block5c_se_squeeze[0][0]         
__________________________________________________________________________________________________
block5c_se_reduce (Conv2D)      (None, 1, 1, 28)     18844       block5c_se_reshape[0][0]         
__________________________________________________________________________________________________
block5c_se_expand (Conv2D)      (None, 1, 1, 672)    19488       block5c_se_reduce[0][0]          
__________________________________________________________________________________________________
block5c_se_excite (Multiply)    (None, 14, 14, 672)  0           block5c_activation[0][0]         
                                                                 block5c_se_expand[0][0]          
__________________________________________________________________________________________________
block5c_project_conv (Conv2D)   (None, 14, 14, 112)  75264       block5c_se_excite[0][0]          
__________________________________________________________________________________________________
block5c_project_bn (BatchNormal (None, 14, 14, 112)  448         block5c_project_conv[0][0]       
__________________________________________________________________________________________________
block5c_drop (Dropout)          (None, 14, 14, 112)  0           block5c_project_bn[0][0]         
__________________________________________________________________________________________________
block5c_add (Add)               (None, 14, 14, 112)  0           block5c_drop[0][0]               
                                                                 block5b_add[0][0]                
__________________________________________________________________________________________________
block6a_expand_conv (Conv2D)    (None, 14, 14, 672)  75264       block5c_add[0][0]                
__________________________________________________________________________________________________
block6a_expand_bn (BatchNormali (None, 14, 14, 672)  2688        block6a_expand_conv[0][0]        
__________________________________________________________________________________________________
block6a_expand_activation (Acti (None, 14, 14, 672)  0           block6a_expand_bn[0][0]          
__________________________________________________________________________________________________
block6a_dwconv_pad (ZeroPadding (None, 17, 17, 672)  0           block6a_expand_activation[0][0]  
__________________________________________________________________________________________________
block6a_dwconv (DepthwiseConv2D (None, 7, 7, 672)    16800       block6a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block6a_bn (BatchNormalization) (None, 7, 7, 672)    2688        block6a_dwconv[0][0]             
__________________________________________________________________________________________________
block6a_activation (Activation) (None, 7, 7, 672)    0           block6a_bn[0][0]                 
__________________________________________________________________________________________________
block6a_se_squeeze (GlobalAvera (None, 672)          0           block6a_activation[0][0]         
__________________________________________________________________________________________________
block6a_se_reshape (Reshape)    (None, 1, 1, 672)    0           block6a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6a_se_reduce (Conv2D)      (None, 1, 1, 28)     18844       block6a_se_reshape[0][0]         
__________________________________________________________________________________________________
block6a_se_expand (Conv2D)      (None, 1, 1, 672)    19488       block6a_se_reduce[0][0]          
__________________________________________________________________________________________________
block6a_se_excite (Multiply)    (None, 7, 7, 672)    0           block6a_activation[0][0]         
                                                                 block6a_se_expand[0][0]          
__________________________________________________________________________________________________
block6a_project_conv (Conv2D)   (None, 7, 7, 192)    129024      block6a_se_excite[0][0]          
__________________________________________________________________________________________________
block6a_project_bn (BatchNormal (None, 7, 7, 192)    768         block6a_project_conv[0][0]       
__________________________________________________________________________________________________
block6b_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6a_project_bn[0][0]         
__________________________________________________________________________________________________
block6b_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block6b_expand_conv[0][0]        
__________________________________________________________________________________________________
block6b_expand_activation (Acti (None, 7, 7, 1152)   0           block6b_expand_bn[0][0]          
__________________________________________________________________________________________________
block6b_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   28800       block6b_expand_activation[0][0]  
__________________________________________________________________________________________________
block6b_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block6b_dwconv[0][0]             
__________________________________________________________________________________________________
block6b_activation (Activation) (None, 7, 7, 1152)   0           block6b_bn[0][0]                 
__________________________________________________________________________________________________
block6b_se_squeeze (GlobalAvera (None, 1152)         0           block6b_activation[0][0]         
__________________________________________________________________________________________________
block6b_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block6b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6b_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block6b_se_reshape[0][0]         
__________________________________________________________________________________________________
block6b_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block6b_se_reduce[0][0]          
__________________________________________________________________________________________________
block6b_se_excite (Multiply)    (None, 7, 7, 1152)   0           block6b_activation[0][0]         
                                                                 block6b_se_expand[0][0]          
__________________________________________________________________________________________________
block6b_project_conv (Conv2D)   (None, 7, 7, 192)    221184      block6b_se_excite[0][0]          
__________________________________________________________________________________________________
block6b_project_bn (BatchNormal (None, 7, 7, 192)    768         block6b_project_conv[0][0]       
__________________________________________________________________________________________________
block6b_drop (Dropout)          (None, 7, 7, 192)    0           block6b_project_bn[0][0]         
__________________________________________________________________________________________________
block6b_add (Add)               (None, 7, 7, 192)    0           block6b_drop[0][0]               
                                                                 block6a_project_bn[0][0]         
__________________________________________________________________________________________________
block6c_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6b_add[0][0]                
__________________________________________________________________________________________________
block6c_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block6c_expand_conv[0][0]        
__________________________________________________________________________________________________
block6c_expand_activation (Acti (None, 7, 7, 1152)   0           block6c_expand_bn[0][0]          
__________________________________________________________________________________________________
block6c_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   28800       block6c_expand_activation[0][0]  
__________________________________________________________________________________________________
block6c_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block6c_dwconv[0][0]             
__________________________________________________________________________________________________
block6c_activation (Activation) (None, 7, 7, 1152)   0           block6c_bn[0][0]                 
__________________________________________________________________________________________________
block6c_se_squeeze (GlobalAvera (None, 1152)         0           block6c_activation[0][0]         
__________________________________________________________________________________________________
block6c_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block6c_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6c_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block6c_se_reshape[0][0]         
__________________________________________________________________________________________________
block6c_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block6c_se_reduce[0][0]          
__________________________________________________________________________________________________
block6c_se_excite (Multiply)    (None, 7, 7, 1152)   0           block6c_activation[0][0]         
                                                                 block6c_se_expand[0][0]          
__________________________________________________________________________________________________
block6c_project_conv (Conv2D)   (None, 7, 7, 192)    221184      block6c_se_excite[0][0]          
__________________________________________________________________________________________________
block6c_project_bn (BatchNormal (None, 7, 7, 192)    768         block6c_project_conv[0][0]       
__________________________________________________________________________________________________
block6c_drop (Dropout)          (None, 7, 7, 192)    0           block6c_project_bn[0][0]         
__________________________________________________________________________________________________
block6c_add (Add)               (None, 7, 7, 192)    0           block6c_drop[0][0]               
                                                                 block6b_add[0][0]                
__________________________________________________________________________________________________
block6d_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6c_add[0][0]                
__________________________________________________________________________________________________
block6d_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block6d_expand_conv[0][0]        
__________________________________________________________________________________________________
block6d_expand_activation (Acti (None, 7, 7, 1152)   0           block6d_expand_bn[0][0]          
__________________________________________________________________________________________________
block6d_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   28800       block6d_expand_activation[0][0]  
__________________________________________________________________________________________________
block6d_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block6d_dwconv[0][0]             
__________________________________________________________________________________________________
block6d_activation (Activation) (None, 7, 7, 1152)   0           block6d_bn[0][0]                 
__________________________________________________________________________________________________
block6d_se_squeeze (GlobalAvera (None, 1152)         0           block6d_activation[0][0]         
__________________________________________________________________________________________________
block6d_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block6d_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6d_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block6d_se_reshape[0][0]         
__________________________________________________________________________________________________
block6d_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block6d_se_reduce[0][0]          
__________________________________________________________________________________________________
block6d_se_excite (Multiply)    (None, 7, 7, 1152)   0           block6d_activation[0][0]         
                                                                 block6d_se_expand[0][0]          
__________________________________________________________________________________________________
block6d_project_conv (Conv2D)   (None, 7, 7, 192)    221184      block6d_se_excite[0][0]          
__________________________________________________________________________________________________
block6d_project_bn (BatchNormal (None, 7, 7, 192)    768         block6d_project_conv[0][0]       
__________________________________________________________________________________________________
block6d_drop (Dropout)          (None, 7, 7, 192)    0           block6d_project_bn[0][0]         
__________________________________________________________________________________________________
block6d_add (Add)               (None, 7, 7, 192)    0           block6d_drop[0][0]               
                                                                 block6c_add[0][0]                
__________________________________________________________________________________________________
block7a_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6d_add[0][0]                
__________________________________________________________________________________________________
block7a_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block7a_expand_conv[0][0]        
__________________________________________________________________________________________________
block7a_expand_activation (Acti (None, 7, 7, 1152)   0           block7a_expand_bn[0][0]          
__________________________________________________________________________________________________
block7a_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   10368       block7a_expand_activation[0][0]  
__________________________________________________________________________________________________
block7a_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block7a_dwconv[0][0]             
__________________________________________________________________________________________________
block7a_activation (Activation) (None, 7, 7, 1152)   0           block7a_bn[0][0]                 
__________________________________________________________________________________________________
block7a_se_squeeze (GlobalAvera (None, 1152)         0           block7a_activation[0][0]         
__________________________________________________________________________________________________
block7a_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block7a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block7a_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block7a_se_reshape[0][0]         
__________________________________________________________________________________________________
block7a_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block7a_se_reduce[0][0]          
__________________________________________________________________________________________________
block7a_se_excite (Multiply)    (None, 7, 7, 1152)   0           block7a_activation[0][0]         
                                                                 block7a_se_expand[0][0]          
__________________________________________________________________________________________________
block7a_project_conv (Conv2D)   (None, 7, 7, 320)    368640      block7a_se_excite[0][0]          
__________________________________________________________________________________________________
block7a_project_bn (BatchNormal (None, 7, 7, 320)    1280        block7a_project_conv[0][0]       
__________________________________________________________________________________________________
top_conv (Conv2D)               (None, 7, 7, 1280)   409600      block7a_project_bn[0][0]         
__________________________________________________________________________________________________
top_bn (BatchNormalization)     (None, 7, 7, 1280)   5120        top_conv[0][0]                   
__________________________________________________________________________________________________
top_activation (Activation)     (None, 7, 7, 1280)   0           top_bn[0][0]                     
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 1280)         0           top_activation[0][0]             
__________________________________________________________________________________________________
top_dropout (Dropout)           (None, 1280)         0           avg_pool[0][0]                   
__________________________________________________________________________________________________
predictions (Dense)             (None, 1000)         1281000     top_dropout[0][0]                
==================================================================================================
Total params: 5,330,564
Trainable params: 5,288,548
Non-trainable params: 42,016
__________________________________________

我们没有用源码中预处理那一步，所以少了7个参数，直接按照源码搭建出来应该参数量为5330571。

到这里，网络就搭建完了，里面细节实在是太多了。

什么？你说上面的代码太麻烦了，那我们直接当调包侠吧，看下面得到迁移学习API

9、Tensorflow官方API快速复现EfficientNetB0(快速实现方式)

直接调API就行，就相当于我们调的就是github上面的tensorflow源码，这样虽然快，但是在没弄懂网络结构之前还是不推荐用这种，虽然爽，但是不知道怎么搭建的。

import tensorflow as tf
from tensorflow.keras.applications import EfficientNetB0
from tensorflow.keras.utils import plot_model

model=EfficientNetB0(
    include_top=True,
    weights='imagenet',
    input_tensor=None,
    input_shape=None,
    pooling=None,
    classes=1000,
    classifier_activation='softmax'
)

model.summary()

Model: "efficientnetb0"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
rescaling (Rescaling)           (None, 224, 224, 3)  0           input_1[0][0]                    
__________________________________________________________________________________________________
normalization (Normalization)   (None, 224, 224, 3)  7           rescaling[0][0]                  
__________________________________________________________________________________________________
stem_conv_pad (ZeroPadding2D)   (None, 225, 225, 3)  0           normalization[0][0]              
__________________________________________________________________________________________________
stem_conv (Conv2D)              (None, 112, 112, 32) 864         stem_conv_pad[0][0]              
__________________________________________________________________________________________________
stem_bn (BatchNormalization)    (None, 112, 112, 32) 128         stem_conv[0][0]                  
__________________________________________________________________________________________________
stem_activation (Activation)    (None, 112, 112, 32) 0           stem_bn[0][0]                    
__________________________________________________________________________________________________
block1a_dwconv (DepthwiseConv2D (None, 112, 112, 32) 288         stem_activation[0][0]            
__________________________________________________________________________________________________
block1a_bn (BatchNormalization) (None, 112, 112, 32) 128         block1a_dwconv[0][0]             
__________________________________________________________________________________________________
block1a_activation (Activation) (None, 112, 112, 32) 0           block1a_bn[0][0]                 
__________________________________________________________________________________________________
block1a_se_squeeze (GlobalAvera (None, 32)           0           block1a_activation[0][0]         
__________________________________________________________________________________________________
block1a_se_reshape (Reshape)    (None, 1, 1, 32)     0           block1a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block1a_se_reduce (Conv2D)      (None, 1, 1, 8)      264         block1a_se_reshape[0][0]         
__________________________________________________________________________________________________
block1a_se_expand (Conv2D)      (None, 1, 1, 32)     288         block1a_se_reduce[0][0]          
__________________________________________________________________________________________________
block1a_se_excite (Multiply)    (None, 112, 112, 32) 0           block1a_activation[0][0]         
                                                                 block1a_se_expand[0][0]          
__________________________________________________________________________________________________
block1a_project_conv (Conv2D)   (None, 112, 112, 16) 512         block1a_se_excite[0][0]          
__________________________________________________________________________________________________
block1a_project_bn (BatchNormal (None, 112, 112, 16) 64          block1a_project_conv[0][0]       
__________________________________________________________________________________________________
block2a_expand_conv (Conv2D)    (None, 112, 112, 96) 1536        block1a_project_bn[0][0]         
__________________________________________________________________________________________________
block2a_expand_bn (BatchNormali (None, 112, 112, 96) 384         block2a_expand_conv[0][0]        
__________________________________________________________________________________________________
block2a_expand_activation (Acti (None, 112, 112, 96) 0           block2a_expand_bn[0][0]          
__________________________________________________________________________________________________
block2a_dwconv_pad (ZeroPadding (None, 113, 113, 96) 0           block2a_expand_activation[0][0]  
__________________________________________________________________________________________________
block2a_dwconv (DepthwiseConv2D (None, 56, 56, 96)   864         block2a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block2a_bn (BatchNormalization) (None, 56, 56, 96)   384         block2a_dwconv[0][0]             
__________________________________________________________________________________________________
block2a_activation (Activation) (None, 56, 56, 96)   0           block2a_bn[0][0]                 
__________________________________________________________________________________________________
block2a_se_squeeze (GlobalAvera (None, 96)           0           block2a_activation[0][0]         
__________________________________________________________________________________________________
block2a_se_reshape (Reshape)    (None, 1, 1, 96)     0           block2a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block2a_se_reduce (Conv2D)      (None, 1, 1, 4)      388         block2a_se_reshape[0][0]         
__________________________________________________________________________________________________
block2a_se_expand (Conv2D)      (None, 1, 1, 96)     480         block2a_se_reduce[0][0]          
__________________________________________________________________________________________________
block2a_se_excite (Multiply)    (None, 56, 56, 96)   0           block2a_activation[0][0]         
                                                                 block2a_se_expand[0][0]          
__________________________________________________________________________________________________
block2a_project_conv (Conv2D)   (None, 56, 56, 24)   2304        block2a_se_excite[0][0]          
__________________________________________________________________________________________________
block2a_project_bn (BatchNormal (None, 56, 56, 24)   96          block2a_project_conv[0][0]       
__________________________________________________________________________________________________
block2b_expand_conv (Conv2D)    (None, 56, 56, 144)  3456        block2a_project_bn[0][0]         
__________________________________________________________________________________________________
block2b_expand_bn (BatchNormali (None, 56, 56, 144)  576         block2b_expand_conv[0][0]        
__________________________________________________________________________________________________
block2b_expand_activation (Acti (None, 56, 56, 144)  0           block2b_expand_bn[0][0]          
__________________________________________________________________________________________________
block2b_dwconv (DepthwiseConv2D (None, 56, 56, 144)  1296        block2b_expand_activation[0][0]  
__________________________________________________________________________________________________
block2b_bn (BatchNormalization) (None, 56, 56, 144)  576         block2b_dwconv[0][0]             
__________________________________________________________________________________________________
block2b_activation (Activation) (None, 56, 56, 144)  0           block2b_bn[0][0]                 
__________________________________________________________________________________________________
block2b_se_squeeze (GlobalAvera (None, 144)          0           block2b_activation[0][0]         
__________________________________________________________________________________________________
block2b_se_reshape (Reshape)    (None, 1, 1, 144)    0           block2b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block2b_se_reduce (Conv2D)      (None, 1, 1, 6)      870         block2b_se_reshape[0][0]         
__________________________________________________________________________________________________
block2b_se_expand (Conv2D)      (None, 1, 1, 144)    1008        block2b_se_reduce[0][0]          
__________________________________________________________________________________________________
block2b_se_excite (Multiply)    (None, 56, 56, 144)  0           block2b_activation[0][0]         
                                                                 block2b_se_expand[0][0]          
__________________________________________________________________________________________________
block2b_project_conv (Conv2D)   (None, 56, 56, 24)   3456        block2b_se_excite[0][0]          
__________________________________________________________________________________________________
block2b_project_bn (BatchNormal (None, 56, 56, 24)   96          block2b_project_conv[0][0]       
__________________________________________________________________________________________________
block2b_drop (Dropout)          (None, 56, 56, 24)   0           block2b_project_bn[0][0]         
__________________________________________________________________________________________________
block2b_add (Add)               (None, 56, 56, 24)   0           block2b_drop[0][0]               
                                                                 block2a_project_bn[0][0]         
__________________________________________________________________________________________________
block3a_expand_conv (Conv2D)    (None, 56, 56, 144)  3456        block2b_add[0][0]                
__________________________________________________________________________________________________
block3a_expand_bn (BatchNormali (None, 56, 56, 144)  576         block3a_expand_conv[0][0]        
__________________________________________________________________________________________________
block3a_expand_activation (Acti (None, 56, 56, 144)  0           block3a_expand_bn[0][0]          
__________________________________________________________________________________________________
block3a_dwconv_pad (ZeroPadding (None, 59, 59, 144)  0           block3a_expand_activation[0][0]  
__________________________________________________________________________________________________
block3a_dwconv (DepthwiseConv2D (None, 28, 28, 144)  3600        block3a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block3a_bn (BatchNormalization) (None, 28, 28, 144)  576         block3a_dwconv[0][0]             
__________________________________________________________________________________________________
block3a_activation (Activation) (None, 28, 28, 144)  0           block3a_bn[0][0]                 
__________________________________________________________________________________________________
block3a_se_squeeze (GlobalAvera (None, 144)          0           block3a_activation[0][0]         
__________________________________________________________________________________________________
block3a_se_reshape (Reshape)    (None, 1, 1, 144)    0           block3a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block3a_se_reduce (Conv2D)      (None, 1, 1, 6)      870         block3a_se_reshape[0][0]         
__________________________________________________________________________________________________
block3a_se_expand (Conv2D)      (None, 1, 1, 144)    1008        block3a_se_reduce[0][0]          
__________________________________________________________________________________________________
block3a_se_excite (Multiply)    (None, 28, 28, 144)  0           block3a_activation[0][0]         
                                                                 block3a_se_expand[0][0]          
__________________________________________________________________________________________________
block3a_project_conv (Conv2D)   (None, 28, 28, 40)   5760        block3a_se_excite[0][0]          
__________________________________________________________________________________________________
block3a_project_bn (BatchNormal (None, 28, 28, 40)   160         block3a_project_conv[0][0]       
__________________________________________________________________________________________________
block3b_expand_conv (Conv2D)    (None, 28, 28, 240)  9600        block3a_project_bn[0][0]         
__________________________________________________________________________________________________
block3b_expand_bn (BatchNormali (None, 28, 28, 240)  960         block3b_expand_conv[0][0]        
__________________________________________________________________________________________________
block3b_expand_activation (Acti (None, 28, 28, 240)  0           block3b_expand_bn[0][0]          
__________________________________________________________________________________________________
block3b_dwconv (DepthwiseConv2D (None, 28, 28, 240)  6000        block3b_expand_activation[0][0]  
__________________________________________________________________________________________________
block3b_bn (BatchNormalization) (None, 28, 28, 240)  960         block3b_dwconv[0][0]             
__________________________________________________________________________________________________
block3b_activation (Activation) (None, 28, 28, 240)  0           block3b_bn[0][0]                 
__________________________________________________________________________________________________
block3b_se_squeeze (GlobalAvera (None, 240)          0           block3b_activation[0][0]         
__________________________________________________________________________________________________
block3b_se_reshape (Reshape)    (None, 1, 1, 240)    0           block3b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block3b_se_reduce (Conv2D)      (None, 1, 1, 10)     2410        block3b_se_reshape[0][0]         
__________________________________________________________________________________________________
block3b_se_expand (Conv2D)      (None, 1, 1, 240)    2640        block3b_se_reduce[0][0]          
__________________________________________________________________________________________________
block3b_se_excite (Multiply)    (None, 28, 28, 240)  0           block3b_activation[0][0]         
                                                                 block3b_se_expand[0][0]          
__________________________________________________________________________________________________
block3b_project_conv (Conv2D)   (None, 28, 28, 40)   9600        block3b_se_excite[0][0]          
__________________________________________________________________________________________________
block3b_project_bn (BatchNormal (None, 28, 28, 40)   160         block3b_project_conv[0][0]       
__________________________________________________________________________________________________
block3b_drop (Dropout)          (None, 28, 28, 40)   0           block3b_project_bn[0][0]         
__________________________________________________________________________________________________
block3b_add (Add)               (None, 28, 28, 40)   0           block3b_drop[0][0]               
                                                                 block3a_project_bn[0][0]         
__________________________________________________________________________________________________
block4a_expand_conv (Conv2D)    (None, 28, 28, 240)  9600        block3b_add[0][0]                
__________________________________________________________________________________________________
block4a_expand_bn (BatchNormali (None, 28, 28, 240)  960         block4a_expand_conv[0][0]        
__________________________________________________________________________________________________
block4a_expand_activation (Acti (None, 28, 28, 240)  0           block4a_expand_bn[0][0]          
__________________________________________________________________________________________________
block4a_dwconv_pad (ZeroPadding (None, 29, 29, 240)  0           block4a_expand_activation[0][0]  
__________________________________________________________________________________________________
block4a_dwconv (DepthwiseConv2D (None, 14, 14, 240)  2160        block4a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block4a_bn (BatchNormalization) (None, 14, 14, 240)  960         block4a_dwconv[0][0]             
__________________________________________________________________________________________________
block4a_activation (Activation) (None, 14, 14, 240)  0           block4a_bn[0][0]                 
__________________________________________________________________________________________________
block4a_se_squeeze (GlobalAvera (None, 240)          0           block4a_activation[0][0]         
__________________________________________________________________________________________________
block4a_se_reshape (Reshape)    (None, 1, 1, 240)    0           block4a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block4a_se_reduce (Conv2D)      (None, 1, 1, 10)     2410        block4a_se_reshape[0][0]         
__________________________________________________________________________________________________
block4a_se_expand (Conv2D)      (None, 1, 1, 240)    2640        block4a_se_reduce[0][0]          
__________________________________________________________________________________________________
block4a_se_excite (Multiply)    (None, 14, 14, 240)  0           block4a_activation[0][0]         
                                                                 block4a_se_expand[0][0]          
__________________________________________________________________________________________________
block4a_project_conv (Conv2D)   (None, 14, 14, 80)   19200       block4a_se_excite[0][0]          
__________________________________________________________________________________________________
block4a_project_bn (BatchNormal (None, 14, 14, 80)   320         block4a_project_conv[0][0]       
__________________________________________________________________________________________________
block4b_expand_conv (Conv2D)    (None, 14, 14, 480)  38400       block4a_project_bn[0][0]         
__________________________________________________________________________________________________
block4b_expand_bn (BatchNormali (None, 14, 14, 480)  1920        block4b_expand_conv[0][0]        
__________________________________________________________________________________________________
block4b_expand_activation (Acti (None, 14, 14, 480)  0           block4b_expand_bn[0][0]          
__________________________________________________________________________________________________
block4b_dwconv (DepthwiseConv2D (None, 14, 14, 480)  4320        block4b_expand_activation[0][0]  
__________________________________________________________________________________________________
block4b_bn (BatchNormalization) (None, 14, 14, 480)  1920        block4b_dwconv[0][0]             
__________________________________________________________________________________________________
block4b_activation (Activation) (None, 14, 14, 480)  0           block4b_bn[0][0]                 
__________________________________________________________________________________________________
block4b_se_squeeze (GlobalAvera (None, 480)          0           block4b_activation[0][0]         
__________________________________________________________________________________________________
block4b_se_reshape (Reshape)    (None, 1, 1, 480)    0           block4b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block4b_se_reduce (Conv2D)      (None, 1, 1, 20)     9620        block4b_se_reshape[0][0]         
__________________________________________________________________________________________________
block4b_se_expand (Conv2D)      (None, 1, 1, 480)    10080       block4b_se_reduce[0][0]          
__________________________________________________________________________________________________
block4b_se_excite (Multiply)    (None, 14, 14, 480)  0           block4b_activation[0][0]         
                                                                 block4b_se_expand[0][0]          
__________________________________________________________________________________________________
block4b_project_conv (Conv2D)   (None, 14, 14, 80)   38400       block4b_se_excite[0][0]          
__________________________________________________________________________________________________
block4b_project_bn (BatchNormal (None, 14, 14, 80)   320         block4b_project_conv[0][0]       
__________________________________________________________________________________________________
block4b_drop (Dropout)          (None, 14, 14, 80)   0           block4b_project_bn[0][0]         
__________________________________________________________________________________________________
block4b_add (Add)               (None, 14, 14, 80)   0           block4b_drop[0][0]               
                                                                 block4a_project_bn[0][0]         
__________________________________________________________________________________________________
block4c_expand_conv (Conv2D)    (None, 14, 14, 480)  38400       block4b_add[0][0]                
__________________________________________________________________________________________________
block4c_expand_bn (BatchNormali (None, 14, 14, 480)  1920        block4c_expand_conv[0][0]        
__________________________________________________________________________________________________
block4c_expand_activation (Acti (None, 14, 14, 480)  0           block4c_expand_bn[0][0]          
__________________________________________________________________________________________________
block4c_dwconv (DepthwiseConv2D (None, 14, 14, 480)  4320        block4c_expand_activation[0][0]  
__________________________________________________________________________________________________
block4c_bn (BatchNormalization) (None, 14, 14, 480)  1920        block4c_dwconv[0][0]             
__________________________________________________________________________________________________
block4c_activation (Activation) (None, 14, 14, 480)  0           block4c_bn[0][0]                 
__________________________________________________________________________________________________
block4c_se_squeeze (GlobalAvera (None, 480)          0           block4c_activation[0][0]         
__________________________________________________________________________________________________
block4c_se_reshape (Reshape)    (None, 1, 1, 480)    0           block4c_se_squeeze[0][0]         
__________________________________________________________________________________________________
block4c_se_reduce (Conv2D)      (None, 1, 1, 20)     9620        block4c_se_reshape[0][0]         
__________________________________________________________________________________________________
block4c_se_expand (Conv2D)      (None, 1, 1, 480)    10080       block4c_se_reduce[0][0]          
__________________________________________________________________________________________________
block4c_se_excite (Multiply)    (None, 14, 14, 480)  0           block4c_activation[0][0]         
                                                                 block4c_se_expand[0][0]          
__________________________________________________________________________________________________
block4c_project_conv (Conv2D)   (None, 14, 14, 80)   38400       block4c_se_excite[0][0]          
__________________________________________________________________________________________________
block4c_project_bn (BatchNormal (None, 14, 14, 80)   320         block4c_project_conv[0][0]       
__________________________________________________________________________________________________
block4c_drop (Dropout)          (None, 14, 14, 80)   0           block4c_project_bn[0][0]         
__________________________________________________________________________________________________
block4c_add (Add)               (None, 14, 14, 80)   0           block4c_drop[0][0]               
                                                                 block4b_add[0][0]                
__________________________________________________________________________________________________
block5a_expand_conv (Conv2D)    (None, 14, 14, 480)  38400       block4c_add[0][0]                
__________________________________________________________________________________________________
block5a_expand_bn (BatchNormali (None, 14, 14, 480)  1920        block5a_expand_conv[0][0]        
__________________________________________________________________________________________________
block5a_expand_activation (Acti (None, 14, 14, 480)  0           block5a_expand_bn[0][0]          
__________________________________________________________________________________________________
block5a_dwconv (DepthwiseConv2D (None, 14, 14, 480)  12000       block5a_expand_activation[0][0]  
__________________________________________________________________________________________________
block5a_bn (BatchNormalization) (None, 14, 14, 480)  1920        block5a_dwconv[0][0]             
__________________________________________________________________________________________________
block5a_activation (Activation) (None, 14, 14, 480)  0           block5a_bn[0][0]                 
__________________________________________________________________________________________________
block5a_se_squeeze (GlobalAvera (None, 480)          0           block5a_activation[0][0]         
__________________________________________________________________________________________________
block5a_se_reshape (Reshape)    (None, 1, 1, 480)    0           block5a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block5a_se_reduce (Conv2D)      (None, 1, 1, 20)     9620        block5a_se_reshape[0][0]         
__________________________________________________________________________________________________
block5a_se_expand (Conv2D)      (None, 1, 1, 480)    10080       block5a_se_reduce[0][0]          
__________________________________________________________________________________________________
block5a_se_excite (Multiply)    (None, 14, 14, 480)  0           block5a_activation[0][0]         
                                                                 block5a_se_expand[0][0]          
__________________________________________________________________________________________________
block5a_project_conv (Conv2D)   (None, 14, 14, 112)  53760       block5a_se_excite[0][0]          
__________________________________________________________________________________________________
block5a_project_bn (BatchNormal (None, 14, 14, 112)  448         block5a_project_conv[0][0]       
__________________________________________________________________________________________________
block5b_expand_conv (Conv2D)    (None, 14, 14, 672)  75264       block5a_project_bn[0][0]         
__________________________________________________________________________________________________
block5b_expand_bn (BatchNormali (None, 14, 14, 672)  2688        block5b_expand_conv[0][0]        
__________________________________________________________________________________________________
block5b_expand_activation (Acti (None, 14, 14, 672)  0           block5b_expand_bn[0][0]          
__________________________________________________________________________________________________
block5b_dwconv (DepthwiseConv2D (None, 14, 14, 672)  16800       block5b_expand_activation[0][0]  
__________________________________________________________________________________________________
block5b_bn (BatchNormalization) (None, 14, 14, 672)  2688        block5b_dwconv[0][0]             
__________________________________________________________________________________________________
block5b_activation (Activation) (None, 14, 14, 672)  0           block5b_bn[0][0]                 
__________________________________________________________________________________________________
block5b_se_squeeze (GlobalAvera (None, 672)          0           block5b_activation[0][0]         
__________________________________________________________________________________________________
block5b_se_reshape (Reshape)    (None, 1, 1, 672)    0           block5b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block5b_se_reduce (Conv2D)      (None, 1, 1, 28)     18844       block5b_se_reshape[0][0]         
__________________________________________________________________________________________________
block5b_se_expand (Conv2D)      (None, 1, 1, 672)    19488       block5b_se_reduce[0][0]          
__________________________________________________________________________________________________
block5b_se_excite (Multiply)    (None, 14, 14, 672)  0           block5b_activation[0][0]         
                                                                 block5b_se_expand[0][0]          
__________________________________________________________________________________________________
block5b_project_conv (Conv2D)   (None, 14, 14, 112)  75264       block5b_se_excite[0][0]          
__________________________________________________________________________________________________
block5b_project_bn (BatchNormal (None, 14, 14, 112)  448         block5b_project_conv[0][0]       
__________________________________________________________________________________________________
block5b_drop (Dropout)          (None, 14, 14, 112)  0           block5b_project_bn[0][0]         
__________________________________________________________________________________________________
block5b_add (Add)               (None, 14, 14, 112)  0           block5b_drop[0][0]               
                                                                 block5a_project_bn[0][0]         
__________________________________________________________________________________________________
block5c_expand_conv (Conv2D)    (None, 14, 14, 672)  75264       block5b_add[0][0]                
__________________________________________________________________________________________________
block5c_expand_bn (BatchNormali (None, 14, 14, 672)  2688        block5c_expand_conv[0][0]        
__________________________________________________________________________________________________
block5c_expand_activation (Acti (None, 14, 14, 672)  0           block5c_expand_bn[0][0]          
__________________________________________________________________________________________________
block5c_dwconv (DepthwiseConv2D (None, 14, 14, 672)  16800       block5c_expand_activation[0][0]  
__________________________________________________________________________________________________
block5c_bn (BatchNormalization) (None, 14, 14, 672)  2688        block5c_dwconv[0][0]             
__________________________________________________________________________________________________
block5c_activation (Activation) (None, 14, 14, 672)  0           block5c_bn[0][0]                 
__________________________________________________________________________________________________
block5c_se_squeeze (GlobalAvera (None, 672)          0           block5c_activation[0][0]         
__________________________________________________________________________________________________
block5c_se_reshape (Reshape)    (None, 1, 1, 672)    0           block5c_se_squeeze[0][0]         
__________________________________________________________________________________________________
block5c_se_reduce (Conv2D)      (None, 1, 1, 28)     18844       block5c_se_reshape[0][0]         
__________________________________________________________________________________________________
block5c_se_expand (Conv2D)      (None, 1, 1, 672)    19488       block5c_se_reduce[0][0]          
__________________________________________________________________________________________________
block5c_se_excite (Multiply)    (None, 14, 14, 672)  0           block5c_activation[0][0]         
                                                                 block5c_se_expand[0][0]          
__________________________________________________________________________________________________
block5c_project_conv (Conv2D)   (None, 14, 14, 112)  75264       block5c_se_excite[0][0]          
__________________________________________________________________________________________________
block5c_project_bn (BatchNormal (None, 14, 14, 112)  448         block5c_project_conv[0][0]       
__________________________________________________________________________________________________
block5c_drop (Dropout)          (None, 14, 14, 112)  0           block5c_project_bn[0][0]         
__________________________________________________________________________________________________
block5c_add (Add)               (None, 14, 14, 112)  0           block5c_drop[0][0]               
                                                                 block5b_add[0][0]                
__________________________________________________________________________________________________
block6a_expand_conv (Conv2D)    (None, 14, 14, 672)  75264       block5c_add[0][0]                
__________________________________________________________________________________________________
block6a_expand_bn (BatchNormali (None, 14, 14, 672)  2688        block6a_expand_conv[0][0]        
__________________________________________________________________________________________________
block6a_expand_activation (Acti (None, 14, 14, 672)  0           block6a_expand_bn[0][0]          
__________________________________________________________________________________________________
block6a_dwconv_pad (ZeroPadding (None, 17, 17, 672)  0           block6a_expand_activation[0][0]  
__________________________________________________________________________________________________
block6a_dwconv (DepthwiseConv2D (None, 7, 7, 672)    16800       block6a_dwconv_pad[0][0]         
__________________________________________________________________________________________________
block6a_bn (BatchNormalization) (None, 7, 7, 672)    2688        block6a_dwconv[0][0]             
__________________________________________________________________________________________________
block6a_activation (Activation) (None, 7, 7, 672)    0           block6a_bn[0][0]                 
__________________________________________________________________________________________________
block6a_se_squeeze (GlobalAvera (None, 672)          0           block6a_activation[0][0]         
__________________________________________________________________________________________________
block6a_se_reshape (Reshape)    (None, 1, 1, 672)    0           block6a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6a_se_reduce (Conv2D)      (None, 1, 1, 28)     18844       block6a_se_reshape[0][0]         
__________________________________________________________________________________________________
block6a_se_expand (Conv2D)      (None, 1, 1, 672)    19488       block6a_se_reduce[0][0]          
__________________________________________________________________________________________________
block6a_se_excite (Multiply)    (None, 7, 7, 672)    0           block6a_activation[0][0]         
                                                                 block6a_se_expand[0][0]          
__________________________________________________________________________________________________
block6a_project_conv (Conv2D)   (None, 7, 7, 192)    129024      block6a_se_excite[0][0]          
__________________________________________________________________________________________________
block6a_project_bn (BatchNormal (None, 7, 7, 192)    768         block6a_project_conv[0][0]       
__________________________________________________________________________________________________
block6b_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6a_project_bn[0][0]         
__________________________________________________________________________________________________
block6b_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block6b_expand_conv[0][0]        
__________________________________________________________________________________________________
block6b_expand_activation (Acti (None, 7, 7, 1152)   0           block6b_expand_bn[0][0]          
__________________________________________________________________________________________________
block6b_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   28800       block6b_expand_activation[0][0]  
__________________________________________________________________________________________________
block6b_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block6b_dwconv[0][0]             
__________________________________________________________________________________________________
block6b_activation (Activation) (None, 7, 7, 1152)   0           block6b_bn[0][0]                 
__________________________________________________________________________________________________
block6b_se_squeeze (GlobalAvera (None, 1152)         0           block6b_activation[0][0]         
__________________________________________________________________________________________________
block6b_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block6b_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6b_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block6b_se_reshape[0][0]         
__________________________________________________________________________________________________
block6b_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block6b_se_reduce[0][0]          
__________________________________________________________________________________________________
block6b_se_excite (Multiply)    (None, 7, 7, 1152)   0           block6b_activation[0][0]         
                                                                 block6b_se_expand[0][0]          
__________________________________________________________________________________________________
block6b_project_conv (Conv2D)   (None, 7, 7, 192)    221184      block6b_se_excite[0][0]          
__________________________________________________________________________________________________
block6b_project_bn (BatchNormal (None, 7, 7, 192)    768         block6b_project_conv[0][0]       
__________________________________________________________________________________________________
block6b_drop (Dropout)          (None, 7, 7, 192)    0           block6b_project_bn[0][0]         
__________________________________________________________________________________________________
block6b_add (Add)               (None, 7, 7, 192)    0           block6b_drop[0][0]               
                                                                 block6a_project_bn[0][0]         
__________________________________________________________________________________________________
block6c_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6b_add[0][0]                
__________________________________________________________________________________________________
block6c_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block6c_expand_conv[0][0]        
__________________________________________________________________________________________________
block6c_expand_activation (Acti (None, 7, 7, 1152)   0           block6c_expand_bn[0][0]          
__________________________________________________________________________________________________
block6c_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   28800       block6c_expand_activation[0][0]  
__________________________________________________________________________________________________
block6c_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block6c_dwconv[0][0]             
__________________________________________________________________________________________________
block6c_activation (Activation) (None, 7, 7, 1152)   0           block6c_bn[0][0]                 
__________________________________________________________________________________________________
block6c_se_squeeze (GlobalAvera (None, 1152)         0           block6c_activation[0][0]         
__________________________________________________________________________________________________
block6c_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block6c_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6c_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block6c_se_reshape[0][0]         
__________________________________________________________________________________________________
block6c_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block6c_se_reduce[0][0]          
__________________________________________________________________________________________________
block6c_se_excite (Multiply)    (None, 7, 7, 1152)   0           block6c_activation[0][0]         
                                                                 block6c_se_expand[0][0]          
__________________________________________________________________________________________________
block6c_project_conv (Conv2D)   (None, 7, 7, 192)    221184      block6c_se_excite[0][0]          
__________________________________________________________________________________________________
block6c_project_bn (BatchNormal (None, 7, 7, 192)    768         block6c_project_conv[0][0]       
__________________________________________________________________________________________________
block6c_drop (Dropout)          (None, 7, 7, 192)    0           block6c_project_bn[0][0]         
__________________________________________________________________________________________________
block6c_add (Add)               (None, 7, 7, 192)    0           block6c_drop[0][0]               
                                                                 block6b_add[0][0]                
__________________________________________________________________________________________________
block6d_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6c_add[0][0]                
__________________________________________________________________________________________________
block6d_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block6d_expand_conv[0][0]        
__________________________________________________________________________________________________
block6d_expand_activation (Acti (None, 7, 7, 1152)   0           block6d_expand_bn[0][0]          
__________________________________________________________________________________________________
block6d_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   28800       block6d_expand_activation[0][0]  
__________________________________________________________________________________________________
block6d_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block6d_dwconv[0][0]             
__________________________________________________________________________________________________
block6d_activation (Activation) (None, 7, 7, 1152)   0           block6d_bn[0][0]                 
__________________________________________________________________________________________________
block6d_se_squeeze (GlobalAvera (None, 1152)         0           block6d_activation[0][0]         
__________________________________________________________________________________________________
block6d_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block6d_se_squeeze[0][0]         
__________________________________________________________________________________________________
block6d_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block6d_se_reshape[0][0]         
__________________________________________________________________________________________________
block6d_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block6d_se_reduce[0][0]          
__________________________________________________________________________________________________
block6d_se_excite (Multiply)    (None, 7, 7, 1152)   0           block6d_activation[0][0]         
                                                                 block6d_se_expand[0][0]          
__________________________________________________________________________________________________
block6d_project_conv (Conv2D)   (None, 7, 7, 192)    221184      block6d_se_excite[0][0]          
__________________________________________________________________________________________________
block6d_project_bn (BatchNormal (None, 7, 7, 192)    768         block6d_project_conv[0][0]       
__________________________________________________________________________________________________
block6d_drop (Dropout)          (None, 7, 7, 192)    0           block6d_project_bn[0][0]         
__________________________________________________________________________________________________
block6d_add (Add)               (None, 7, 7, 192)    0           block6d_drop[0][0]               
                                                                 block6c_add[0][0]                
__________________________________________________________________________________________________
block7a_expand_conv (Conv2D)    (None, 7, 7, 1152)   221184      block6d_add[0][0]                
__________________________________________________________________________________________________
block7a_expand_bn (BatchNormali (None, 7, 7, 1152)   4608        block7a_expand_conv[0][0]        
__________________________________________________________________________________________________
block7a_expand_activation (Acti (None, 7, 7, 1152)   0           block7a_expand_bn[0][0]          
__________________________________________________________________________________________________
block7a_dwconv (DepthwiseConv2D (None, 7, 7, 1152)   10368       block7a_expand_activation[0][0]  
__________________________________________________________________________________________________
block7a_bn (BatchNormalization) (None, 7, 7, 1152)   4608        block7a_dwconv[0][0]             
__________________________________________________________________________________________________
block7a_activation (Activation) (None, 7, 7, 1152)   0           block7a_bn[0][0]                 
__________________________________________________________________________________________________
block7a_se_squeeze (GlobalAvera (None, 1152)         0           block7a_activation[0][0]         
__________________________________________________________________________________________________
block7a_se_reshape (Reshape)    (None, 1, 1, 1152)   0           block7a_se_squeeze[0][0]         
__________________________________________________________________________________________________
block7a_se_reduce (Conv2D)      (None, 1, 1, 48)     55344       block7a_se_reshape[0][0]         
__________________________________________________________________________________________________
block7a_se_expand (Conv2D)      (None, 1, 1, 1152)   56448       block7a_se_reduce[0][0]          
__________________________________________________________________________________________________
block7a_se_excite (Multiply)    (None, 7, 7, 1152)   0           block7a_activation[0][0]         
                                                                 block7a_se_expand[0][0]          
__________________________________________________________________________________________________
block7a_project_conv (Conv2D)   (None, 7, 7, 320)    368640      block7a_se_excite[0][0]          
__________________________________________________________________________________________________
block7a_project_bn (BatchNormal (None, 7, 7, 320)    1280        block7a_project_conv[0][0]       
__________________________________________________________________________________________________
top_conv (Conv2D)               (None, 7, 7, 1280)   409600      block7a_project_bn[0][0]         
__________________________________________________________________________________________________
top_bn (BatchNormalization)     (None, 7, 7, 1280)   5120        top_conv[0][0]                   
__________________________________________________________________________________________________
top_activation (Activation)     (None, 7, 7, 1280)   0           top_bn[0][0]                     
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 1280)         0           top_activation[0][0]             
__________________________________________________________________________________________________
top_dropout (Dropout)           (None, 1280)         0           avg_pool[0][0]                   
__________________________________________________________________________________________________
predictions (Dense)             (None, 1000)         1281000     top_dropout[0][0]                
==================================================================================================
Total params: 5,330,571
Trainable params: 5,288,548
Non-trainable params: 42,023

References

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

https://github.com/tensorflow/tensorflow/blob/v2.4.0/tensorflow/python/keras/applications/efficientnet.py#L517-L540

https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet

SENet架构-通道注意力机制

MobileNetV3架构解析与代码复现

9.1 EfficientNet网络详解

9.3 使用Tensorflow2搭建EfficientNet网络