文章目录
DeepDream 简介
DeepDream 是 Google 公司在 2015 年公布的一项十分有趣的技术。在训练好的卷积神经网络中，只需要设定几个参数，就可以通过这项技术生成一张有趣的图像。生成出的图像不仅令人印象深刻，而且还能帮助我们理解卷积神网络背后的运行机制，此篇博客将介绍 DeepDream 的基本原理 ，并使用 TensorFlow 1.x 与 TensorFlow 2.x 实现DeepDream 生成模型，其中在 TensorFlow 1.x 程序基础上对 TensorFlow 2.x 有更加细节的优化
DeepDream 原理
在前面的 Tensorflow 笔记 Ⅶ——cifar10卷积神经网络中
，介绍了如何利用深度卷积网络进行图像识别 。在
卷积神经网络中，输入数据一般是图像信息，可以是常规的 numpy 数据，自定义的数据，或者张量，中间层是若干卷积运算，输出是图像的类别概率。在训练阶段，会使用大量的训练图片计算梯度，网络根据梯度不断地调整和学习最佳的参数。使得整体的梯度呈现下降的态势，并逐步的降低 loss 值，那么卷积神经网络中的卷积层到底学到了什么，毕竟深度学习是一个端到端的过程，卷积神经网络就像一个黑匣子，我们难以直观的看出卷积层都发生了什么，因此不禁有一下疑问：
•卷积层究竟学习到了什么内容
•卷积层的参数代表的意义是什么
•浅层的卷积和深层的卷积学习到的内容有哪些区别
DeepDream 则可以较好的解答上述问题
我们以 ImageNet 作为输入数据为例
设输入网络的图像为 x，网络输出的各个类别的概率为 p (如ImageNet
1000 种分类，因此 t 是一个 1000 维的向量，代表了 1000 个类别中各个类别的概率，以狗类别为例，假设输入的图像数据对应的概率输出值在经过 tf.argmax 或者 np.argmax，其所属类别为 p[520]，换句话说
p[520] 代表了神经网络认为一张图片是花的概率 ，这是图像识别的基本流程，那么我们设定 p[520] 为优化目标，不断地让神经网络去调整输入图像 x 的像素值，让输出 t[520] 尽可能的大，则我们可以得到如下图像
不断调整输入的随机噪音图像 x 像素值
为此，实现 DeepDream 的原理如下
只需要最大化卷积层某一通道的输出(或者某几个通道，某几个层的所有通道)，设输入图像为
，中间某个卷积层的输出是
，其中
的形状应该是
真中
为高度，
为宽度，
则代表“通道数”，原始图像有 R、G、B 三个通道 而在大多数卷积层中，通道数都远远不止 3 个。卷积的一个通道就可以代表一种学习到的“特征”，以某一个通道的平均值(类似于前向传播中求 loss 的均值)作为优化目标，就可以弄清楚这个通道究竟学习到了什么，这就是 DeepDream 的基本原理，简单来说就是与训练的过程相反，训练神经网络，我们使用梯度下降，减小 loss，那么 DeepDream 就是loss增加，梯度上升，并且此原理是依据在2013年“Visualizing and Understanding Convolutional Neural Networks”这篇文章提出了使用梯度上升的方法可视化网络每一层的特征，即用一张噪声图像输入网络，反向更新的时候不更新网络权重，而是更新初始图像的像素值，以这种“训练图像”的方式可视化网络
同一卷积层中不同通道学习到不同的特征
浅层的卷积得到的是特征图 感受野更加关注的是图像细节纹理等特征
深层的卷积得到的是特征图信息 在语义语境方面更加抽象的高层信息
DeepDream TensorFLow 1.x 实现
前情函数
tf.reduce_mean()
tf.reduce_mean() 是 tf.math 库中的函数，两个名称等价
tf.math.reduce_mean(
input_tensor,
axis=None,
keepdims=None,
name=None,
reduction_indices=None,
keep_dims=None
)
input_tensor: 输入值为张量
axis: 求均值的轴向，默认为 None，默认对所有轴向求均值，其轴取值范围为 [-rank(input_tensor), rank(input_tensor))
keepdims: 如果为true，则保留长度为1的缩小尺寸
name: 命名
reduction_indices: 轴向的旧名称(不推荐使用)
keep_dims: keepdims 旧名称，不推荐使用
主要掌握 input_tensor 与 axis 就够了
返回一个张量值
tf.reduce_mean()是一个求均值操作，可以对为指定的具体值执行均值操作
import tensorflow as tf
import numpy as np
x = tf.constant([[1., 1.], [2., 2.], [3., 4.]])
a = tf.reduce_mean(x)
b = tf.reduce_mean(x, 0)
c = tf.reduce_mean(x, 1)
y = tf.placeholder(tf.float32, shape=(3, 3, 4))
sess = tf.Session()
print('a value:', sess.run(a))
print('b value:', sess.run(b))
print('c value:', sess.run(c))
y_mean = tf.reduce_mean(y)
sess.close()
del y
a value: 2.1666667
b value: [2. 2.3333333]
c value: [1. 2. 3.5]
tf.gradients
tf.gradients(
ys,
xs,
grad_ys=None,
name=‘gradients’,
colocate_gradients_with_ops=False,
gate_gradients=False,
aggregation_method=None,
stop_gradients=None,
unconnected_gradients=tf.UnconnectedGradients.NONE
)
函数作用为求梯度
ys，xs: 输入张量值
stop_gradients: 求偏导还是求全导，需要在变量之间具有转换关系
unconnected_gradients :没有联系的参数，这种求导一般为 None ，可用 unconnected_gradients 指定，让求导结果为 0
对c使用stop_gradients参数，c与b就没有联系，因此对b求导为1.0
a = tf.constant(0.)
b = 2 * a
c = 3 * b
g = tf.gradients(a + b + c, [a, b, c], stop_gradients=[c])
tf.Session().run(g)
[3.0, 1.0, 1.0]
a与b与c有联系，b与a没联系，b与c有联系，c与b与a无联系，根据 b = 2 * a，c = 3 * b 对a就是9a，对b就是4b，对c就是1c
a = tf.constant(0.)
b = 2 * a
c = 3 * b
g = tf.gradients(a + b + c, [a, b, c])
tf.Session().run(g)
[9.0, 4.0, 1.0]
stop_gradient 也可以进行单独指定
a = tf.stop_gradient(tf.constant(0.))
b = tf.stop_gradient(2 * a)
g = tf.gradients(a + b, [a, b])
tf.Session().run(g)
[1.0, 1.0]
a = tf.constant(0.)
b = 2 * a
c = 3 * b
g = tf.gradients(a + b + c, [a, b, c])[0]
tf.Session().run(g)
9.0
a = tf.ones([1, 2])
b = tf.ones([3, 1])
try:
g1 = tf.gradients([b], [a], unconnected_gradients='none')
print(tf.Session().run(g1))
except TypeError:
print('TypeError: Fetch argument None has invalid type <class \'NoneType\'>')
finally:
g2 = tf.gradients([b], [a], unconnected_gradients='zero')
tf.Session().run(g2)
print(tf.Session().run(g2))
TypeError: Fetch argument None has invalid type <class 'NoneType'>
[array([[0., 0.]], dtype=float32)]
numpy.clip() 与 numpy 的 std()
numpy.clip(a, a_min, a_max, out=None)
作用将 数据限制在一个范围，小于 a_min 为 a_min，大于 a_max 为 a_max
a_min: 输入的数据，可以是列表，int 等可识别类型
a_max: 输入的数据，可以是列表，int 等可识别类型
std() 用于求标准差
list clip 的每一个对应数据限制在 list 相应数据与 8 之间
a = np.arange(10)
print('a value:\n', a)
print('a clip 4~7:\n', a.clip(4, 7))
print('list clip:\n',a.clip([3, 4, 1, 4, 1, 4, 4, 4, 4, 4], 8))
a = np.array([[1, 3], [4, 5]])
print('a value:\n', a)
print('a std:', a.std())
a value:
[0 1 2 3 4 5 6 7 8 9]
a clip 4~7:
[4 4 4 4 4 5 6 7 7 7]
list clip:
[3 4 2 4 4 5 6 7 8 8]
a value:
[[1 3]
[4 5]]
a std: 1.479019945774904
numpy.roll()
numpy.roll(a, shift, axis=None)
a : 输入的 array
shift : 移动的值，可以是整数，也可以是元组，元组需与轴对应
axis : 操作的轴
返回值为 array
x = np.arange(15)
print('x value:\n', x)
print('np.roll(x, 2):\n', np.roll(x, 2))
print('np.roll(x, -2):\n', np.roll(x, -2))
x value:
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14]
np.roll(x, 2):
[13 14 0 1 2 3 4 5 6 7 8 9 10 11 12]
np.roll(x, -2):
[ 2 3 4 5 6 7 8 9 10 11 12 13 14 0 1]
x2 = np.reshape(x, (3,5))
print('x value:\n', x2)
print('np.roll(x2, 1):\n', np.roll(x2, 1))
print('np.roll(x2, 1, axis=0):\n', np.roll(x2, 1, axis=0))
print('np.roll(x2, 1, axis=1):\n', np.roll(x2, 1, axis=1))
print('np.roll(x2, (1, 1)):\n', np.roll(x2, (1, 1)))
x value:
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]]
np.roll(x2, 1):
[[14 0 1 2 3]
[ 4 5 6 7 8]
[ 9 10 11 12 13]]
np.roll(x2, 1, axis=0):
[[10 11 12 13 14]
[ 0 1 2 3 4]
[ 5 6 7 8 9]]
np.roll(x2, 1, axis=1):
[[ 4 0 1 2 3]
[ 9 5 6 7 8]
[14 10 11 12 13]]
np.roll(x2, (1, 1)):
[[13 14 0 1 2]
[ 3 4 5 6 7]
[ 8 9 10 11 12]]
tf.expand_dims()
作用是增加一个维度，numpy 也具有同样功能的函数
tf.expand_dims(
input,
axis=None,
name=None,
dim=None
)
input: 输入张量
axis: 标量，指定具体拓展的维度，其值范围为 range[-rank(input) - 1, rank(input)]
name: 命名
dim: 与 axis 相同，但不推荐使用
返回与输入同类型张量
站长警告: 不推荐使用 dim，它将在以后的版本中删除，改用axis参数
import tensorflow as tf
import numpy as np
demo = [[2, 3, 4], [1, 5, 5]]
print('demo_shape:', np.shape(demo))
demo_shape: (2, 3)
增加第零维度
demo1 = tf.expand_dims(demo, 0)
print('demo1_shape:', np.shape(demo1))
demo1_shape: (1, 2, 3)
增加第一维度
demo2 = tf.expand_dims(demo, 1)
print('demo2_shape:', np.shape(demo2))
demo2_shape: (2, 1, 3)
增加第二维度
demo3 = tf.expand_dims(demo, 2)
print('demo3_shape:', np.shape(demo3))
demo3_shape: (2, 3, 1)
增加最后一维度(第二维度)
demo4 = tf.expand_dims(demo, -1)
print('demo4_shape:', np.shape(demo4))
demo4_shape: (2, 3, 1)
图保存与加载
图的保存
variable = tf.Variable(1.0, name='my_variable')
with tf.Session() as sess:
tf.train.write_graph(sess.graph_def, './tfmodel', 'test_pb.pb', as_text=False)
图的加载
with tf.Session() as sess:
with tf.gfile.FastGFile('./tfmodel/test_pb.pb', 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sess.graph.as_default()
tf.import_graph_def(graph_def, name='tf.graph')
print(graph_def)
仅展示部分输出
node {
name: "Const"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
...
...
...
versions {
producer: 134
}
创建、重置、获得默认图
g = tf.Graph()
with g.as_default():
c1 = tf.constant(0.0)
print(c1)
print('c1.graph:', c1.graph)
Tensor("Const:0", shape=(), dtype=float32)
c1.graph: <tensorflow.python.framework.ops.Graph object at 0x000002692EFB05F8>
tf.reset_default_graph()
g2 = tf.get_default_graph()
print('g2:', g2)
g2: <tensorflow.python.framework.ops.Graph object at 0x000002692EFB0470>
获取张量 get_tensor_by_name()
通过 .name 操作获取张量的名字，因此我们不必关注 TensorFlow 的默认命名规则，我们可以指定名称，就算不知道也可以根据 .name 操作打印其名称，DeepDream 一般步骤为先获取张量名，再将张量名回填到输入操作中
print('Name:', c1.name)
Name: Const:0
t = g.get_tensor_by_name(name='Const:0')
print(t)
Tensor("Const:0", shape=(), dtype=float32)
指定名称
g_name = tf.Graph()
with g_name.as_default():
c1_name = tf.constant(0.0, name='c1_name')
print(c1_name)
print('c1_name.graph:', c1_name.graph)
print('Name:', c1_name.name)
t_name = g_name.get_tensor_by_name(name='c1_name:0')
print(t_name)
Tensor("c1_name:0", shape=(), dtype=float32)
c1_name.graph: <tensorflow.python.framework.ops.Graph object at 0x00000269300C49B0>
Name: c1_name:0
Tensor("c1_name:0", shape=(), dtype=float32)
获取节点操作 get_operation_by_name
DeepDream 一般步骤为先获取操作名，再将操作名回填到输入操作中
a = tf.constant([[1.0, 2.0]])
b = tf.constant([[1.0],[3.0]])
tensor1 = tf.matmul(a, b, name='example_op')
print(tensor1)
print('Name:', tensor1.name)
Tensor("example_op:0", shape=(1, 1), dtype=float32)
Name: example_op:0
print(tensor1.op.name)
example_op
test_op = g2.get_operation_by_name('example_op')
print(test_op)
name: "example_op"
op: "MatMul"
input: "Const"
input: "Const_1"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "transpose_a"
value {
b: false
}
}
attr {
key: "transpose_b"
value {
b: false
}
}
tf.reset_default_graph()
正式开始
import os
from io import BytesIO
import numpy as np
import PIL.Image
import urllib
import tensorflow as tf
import matplotlib.pyplot as plt
import zipfile
导入模型
graph = tf.Graph()
sess = tf.InteractiveSession(graph=graph)
模型下载
data_url = 'https://storage.googleapis.com/download.tensorflow.org/models/inception5h.zip'
data_file = './data/inception5h.zip'
if not os.path.exists(data_file):
operation = urllib.request.urlretrieve(data_url, data_file)
print('downloading from %s' % data_url)
else:
print('inception5h.zip is exists in the data directory')
inception5h.zip is exists in the data directory
模型解压
可以使用如下的 Python 解压方式，也可以使用 unzip 命令
!unzip -n data/inception5h.zip -d /data
dst_dir = './data'
fz = zipfile.ZipFile(data_file, 'r')
for file in fz.namelist():
fz.extract(file, dst_dir)
print('INFO: ' + file)
INFO: imagenet_comp_graph_label_strings.txt
INFO: tensorflow_inception_graph.pb
INFO: LICENSE
模型导入
model_fn = './data/tensorflow_inception_graph.pb'
with tf.gfile.GFile(model_fn, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
我们需要导入一张图片用于生成 DeepDream 图像，所以需要定义一个占位符，同时需要减去一个均值，因为在训练 inception 模型在定义时减去了均值，由于 inception 的图像格式是三通道加上一个 batch 维度，所以其 shape 为 [batch, height, width, channels]，我们需要拓展维度，t_input 作为输入的占位符
t_input = tf.placeholder(tf.float32, name='input')
imagenet_mean = 117.0
t_preprocessrd = tf.expand_dims(t_input - imagenet_mean, 0)
tf.import_graph_def(graph_def, {'input':t_preprocessrd})
寻觅卷积层
layers = [op.name for op in graph.get_operations() if op.type == 'Conv2D']
print('Number of layers:', len(layers))
Number of layers: 59
print(layers)
['import/conv2d0_pre_relu/conv', 'import/conv2d1_pre_relu/conv', 'import/conv2d2_pre_relu/conv', 'import/mixed3a_1x1_pre_relu/conv', 'import/mixed3a_3x3_bottleneck_pre_relu/conv', 'import/mixed3a_3x3_pre_relu/conv', 'import/mixed3a_5x5_bottleneck_pre_relu/conv', 'import/mixed3a_5x5_pre_relu/conv', 'import/mixed3a_pool_reduce_pre_relu/conv', 'import/mixed3b_1x1_pre_relu/conv', 'import/mixed3b_3x3_bottleneck_pre_relu/conv', 'import/mixed3b_3x3_pre_relu/conv', 'import/mixed3b_5x5_bottleneck_pre_relu/conv', 'import/mixed3b_5x5_pre_relu/conv', 'import/mixed3b_pool_reduce_pre_relu/conv', 'import/mixed4a_1x1_pre_relu/conv', 'import/mixed4a_3x3_bottleneck_pre_relu/conv', 'import/mixed4a_3x3_pre_relu/conv', 'import/mixed4a_5x5_bottleneck_pre_relu/conv', 'import/mixed4a_5x5_pre_relu/conv', 'import/mixed4a_pool_reduce_pre_relu/conv', 'import/mixed4b_1x1_pre_relu/conv', 'import/mixed4b_3x3_bottleneck_pre_relu/conv', 'import/mixed4b_3x3_pre_relu/conv', 'import/mixed4b_5x5_bottleneck_pre_relu/conv', 'import/mixed4b_5x5_pre_relu/conv', 'import/mixed4b_pool_reduce_pre_relu/conv', 'import/mixed4c_1x1_pre_relu/conv', 'import/mixed4c_3x3_bottleneck_pre_relu/conv', 'import/mixed4c_3x3_pre_relu/conv', 'import/mixed4c_5x5_bottleneck_pre_relu/conv', 'import/mixed4c_5x5_pre_relu/conv', 'import/mixed4c_pool_reduce_pre_relu/conv', 'import/mixed4d_1x1_pre_relu/conv', 'import/mixed4d_3x3_bottleneck_pre_relu/conv', 'import/mixed4d_3x3_pre_relu/conv', 'import/mixed4d_5x5_bottleneck_pre_relu/conv', 'import/mixed4d_5x5_pre_relu/conv', 'import/mixed4d_pool_reduce_pre_relu/conv', 'import/mixed4e_1x1_pre_relu/conv', 'import/mixed4e_3x3_bottleneck_pre_relu/conv', 'import/mixed4e_3x3_pre_relu/conv', 'import/mixed4e_5x5_bottleneck_pre_relu/conv', 'import/mixed4e_5x5_pre_relu/conv', 'import/mixed4e_pool_reduce_pre_relu/conv', 'import/mixed5a_1x1_pre_relu/conv', 'import/mixed5a_3x3_bottleneck_pre_relu/conv', 'import/mixed5a_3x3_pre_relu/conv', 'import/mixed5a_5x5_bottleneck_pre_relu/conv', 'import/mixed5a_5x5_pre_relu/conv', 'import/mixed5a_pool_reduce_pre_relu/conv', 'import/mixed5b_1x1_pre_relu/conv', 'import/mixed5b_3x3_bottleneck_pre_relu/conv', 'import/mixed5b_3x3_pre_relu/conv', 'import/mixed5b_5x5_bottleneck_pre_relu/conv', 'import/mixed5b_5x5_pre_relu/conv', 'import/mixed5b_pool_reduce_pre_relu/conv', 'import/head0_bottleneck_pre_relu/conv', 'import/head1_bottleneck_pre_relu/conv']
打印特定卷积层形状，由于输入未定，但卷积神经网络结构中的通道数是固定的，所以出现三个 ？
name1 = 'conv2d0_pre_relu'
print('shape of %s:%s' % (name1, str(graph.get_tensor_by_name('import/' + name1 + ':0').get_shape())))
name2 = 'conv2d1_pre_relu'
print('shape of %s:%s' % (name1, str(graph.get_tensor_by_name('import/' + name2 + ':0').get_shape())))
# mixed4d_3x3_bottleneck_pre_relu
name3 = 'mixed3a_pool_reduce_pre_relu'
print('shape of %s:%s' % (name3, str(graph.get_tensor_by_name('import/' + name3 + ':0').get_shape())))
name4 = 'mixed4d_3x3_bottleneck_pre_relu'
print('shape of %s:%s' % (name4, str(graph.get_tensor_by_name('import/' + name4 + ':0').get_shape())))
shape of conv2d0_pre_relu:(?, ?, ?, 64)
shape of conv2d0_pre_relu:(?, ?, ?, 64)
shape of mixed3a_pool_reduce_pre_relu:(?, ?, ?, 32)
shape of mixed4d_3x3_bottleneck_pre_relu:(?, ?, ?, 144)
定义需求函数
渲染函数
算法思路为先求梯度，再规范化，再乘上一个步长因子加到图像上，步长因子相当于学习率，其目的就是增大某一通道的值，step 可以看作模型训练中的学习率，score 可以看作是损失值，我们向梯度增加的方向与损失值上升的方向运算来最大化通道值
def render_naive(t_obj, img0, iter_n=20, step=1.0):
t_score = tf.reduce_mean(t_obj)
t_grad = tf.gradients(t_score, t_input)[0]
img = img0.copy()
for i in range(iter_n):
g, score = sess.run([t_grad, t_score], {t_input:img})
g /= g.std() + 1e-8
img += g * step
print('iter:%2d' % (i + 1), 'score(mean)=%11f' % score)
img = img.astype('uint8')
savearray(img, './data/naive_deepdream.jpg')
return img
改进版渲染函数，利用图像金字塔将图像缩小 octaves 保存每次缩小图像并放大后与原图像的差值，保存差值而不直接放大的目的是为了还原原始图像
def render_deepdream(dist, t_obj, img0, iter_n=10, step=1.5, octave_n=4, octave_scale=1.4):
t_score = tf.reduce_mean(t_obj)
t_grad = tf.gradients(t_score, t_input)[0]
img = img0.copy()
octaves = []
for i in range(octave_n - 1):
hw = img.shape[:2]
lo = resize(img, np.int32(np.float32(hw) / octave_scale))
hi = img - resize(lo, hw)
img = lo
octaves.append(hi)
for octave in range(octave_n):
if octave > 0:
hi = octaves[-octave]
img = resize(img, hi.shape[:2]) + hi
for i in range(iter_n):
g = calc_grad_tiled(img, t_grad)
img += g * (step / (np.abs(g).mean() + 1e-7))
img = img.clip(0, 255)
savearray(img, dist)
return img
梯度计算函数，将输入的图像进行梯度运算，输入的图像为原始图像的金字塔分层后最小的图像，利用图像矩阵平移可以消除边缘效应，运算完成再平移回来，每次计算的大小是固定的 512x512，其目的是防止内存耗尽，利用遍历的方式计算整张图
def calc_grad_tiled(img, t_grad, tile_size=512):
sz = tile_size
h, w = img.shape[:2]
sx, sy = np.random.randint(sz, size=2)
img_shift = np.roll(np.roll(img, sx, 1), sy, 0)
grad = np.zeros_like(img)
for y in range(0, max(h - sz // 2, sz), sz):
for x in range(0, max(w - sz // 2, sz), sz):
sub = img_shift[y:y + sz, x:x + sz]
g = sess.run(t_grad, {t_input:sub})
grad[y:y + sz, x: x + sz] = g
return np.roll(np.roll(grad, -sx, 1), -sy, 0)
图像标准化
def normalize_image(img):
img = 255 * (img + 1.0) / 2.0
return img.astype('uint8')
保存图像
def savearray(img_array, img_name):
PIL.Image.fromarray(img_array.astype('uint8')).save(img_name)
print('img_saved:%s' % img_name)
resize 图像
(img - min) / (max - min) * 255 先对图像进行归一化，在乘上255，使图像的像素值在 0 到 255
def resize(img, hw):
min = img.min()
max = img.max()
img = (img - min) / (max - min) * 255
img = np.float32(np.array(PIL.Image.fromarray(img.astype('uint8')).resize(hw)).swapaxes(0, 1))
img = img / 255 * (max - min) + min
return img
ratio 倍数放大
通过单通道特征生成 DeepDream
随机噪音生成 DeepDream 图像
name = 'mixed4d_3x3_bottleneck_pre_relu'
channel = 30
layer_output = graph.get_tensor_by_name("import/%s:0" % name)
img_noise = np.random.uniform(size=(224, 224, 3)) + 100.0
img = render_naive(layer_output[:,:,:,channel], img_noise, iter_n=20)
fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = plt.subplot(1, 2, 1)
ax1.imshow(normalize_image(img_noise))
ax1.set_title('noise')
ax1.set_xticks([])
ax1.set_yticks([])
ax2 = plt.subplot(1, 2, 2)
ax2.imshow(normalize_image(img))
ax2.set_title('DeepDream')
ax2.set_xticks([])
ax2.set_yticks([])
plt.show()
iter: 1 score(mean)= -17.953234
iter: 2 score(mean)= -37.590721
iter: 3 score(mean)= -24.524006
iter: 4 score(mean)= 10.407385
iter: 5 score(mean)= 38.392395
iter: 6 score(mean)= 79.842209
iter: 7 score(mean)= 110.087914
iter: 8 score(mean)= 149.972092
iter: 9 score(mean)= 174.243271
iter:10 score(mean)= 216.072006
iter:11 score(mean)= 236.871964
iter:12 score(mean)= 279.913971
iter:13 score(mean)= 302.553741
iter:14 score(mean)= 339.833832
iter:15 score(mean)= 360.929840
iter:16 score(mean)= 401.176300
iter:17 score(mean)= 413.377625
iter:18 score(mean)= 445.812988
iter:19 score(mean)= 468.058167
iter:20 score(mean)= 489.575775
img_saved:./data/naive_deepdream.jpg
输入图像生成 DeepDream
name = 'mixed4c'
layer_output = graph.get_tensor_by_name("import/%s:0" % name)
img0 = PIL.Image.open('./data/mountain1.jpg')
img0 = np.float32(img0)
dis = './data/mountain1_deepdream.jpg'
img = render_deepdream(dis, tf.square(layer_output), img0)
img_saved:./data/mountain1_deepdream.jpg
fig = plt.gcf()
fig.set_size_inches(20, 10)
ax1 = plt.subplot(1, 2, 1)
ax1.imshow(img0.astype('uint8'))
ax1.set_title('raw')
ax1.set_xticks([])
ax1.set_yticks([])
ax2 = plt.subplot(1, 2, 2)
ax2.imshow(img.astype('uint8'))
ax2.set_title('DeepDream')
ax2.set_xticks([])
ax2.set_yticks([])
plt.show()
DeepDream TensorFLow 2.x 实现
特别说明
TensorFLow2.x 经过两次优化，TensorFlow2.x 大部分都没有使用 numpy，都使用了 TensorFlow 自定义的函数，但其实现方式与 numpy 相同，就不做过多的函数解析，观察函数名称就能看出与之对应功能的 numpy 函数
正式开始
导入必要的包
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import IPython.display as display
import PIL.Image
from tensorflow.keras.preprocessing import image
print('GPU is', 'available' if tf.test.is_gpu_available() else 'Not available')
GPU is available
tf.__version__
'2.0.0'
定义公共函数
def normalize_image(img):
img = 255 * (img + 1.0) / 2.0
return tf.cast(img, tf.uint8)
def show_image(img):
display.display(PIL.Image.fromarray(np.array(img)))
def save_image(img, file_name):
PIL.Image.fromarray(np.array(img)).save(file_name)
def read_image(file_name, max_dim=None):
img = PIL.Image.open(file_name)
if max_dim:
img.thumbnail((max_dim, max_dim))
return np.array(img)
tf.keras.applications包含了多种预训练的经典深度模型
由于不同大小图片的特征数量不同，因此我们利用 include_top=False 去掉顶层以此可以根据自己的需要决定输入的 shape，weights=‘imagenet’ 表明 imagenet 样本训练出的模型的权重
公共 base_model
base_model = tf.keras.applications.InceptionV3(include_top=False, weights='imagenet')
base_model.summary()
Model: "inception_v3"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, None, None, 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, None, None, 3 864 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, None, None, 3 96 conv2d[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, None, None, 3 0 batch_normalization[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, None, None, 3 9216 activation[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, None, None, 3 96 conv2d_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, None, None, 3 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, None, None, 6 18432 activation_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, None, None, 6 192 conv2d_2[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, None, None, 6 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, None, None, 6 0 activation_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, None, None, 8 5120 max_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, None, None, 8 240 conv2d_3[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, None, None, 8 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, None, None, 1 138240 activation_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, None, None, 1 576 conv2d_4[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, None, None, 1 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D) (None, None, None, 1 0 activation_4[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, None, None, 6 12288 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, None, 6 192 conv2d_8[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, None, None, 6 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, None, None, 4 9216 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, None, None, 9 55296 activation_8[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, None, 4 144 conv2d_6[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, None, 9 288 conv2d_9[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, None, None, 4 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, None, None, 9 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, None, None, 1 0 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, None, None, 6 12288 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, None, None, 6 76800 activation_6[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, None, None, 9 82944 activation_9[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, None, None, 3 6144 average_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, None, 6 192 conv2d_5[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, None, 6 192 conv2d_7[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, None, 9 288 conv2d_10[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, None, 3 96 conv2d_11[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, None, None, 6 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, None, None, 6 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, None, None, 9 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, None, None, 3 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
mixed0 (Concatenate) (None, None, None, 2 0 activation_5[0][0]
activation_7[0][0]
activation_10[0][0]
activation_11[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, None, None, 6 16384 mixed0[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, None, None, 6 192 conv2d_15[0][0]
__________________________________________________________________________________________________
activation_15 (Activation) (None, None, None, 6 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, None, None, 4 12288 mixed0[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, None, None, 9 55296 activation_15[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, None, 4 144 conv2d_13[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, None, None, 9 288 conv2d_16[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, None, None, 4 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, None, None, 9 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, None, None, 2 0 mixed0[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, None, None, 6 16384 mixed0[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, None, None, 6 76800 activation_13[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, None, None, 9 82944 activation_16[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, None, None, 6 16384 average_pooling2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, None, 6 192 conv2d_12[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, None, 6 192 conv2d_14[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, None, None, 9 288 conv2d_17[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, None, None, 6 192 conv2d_18[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, None, None, 6 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, None, None, 6 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, None, None, 9 0 batch_normalization_17[0][0]
__________________________________________________________________________________________________
activation_18 (Activation) (None, None, None, 6 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
mixed1 (Concatenate) (None, None, None, 2 0 activation_12[0][0]
activation_14[0][0]
activation_17[0][0]
activation_18[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, None, None, 6 18432 mixed1[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, None, None, 6 192 conv2d_22[0][0]
__________________________________________________________________________________________________
activation_22 (Activation) (None, None, None, 6 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, None, None, 4 13824 mixed1[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, None, None, 9 55296 activation_22[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, None, None, 4 144 conv2d_20[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, None, None, 9 288 conv2d_23[0][0]
__________________________________________________________________________________________________
activation_20 (Activation) (None, None, None, 4 0 batch_normalization_20[0][0]
__________________________________________________________________________________________________
activation_23 (Activation) (None, None, None, 9 0 batch_normalization_23[0][0]
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, None, None, 2 0 mixed1[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, None, None, 6 18432 mixed1[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, None, None, 6 76800 activation_20[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, None, None, 9 82944 activation_23[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, None, None, 6 18432 average_pooling2d_2[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, None, None, 6 192 conv2d_19[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, None, None, 6 192 conv2d_21[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, None, None, 9 288 conv2d_24[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, None, None, 6 192 conv2d_25[0][0]
__________________________________________________________________________________________________
activation_19 (Activation) (None, None, None, 6 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
activation_21 (Activation) (None, None, None, 6 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
activation_24 (Activation) (None, None, None, 9 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
activation_25 (Activation) (None, None, None, 6 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
mixed2 (Concatenate) (None, None, None, 2 0 activation_19[0][0]
activation_21[0][0]
activation_24[0][0]
activation_25[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, None, None, 6 18432 mixed2[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, None, None, 6 192 conv2d_27[0][0]
__________________________________________________________________________________________________
activation_27 (Activation) (None, None, None, 6 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, None, None, 9 55296 activation_27[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, None, None, 9 288 conv2d_28[0][0]
__________________________________________________________________________________________________
activation_28 (Activation) (None, None, None, 9 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, None, None, 3 995328 mixed2[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, None, None, 9 82944 activation_28[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, None, None, 3 1152 conv2d_26[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, None, None, 9 288 conv2d_29[0][0]
__________________________________________________________________________________________________
activation_26 (Activation) (None, None, None, 3 0 batch_normalization_26[0][0]
__________________________________________________________________________________________________
activation_29 (Activation) (None, None, None, 9 0 batch_normalization_29[0][0]
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D) (None, None, None, 2 0 mixed2[0][0]
__________________________________________________________________________________________________
mixed3 (Concatenate) (None, None, None, 7 0 activation_26[0][0]
activation_29[0][0]
max_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, None, None, 1 98304 mixed3[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, None, None, 1 384 conv2d_34[0][0]
__________________________________________________________________________________________________
activation_34 (Activation) (None, None, None, 1 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, None, None, 1 114688 activation_34[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, None, None, 1 384 conv2d_35[0][0]
__________________________________________________________________________________________________
activation_35 (Activation) (None, None, None, 1 0 batch_normalization_35[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, None, None, 1 98304 mixed3[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, None, None, 1 114688 activation_35[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, None, None, 1 384 conv2d_31[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, None, None, 1 384 conv2d_36[0][0]
__________________________________________________________________________________________________
activation_31 (Activation) (None, None, None, 1 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
activation_36 (Activation) (None, None, None, 1 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, None, None, 1 114688 activation_31[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, None, None, 1 114688 activation_36[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, None, None, 1 384 conv2d_32[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, None, None, 1 384 conv2d_37[0][0]
__________________________________________________________________________________________________
activation_32 (Activation) (None, None, None, 1 0 batch_normalization_32[0][0]
__________________________________________________________________________________________________
activation_37 (Activation) (None, None, None, 1 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
average_pooling2d_3 (AveragePoo (None, None, None, 7 0 mixed3[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, None, None, 1 147456 mixed3[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, None, None, 1 172032 activation_32[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, None, None, 1 172032 activation_37[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, None, None, 1 147456 average_pooling2d_3[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, None, None, 1 576 conv2d_30[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, None, None, 1 576 conv2d_33[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, None, None, 1 576 conv2d_38[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, None, None, 1 576 conv2d_39[0][0]
__________________________________________________________________________________________________
activation_30 (Activation) (None, None, None, 1 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
activation_33 (Activation) (None, None, None, 1 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
activation_38 (Activation) (None, None, None, 1 0 batch_normalization_38[0][0]
__________________________________________________________________________________________________
activation_39 (Activation) (None, None, None, 1 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
mixed4 (Concatenate) (None, None, None, 7 0 activation_30[0][0]
activation_33[0][0]
activation_38[0][0]
activation_39[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, None, None, 1 122880 mixed4[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, None, None, 1 480 conv2d_44[0][0]
__________________________________________________________________________________________________
activation_44 (Activation) (None, None, None, 1 0 batch_normalization_44[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, None, None, 1 179200 activation_44[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, None, None, 1 480 conv2d_45[0][0]
__________________________________________________________________________________________________
activation_45 (Activation) (None, None, None, 1 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, None, None, 1 122880 mixed4[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, None, None, 1 179200 activation_45[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, None, None, 1 480 conv2d_41[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, None, None, 1 480 conv2d_46[0][0]
__________________________________________________________________________________________________
activation_41 (Activation) (None, None, None, 1 0 batch_normalization_41[0][0]
__________________________________________________________________________________________________
activation_46 (Activation) (None, None, None, 1 0 batch_normalization_46[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, None, None, 1 179200 activation_41[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, None, None, 1 179200 activation_46[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, None, None, 1 480 conv2d_42[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, None, None, 1 480 conv2d_47[0][0]
__________________________________________________________________________________________________
activation_42 (Activation) (None, None, None, 1 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
activation_47 (Activation) (None, None, None, 1 0 batch_normalization_47[0][0]
__________________________________________________________________________________________________
average_pooling2d_4 (AveragePoo (None, None, None, 7 0 mixed4[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, None, None, 1 147456 mixed4[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, None, None, 1 215040 activation_42[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, None, None, 1 215040 activation_47[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, None, None, 1 147456 average_pooling2d_4[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, None, None, 1 576 conv2d_40[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, None, None, 1 576 conv2d_43[0][0]
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, None, None, 1 576 conv2d_48[0][0]
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, None, None, 1 576 conv2d_49[0][0]
__________________________________________________________________________________________________
activation_40 (Activation) (None, None, None, 1 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
activation_43 (Activation) (None, None, None, 1 0 batch_normalization_43[0][0]
__________________________________________________________________________________________________
activation_48 (Activation) (None, None, None, 1 0 batch_normalization_48[0][0]
__________________________________________________________________________________________________
activation_49 (Activation) (None, None, None, 1 0 batch_normalization_49[0][0]
__________________________________________________________________________________________________
mixed5 (Concatenate) (None, None, None, 7 0 activation_40[0][0]
activation_43[0][0]
activation_48[0][0]
activation_49[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, None, None, 1 122880 mixed5[0][0]
__________________________________________________________________________________________________
batch_normalization_54 (BatchNo (None, None, None, 1 480 conv2d_54[0][0]
__________________________________________________________________________________________________
activation_54 (Activation) (None, None, None, 1 0 batch_normalization_54[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, None, None, 1 179200 activation_54[0][0]
__________________________________________________________________________________________________
batch_normalization_55 (BatchNo (None, None, None, 1 480 conv2d_55[0][0]
__________________________________________________________________________________________________
activation_55 (Activation) (None, None, None, 1 0 batch_normalization_55[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, None, None, 1 122880 mixed5[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, None, None, 1 179200 activation_55[0][0]
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, None, None, 1 480 conv2d_51[0][0]
__________________________________________________________________________________________________
batch_normalization_56 (BatchNo (None, None, None, 1 480 conv2d_56[0][0]
__________________________________________________________________________________________________
activation_51 (Activation) (None, None, None, 1 0 batch_normalization_51[0][0]
__________________________________________________________________________________________________
activation_56 (Activation) (None, None, None, 1 0 batch_normalization_56[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, None, None, 1 179200 activation_51[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (None, None, None, 1 179200 activation_56[0][0]
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, None, None, 1 480 conv2d_52[0][0]
__________________________________________________________________________________________________
batch_normalization_57 (BatchNo (None, None, None, 1 480 conv2d_57[0][0]
__________________________________________________________________________________________________
activation_52 (Activation) (None, None, None, 1 0 batch_normalization_52[0][0]
__________________________________________________________________________________________________
activation_57 (Activation) (None, None, None, 1 0 batch_normalization_57[0][0]
__________________________________________________________________________________________________
average_pooling2d_5 (AveragePoo (None, None, None, 7 0 mixed5[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, None, None, 1 147456 mixed5[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, None, None, 1 215040 activation_52[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (None, None, None, 1 215040 activation_57[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (None, None, None, 1 147456 average_pooling2d_5[0][0]
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, None, None, 1 576 conv2d_50[0][0]
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, None, None, 1 576 conv2d_53[0][0]
__________________________________________________________________________________________________
batch_normalization_58 (BatchNo (None, None, None, 1 576 conv2d_58[0][0]
__________________________________________________________________________________________________
batch_normalization_59 (BatchNo (None, None, None, 1 576 conv2d_59[0][0]
__________________________________________________________________________________________________
activation_50 (Activation) (None, None, None, 1 0 batch_normalization_50[0][0]
__________________________________________________________________________________________________
activation_53 (Activation) (None, None, None, 1 0 batch_normalization_53[0][0]
__________________________________________________________________________________________________
activation_58 (Activation) (None, None, None, 1 0 batch_normalization_58[0][0]
__________________________________________________________________________________________________
activation_59 (Activation) (None, None, None, 1 0 batch_normalization_59[0][0]
__________________________________________________________________________________________________
mixed6 (Concatenate) (None, None, None, 7 0 activation_50[0][0]
activation_53[0][0]
activation_58[0][0]
activation_59[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (None, None, None, 1 147456 mixed6[0][0]
__________________________________________________________________________________________________
batch_normalization_64 (BatchNo (None, None, None, 1 576 conv2d_64[0][0]
__________________________________________________________________________________________________
activation_64 (Activation) (None, None, None, 1 0 batch_normalization_64[0][0]
__________________________________________________________________________________________________
conv2d_65 (Conv2D) (None, None, None, 1 258048 activation_64[0][0]
__________________________________________________________________________________________________
batch_normalization_65 (BatchNo (None, None, None, 1 576 conv2d_65[0][0]
__________________________________________________________________________________________________
activation_65 (Activation) (None, None, None, 1 0 batch_normalization_65[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (None, None, None, 1 147456 mixed6[0][0]
__________________________________________________________________________________________________
conv2d_66 (Conv2D) (None, None, None, 1 258048 activation_65[0][0]
__________________________________________________________________________________________________
batch_normalization_61 (BatchNo (None, None, None, 1 576 conv2d_61[0][0]
__________________________________________________________________________________________________
batch_normalization_66 (BatchNo (None, None, None, 1 576 conv2d_66[0][0]
__________________________________________________________________________________________________
activation_61 (Activation) (None, None, None, 1 0 batch_normalization_61[0][0]
__________________________________________________________________________________________________
activation_66 (Activation) (None, None, None, 1 0 batch_normalization_66[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (None, None, None, 1 258048 activation_61[0][0]
__________________________________________________________________________________________________
conv2d_67 (Conv2D) (None, None, None, 1 258048 activation_66[0][0]
__________________________________________________________________________________________________
batch_normalization_62 (BatchNo (None, None, None, 1 576 conv2d_62[0][0]
__________________________________________________________________________________________________
batch_normalization_67 (BatchNo (None, None, None, 1 576 conv2d_67[0][0]
__________________________________________________________________________________________________
activation_62 (Activation) (None, None, None, 1 0 batch_normalization_62[0][0]
__________________________________________________________________________________________________
activation_67 (Activation) (None, None, None, 1 0 batch_normalization_67[0][0]
__________________________________________________________________________________________________
average_pooling2d_6 (AveragePoo (None, None, None, 7 0 mixed6[0][0]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (None, None, None, 1 147456 mixed6[0][0]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (None, None, None, 1 258048 activation_62[0][0]
__________________________________________________________________________________________________
conv2d_68 (Conv2D) (None, None, None, 1 258048 activation_67[0][0]
__________________________________________________________________________________________________
conv2d_69 (Conv2D) (None, None, None, 1 147456 average_pooling2d_6[0][0]
__________________________________________________________________________________________________
batch_normalization_60 (BatchNo (None, None, None, 1 576 conv2d_60[0][0]
__________________________________________________________________________________________________
batch_normalization_63 (BatchNo (None, None, None, 1 576 conv2d_63[0][0]
__________________________________________________________________________________________________
batch_normalization_68 (BatchNo (None, None, None, 1 576 conv2d_68[0][0]
__________________________________________________________________________________________________
batch_normalization_69 (BatchNo (None, None, None, 1 576 conv2d_69[0][0]
__________________________________________________________________________________________________
activation_60 (Activation) (None, None, None, 1 0 batch_normalization_60[0][0]
__________________________________________________________________________________________________
activation_63 (Activation) (None, None, None, 1 0 batch_normalization_63[0][0]
__________________________________________________________________________________________________
activation_68 (Activation) (None, None, None, 1 0 batch_normalization_68[0][0]
__________________________________________________________________________________________________
activation_69 (Activation) (None, None, None, 1 0 batch_normalization_69[0][0]
__________________________________________________________________________________________________
mixed7 (Concatenate) (None, None, None, 7 0 activation_60[0][0]
activation_63[0][0]
activation_68[0][0]
activation_69[0][0]
__________________________________________________________________________________________________
conv2d_72 (Conv2D) (None, None, None, 1 147456 mixed7[0][0]
__________________________________________________________________________________________________
batch_normalization_72 (BatchNo (None, None, None, 1 576 conv2d_72[0][0]
__________________________________________________________________________________________________
activation_72 (Activation) (None, None, None, 1 0 batch_normalization_72[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D) (None, None, None, 1 258048 activation_72[0][0]
__________________________________________________________________________________________________
batch_normalization_73 (BatchNo (None, None, None, 1 576 conv2d_73[0][0]
__________________________________________________________________________________________________
activation_73 (Activation) (None, None, None, 1 0 batch_normalization_73[0][0]
__________________________________________________________________________________________________
conv2d_70 (Conv2D) (None, None, None, 1 147456 mixed7[0][0]
__________________________________________________________________________________________________
conv2d_74 (Conv2D) (None, None, None, 1 258048 activation_73[0][0]
__________________________________________________________________________________________________
batch_normalization_70 (BatchNo (None, None, None, 1 576 conv2d_70[0][0]
__________________________________________________________________________________________________
batch_normalization_74 (BatchNo (None, None, None, 1 576 conv2d_74[0][0]
__________________________________________________________________________________________________
activation_70 (Activation) (None, None, None, 1 0 batch_normalization_70[0][0]
__________________________________________________________________________________________________
activation_74 (Activation) (None, None, None, 1 0 batch_normalization_74[0][0]
__________________________________________________________________________________________________
conv2d_71 (Conv2D) (None, None, None, 3 552960 activation_70[0][0]
__________________________________________________________________________________________________
conv2d_75 (Conv2D) (None, None, None, 1 331776 activation_74[0][0]
__________________________________________________________________________________________________
batch_normalization_71 (BatchNo (None, None, None, 3 960 conv2d_71[0][0]
__________________________________________________________________________________________________
batch_normalization_75 (BatchNo (None, None, None, 1 576 conv2d_75[0][0]
__________________________________________________________________________________________________
activation_71 (Activation) (None, None, None, 3 0 batch_normalization_71[0][0]
__________________________________________________________________________________________________
activation_75 (Activation) (None, None, None, 1 0 batch_normalization_75[0][0]
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D) (None, None, None, 7 0 mixed7[0][0]
__________________________________________________________________________________________________
mixed8 (Concatenate) (None, None, None, 1 0 activation_71[0][0]
activation_75[0][0]
max_pooling2d_3[0][0]
__________________________________________________________________________________________________
conv2d_80 (Conv2D) (None, None, None, 4 573440 mixed8[0][0]
__________________________________________________________________________________________________
batch_normalization_80 (BatchNo (None, None, None, 4 1344 conv2d_80[0][0]
__________________________________________________________________________________________________
activation_80 (Activation) (None, None, None, 4 0 batch_normalization_80[0][0]
__________________________________________________________________________________________________
conv2d_77 (Conv2D) (None, None, None, 3 491520 mixed8[0][0]
__________________________________________________________________________________________________
conv2d_81 (Conv2D) (None, None, None, 3 1548288 activation_80[0][0]
__________________________________________________________________________________________________
batch_normalization_77 (BatchNo (None, None, None, 3 1152 conv2d_77[0][0]
__________________________________________________________________________________________________
batch_normalization_81 (BatchNo (None, None, None, 3 1152 conv2d_81[0][0]
__________________________________________________________________________________________________
activation_77 (Activation) (None, None, None, 3 0 batch_normalization_77[0][0]
__________________________________________________________________________________________________
activation_81 (Activation) (None, None, None, 3 0 batch_normalization_81[0][0]
__________________________________________________________________________________________________
conv2d_78 (Conv2D) (None, None, None, 3 442368 activation_77[0][0]
__________________________________________________________________________________________________
conv2d_79 (Conv2D) (None, None, None, 3 442368 activation_77[0][0]
__________________________________________________________________________________________________
conv2d_82 (Conv2D) (None, None, None, 3 442368 activation_81[0][0]
__________________________________________________________________________________________________
conv2d_83 (Conv2D) (None, None, None, 3 442368 activation_81[0][0]
__________________________________________________________________________________________________
average_pooling2d_7 (AveragePoo (None, None, None, 1 0 mixed8[0][0]
__________________________________________________________________________________________________
conv2d_76 (Conv2D) (None, None, None, 3 409600 mixed8[0][0]
__________________________________________________________________________________________________
batch_normalization_78 (BatchNo (None, None, None, 3 1152 conv2d_78[0][0]
__________________________________________________________________________________________________
batch_normalization_79 (BatchNo (None, None, None, 3 1152 conv2d_79[0][0]
__________________________________________________________________________________________________
batch_normalization_82 (BatchNo (None, None, None, 3 1152 conv2d_82[0][0]
__________________________________________________________________________________________________
batch_normalization_83 (BatchNo (None, None, None, 3 1152 conv2d_83[0][0]
__________________________________________________________________________________________________
conv2d_84 (Conv2D) (None, None, None, 1 245760 average_pooling2d_7[0][0]
__________________________________________________________________________________________________
batch_normalization_76 (BatchNo (None, None, None, 3 960 conv2d_76[0][0]
__________________________________________________________________________________________________
activation_78 (Activation) (None, None, None, 3 0 batch_normalization_78[0][0]
__________________________________________________________________________________________________
activation_79 (Activation) (None, None, None, 3 0 batch_normalization_79[0][0]
__________________________________________________________________________________________________
activation_82 (Activation) (None, None, None, 3 0 batch_normalization_82[0][0]
__________________________________________________________________________________________________
activation_83 (Activation) (None, None, None, 3 0 batch_normalization_83[0][0]
__________________________________________________________________________________________________
batch_normalization_84 (BatchNo (None, None, None, 1 576 conv2d_84[0][0]
__________________________________________________________________________________________________
activation_76 (Activation) (None, None, None, 3 0 batch_normalization_76[0][0]
__________________________________________________________________________________________________
mixed9_0 (Concatenate) (None, None, None, 7 0 activation_78[0][0]
activation_79[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, None, None, 7 0 activation_82[0][0]
activation_83[0][0]
__________________________________________________________________________________________________
activation_84 (Activation) (None, None, None, 1 0 batch_normalization_84[0][0]
__________________________________________________________________________________________________
mixed9 (Concatenate) (None, None, None, 2 0 activation_76[0][0]
mixed9_0[0][0]
concatenate[0][0]
activation_84[0][0]
__________________________________________________________________________________________________
conv2d_89 (Conv2D) (None, None, None, 4 917504 mixed9[0][0]
__________________________________________________________________________________________________
batch_normalization_89 (BatchNo (None, None, None, 4 1344 conv2d_89[0][0]
__________________________________________________________________________________________________
activation_89 (Activation) (None, None, None, 4 0 batch_normalization_89[0][0]
__________________________________________________________________________________________________
conv2d_86 (Conv2D) (None, None, None, 3 786432 mixed9[0][0]
__________________________________________________________________________________________________
conv2d_90 (Conv2D) (None, None, None, 3 1548288 activation_89[0][0]
__________________________________________________________________________________________________
batch_normalization_86 (BatchNo (None, None, None, 3 1152 conv2d_86[0][0]
__________________________________________________________________________________________________
batch_normalization_90 (BatchNo (None, None, None, 3 1152 conv2d_90[0][0]
__________________________________________________________________________________________________
activation_86 (Activation) (None, None, None, 3 0 batch_normalization_86[0][0]
__________________________________________________________________________________________________
activation_90 (Activation) (None, None, None, 3 0 batch_normalization_90[0][0]
__________________________________________________________________________________________________
conv2d_87 (Conv2D) (None, None, None, 3 442368 activation_86[0][0]
__________________________________________________________________________________________________
conv2d_88 (Conv2D) (None, None, None, 3 442368 activation_86[0][0]
__________________________________________________________________________________________________
conv2d_91 (Conv2D) (None, None, None, 3 442368 activation_90[0][0]
__________________________________________________________________________________________________
conv2d_92 (Conv2D) (None, None, None, 3 442368 activation_90[0][0]
__________________________________________________________________________________________________
average_pooling2d_8 (AveragePoo (None, None, None, 2 0 mixed9[0][0]
__________________________________________________________________________________________________
conv2d_85 (Conv2D) (None, None, None, 3 655360 mixed9[0][0]
__________________________________________________________________________________________________
batch_normalization_87 (BatchNo (None, None, None, 3 1152 conv2d_87[0][0]
__________________________________________________________________________________________________
batch_normalization_88 (BatchNo (None, None, None, 3 1152 conv2d_88[0][0]
__________________________________________________________________________________________________
batch_normalization_91 (BatchNo (None, None, None, 3 1152 conv2d_91[0][0]
__________________________________________________________________________________________________
batch_normalization_92 (BatchNo (None, None, None, 3 1152 conv2d_92[0][0]
__________________________________________________________________________________________________
conv2d_93 (Conv2D) (None, None, None, 1 393216 average_pooling2d_8[0][0]
__________________________________________________________________________________________________
batch_normalization_85 (BatchNo (None, None, None, 3 960 conv2d_85[0][0]
__________________________________________________________________________________________________
activation_87 (Activation) (None, None, None, 3 0 batch_normalization_87[0][0]
__________________________________________________________________________________________________
activation_88 (Activation) (None, None, None, 3 0 batch_normalization_88[0][0]
__________________________________________________________________________________________________
activation_91 (Activation) (None, None, None, 3 0 batch_normalization_91[0][0]
__________________________________________________________________________________________________
activation_92 (Activation) (None, None, None, 3 0 batch_normalization_92[0][0]
__________________________________________________________________________________________________
batch_normalization_93 (BatchNo (None, None, None, 1 576 conv2d_93[0][0]
__________________________________________________________________________________________________
activation_85 (Activation) (None, None, None, 3 0 batch_normalization_85[0][0]
__________________________________________________________________________________________________
mixed9_1 (Concatenate) (None, None, None, 7 0 activation_87[0][0]
activation_88[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, None, None, 7 0 activation_91[0][0]
activation_92[0][0]
__________________________________________________________________________________________________
activation_93 (Activation) (None, None, None, 1 0 batch_normalization_93[0][0]
__________________________________________________________________________________________________
mixed10 (Concatenate) (None, None, None, 2 0 activation_85[0][0]
mixed9_1[0][0]
concatenate_1[0][0]
activation_93[0][0]
==================================================================================================
Total params: 21,802,784
Trainable params: 21,768,352
Non-trainable params: 34,432
__________________________________________________________________________________________________
单层单通道
指定层
layers_names = 'conv2d_23'
layers=base_model.get_layer(layers_names).output
print(layers)
Tensor("conv2d_23/Identity:0", shape=(None, None, None, 96), dtype=float32)
dream_model = tf.keras.Model(inputs=base_model.input, outputs=layers)
dream_model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, None, None, 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, None, None, 3 864 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, None, None, 3 96 conv2d[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, None, None, 3 0 batch_normalization[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, None, None, 3 9216 activation[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, None, None, 3 96 conv2d_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, None, None, 3 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, None, None, 6 18432 activation_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, None, None, 6 192 conv2d_2[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, None, None, 6 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, None, None, 6 0 activation_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, None, None, 8 5120 max_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, None, None, 8 240 conv2d_3[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, None, None, 8 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, None, None, 1 138240 activation_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, None, None, 1 576 conv2d_4[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, None, None, 1 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D) (None, None, None, 1 0 activation_4[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, None, None, 6 12288 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, None, 6 192 conv2d_8[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, None, None, 6 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, None, None, 4 9216 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, None, None, 9 55296 activation_8[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, None, 4 144 conv2d_6[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, None, 9 288 conv2d_9[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, None, None, 4 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, None, None, 9 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, None, None, 1 0 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, None, None, 6 12288 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, None, None, 6 76800 activation_6[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, None, None, 9 82944 activation_9[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, None, None, 3 6144 average_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, None, 6 192 conv2d_5[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, None, 6 192 conv2d_7[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, None, 9 288 conv2d_10[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, None, 3 96 conv2d_11[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, None, None, 6 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, None, None, 6 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, None, None, 9 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, None, None, 3 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
mixed0 (Concatenate) (None, None, None, 2 0 activation_5[0][0]
activation_7[0][0]
activation_10[0][0]
activation_11[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, None, None, 6 16384 mixed0[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, None, None, 6 192 conv2d_15[0][0]
__________________________________________________________________________________________________
activation_15 (Activation) (None, None, None, 6 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, None, None, 4 12288 mixed0[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, None, None, 9 55296 activation_15[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, None, 4 144 conv2d_13[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, None, None, 9 288 conv2d_16[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, None, None, 4 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, None, None, 9 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, None, None, 2 0 mixed0[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, None, None, 6 16384 mixed0[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, None, None, 6 76800 activation_13[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, None, None, 9 82944 activation_16[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, None, None, 6 16384 average_pooling2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, None, 6 192 conv2d_12[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, None, 6 192 conv2d_14[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, None, None, 9 288 conv2d_17[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, None, None, 6 192 conv2d_18[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, None, None, 6 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, None, None, 6 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, None, None, 9 0 batch_normalization_17[0][0]
__________________________________________________________________________________________________
activation_18 (Activation) (None, None, None, 6 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
mixed1 (Concatenate) (None, None, None, 2 0 activation_12[0][0]
activation_14[0][0]
activation_17[0][0]
activation_18[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, None, None, 6 18432 mixed1[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, None, None, 6 192 conv2d_22[0][0]
__________________________________________________________________________________________________
activation_22 (Activation) (None, None, None, 6 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, None, None, 9 55296 activation_22[0][0]
==================================================================================================
Total params: 781,328
Trainable params: 778,480
Non-trainable params: 2,848
__________________________________________________________________________________________________
def calc_loss(img, model, channel):
img = tf.expand_dims(img, axis=0)
layer_activations = model(img)
act = layer_activations[:,:,:,channel]
loss = tf.math.reduce_mean(act)
return loss
def render_deepdream(model, img, channel, steps=100, step_size=0.01, verbose=2):
for n in tf.range(steps):
with tf.GradientTape() as tape:
tape.watch(img)
loss = calc_loss(img, model, channel)
gradients = tape.gradient(loss, img)
gradients /= tf.math.reduce_std(gradients) + 1e-8
img += gradients * step_size
img = tf.clip_by_value(img, -1, 1)
if (verbose == 2):
if ((n + 1) % 10 == 0):
print('Step {:3}/{:3}, loss {:4}'.format(n + 1, steps, loss))
return img
img_noise = np.random.uniform(size=(300, 300, 3)) + 100.0
img_noise = img_noise.astype(np.float32)
img_Noise = img_noise.copy()
img = tf.keras.applications.inception_v3.preprocess_input(img_noise)
img = tf.convert_to_tensor(img)
指定通道时，其值应在 channel 范围内
import time
startTime = time.time()
print('DeepDream is deepdreaming......')
dream_img = render_deepdream(dream_model, img, 10)
endTime = time.time()
print('Wake up! {} flies'.format(endTime-startTime))
dream_img = normalize_image(dream_img)
file_name = './data/deepdream_{}.jpg'.format(layers_names)
save_image(dream_img, file_name)
print('DeepDream is saved!'.format(file_name))
fig = plt.gcf()
fig.set_size_inches(16, 8)
ax1 = plt.subplot(1, 2, 1)
ax1.imshow(normalize_image(img_Noise))
ax1.set_title('noise image', fontsize=20)
ax2 = plt.subplot(1, 2, 2)
ax2.imshow(dream_img)
ax2.set_title('deepdream image', fontsize=20)
plt.show()
DeepDream is deepdreaming......
Step 10/100, loss 0.37168312072753906
Step 20/100, loss 0.9784045219421387
Step 30/100, loss 1.317335844039917
Step 40/100, loss 1.5503880977630615
Step 50/100, loss 1.7381901741027832
Step 60/100, loss 1.8932645320892334
Step 70/100, loss 2.027413845062256
Step 80/100, loss 2.139453411102295
Step 90/100, loss 2.24456524848938
Step 100/100, loss 2.3410422801971436
Wake up! 8.418466806411743 flies
DeepDream is saved!
单层多通道
layers_names = 'conv2d_23'
layers=base_model.get_layer(layers_names).output
dream_model = tf.keras.Model(inputs=base_model.input, outputs=layers)
def calc_loss(img, model, channels):
img = tf.expand_dims(img, axis=0)
layer_activations = model(img)
losses = []
for cn in channels:
act = layer_activations[:,:,:,cn]
loss = tf.math.reduce_mean(act)
losses.append(loss)
return tf.reduce_sum(losses)
def render_deepdream(model, img, channels, steps=100, step_size=0.01, verbose=2):
for n in tf.range(steps):
with tf.GradientTape() as tape:
tape.watch(img)
loss = calc_loss(img, model, [10, 20])
gradients = tape.gradient(loss, img)
gradients /= tf.math.reduce_std(gradients) + 1e-8
img += gradients * step_size
img = tf.clip_by_value(img, -1, 1)
if (verbose == 2):
if ((n + 1) % 10 == 0):
print('Step {:3}/{:3}, loss {:4}'.format(n + 1, steps, loss))
return img
img_noise = np.random.uniform(size=(300, 300, 3)) + 100.0
img_noise = img_noise.astype(np.float32)
img_Noise = img_noise.copy()
img = tf.keras.applications.inception_v3.preprocess_input(img_noise)
img = tf.convert_to_tensor(img)
import time
startTime = time.time()
print('DeepDream is deepdreaming......')
dream_img = render_deepdream(dream_model, img, 10)
endTime = time.time()
print('Wake up! {} flies'.format(endTime-startTime))
dream_img = normalize_image(dream_img)
file_name = './data/deepdream_{}.jpg'.format(layers_names)
save_image(dream_img, file_name)
print('DeepDream is saved!'.format(file_name))
fig = plt.gcf()
fig.set_size_inches(16, 8)
ax1 = plt.subplot(1, 2, 1)
ax1.imshow(normalize_image(img_Noise))
ax1.set_title('noise image', fontsize=20)
ax2 = plt.subplot(1, 2, 2)
ax2.imshow(dream_img)
ax2.set_title('deepdream image', fontsize=20)
plt.show()
DeepDream is deepdreaming......
Step 10/100, loss 1.8195652961730957
Step 20/100, loss 2.877413749694824
Step 30/100, loss 3.53367280960083
Step 40/100, loss 4.005433082580566
Step 50/100, loss 4.389712333679199
Step 60/100, loss 4.730706214904785
Step 70/100, loss 5.0415239334106445
Step 80/100, loss 5.3287835121154785
Step 90/100, loss 5.600303649902344
Step 100/100, loss 5.864288330078125
Wake up! 5.1971118450164795 flies
DeepDream is saved!
多层全通道
此处 dream_model 也适用于后面两个背景图像
layers_names = ['conv2d_23', 'mixed5']
layers = [base_model.get_layer(name).output for name in layers_names]
for i in range(len(layers)):
print(layers[i])
dream_model = tf.keras.Model(inputs=base_model.input, outputs=layers)
Tensor("conv2d_23/Identity:0", shape=(None, None, None, 96), dtype=float32)
Tensor("mixed5/Identity:0", shape=(None, None, None, 768), dtype=float32)
def calc_loss(img, model):
img = tf.expand_dims(img, axis=0)
layer_activations = model(img)
losses = []
for act in layer_activations:
loss = tf.math.reduce_mean(act)
losses.append(loss)
return tf.reduce_sum(losses)
def render_deepdream(model, img, steps=100, step_size=0.01, verbose=2):
for n in tf.range(steps):
with tf.GradientTape() as tape:
tape.watch(img)
loss = calc_loss(img, model)
gradients = tape.gradient(loss, img)
gradients /= tf.math.reduce_std(gradients) + 1e-8
img += gradients * step_size
img = tf.clip_by_value(img, -1, 1)
if (verbose == 2):
if ((n + 1) % 10 == 0):
print('Step {:3}/{:3}, loss {}'.format(n + 1, steps, loss))
return img
img_noise = np.random.uniform(size=(300, 300, 3)) + 100.0
img_noise = img_noise.astype(np.float32)
img_Noise = img_noise.copy()
img = tf.keras.applications.inception_v3.preprocess_input(img_noise)
img = tf.convert_to_tensor(img)
import time
startTime = time.time()
print('DeepDream is deepdreaming......')
dream_img = render_deepdream(dream_model, img)
endTime = time.time()
print('Wake up! {} flies'.format(endTime-startTime))
dream_img = normalize_image(dream_img)
file_name = './data/deepdream_{}.jpg'.format(layers_names)
save_image(dream_img, file_name)
print('DeepDream is saved!'.format(file_name))
fig = plt.gcf()
fig.set_size_inches(16, 8)
ax1 = plt.subplot(1, 2, 1)
ax1.imshow(normalize_image(img_Noise))
ax1.set_title('noise image', fontsize=20)
ax2 = plt.subplot(1, 2, 2)
ax2.imshow(dream_img)
ax2.set_title('deepdream image', fontsize=20)
plt.show()
DeepDream is deepdreaming......
Step 10/100, loss 0.29908475279808044
Step 20/100, loss 0.5156078338623047
Step 30/100, loss 0.7052187323570251
Step 40/100, loss 0.870996356010437
Step 50/100, loss 1.0650714635849
Step 60/100, loss 1.2940750122070312
Step 70/100, loss 1.4968801736831665
Step 80/100, loss 1.7075774669647217
Step 90/100, loss 1.9374886751174927
Step 100/100, loss 2.1055333614349365
Wake up! 11.016572713851929 flies
DeepDream is saved!
背景图像
def calc_loss(img, model):
img = tf.expand_dims(img, axis=0)
layer_activations = model(img)
losses = []
for act in layer_activations:
loss = tf.math.reduce_mean(act)
losses.append(loss)
return tf.reduce_sum(losses)
def render_deepdream(model, img, steps=100, step_size=0.01, verbose=2):
for n in tf.range(steps):
with tf.GradientTape() as tape:
tape.watch(img)
loss = calc_loss(img, model)
gradients = tape.gradient(loss, img)
gradients /= tf.math.reduce_std(gradients) + 1e-8
img += gradients * step_size
img = tf.clip_by_value(img, -1, 1)
if (verbose == 2):
if ((n + 1) % 10 == 0):
print('Step {:3}/{:3}, loss {}'.format(n + 1, steps, loss))
return img
layers_names = ['conv2d_23', 'mixed5']
layers = [base_model.get_layer(name).output for name in layers_names]
for i in range(len(layers)):
print(layers[i])
dream_model = tf.keras.Model(inputs=base_model.input, outputs=layers)
Tensor("conv2d_23/Identity:0", shape=(None, None, None, 96), dtype=float32)
Tensor("mixed5/Identity:0", shape=(None, None, None, 768), dtype=float32)
img_scene = read_image('./data/uestc.jpg')
img_scene = img_scene.astype(np.float32)
img_Scene = img_scene.copy()
img = tf.keras.applications.inception_v3.preprocess_input(img_scene)
img = tf.convert_to_tensor(img)
import time
startTime = time.time()
print('DeepDream is deepdreaming......')
dream_img = render_deepdream(dream_model, img, steps=400)
endTime = time.time()
print('Wake up! {} flies'.format(endTime-startTime))
dream_img = normalize_image(dream_img)
file_name = './data/deepdream_{}.jpg'.format(layers_names)
save_image(dream_img, file_name)
print('DeepDream is saved!'.format(file_name))
fig = plt.gcf()
fig.set_size_inches(16, 16)
ax1 = plt.subplot(2, 1, 1)
ax1.imshow(img_Scene.astype('uint8'))
ax1.set_title('raw image', fontsize=20)
ax2 = plt.subplot(2, 1, 2)
ax2.imshow(dream_img)
ax2.set_title('deepdream image', fontsize=20)
plt.show()
DeepDream is deepdreaming......
Step 10/400, loss 0.09298734366893768
Step 20/400, loss 0.24137751758098602
Step 30/400, loss 0.3336787521839142
Step 40/400, loss 0.4055805802345276
Step 50/400, loss 0.46549665927886963
Step 60/400, loss 0.5207889676094055
Step 70/400, loss 0.5699973702430725
Step 80/400, loss 0.6171151399612427
Step 90/400, loss 0.653640866279602
Step 100/400, loss 0.6943302154541016
Step 110/400, loss 0.7312059998512268
Step 120/400, loss 0.7702720761299133
Step 130/400, loss 0.8076008558273315
Step 140/400, loss 0.8459428548812866
Step 150/400, loss 0.8861863613128662
Step 160/400, loss 0.9219092726707458
Step 170/400, loss 0.9596524238586426
Step 180/400, loss 0.9966444373130798
Step 190/400, loss 1.0375070571899414
Step 200/400, loss 1.0739365816116333
Step 210/400, loss 1.1114788055419922
Step 220/400, loss 1.1500513553619385
Step 230/400, loss 1.1858978271484375
Step 240/400, loss 1.2259176969528198
Step 250/400, loss 1.2619391679763794
Step 260/400, loss 1.305344820022583
Step 270/400, loss 1.3415778875350952
Step 280/400, loss 1.3795169591903687
Step 290/400, loss 1.4158217906951904
Step 300/400, loss 1.4519184827804565
Step 310/400, loss 1.4838485717773438
Step 320/400, loss 1.520896553993225
Step 330/400, loss 1.5560493469238281
Step 340/400, loss 1.5878287553787231
Step 350/400, loss 1.621181607246399
Step 360/400, loss 1.6572657823562622
Step 370/400, loss 1.6896072626113892
Step 380/400, loss 1.7202728986740112
Step 390/400, loss 1.7503917217254639
Step 400/400, loss 1.7812590599060059
Wake up! 127.6007616519928 flies
DeepDream is saved!
背景图像过程优化1
采取由小到大的方式生成，先将图像缩小指定倍数，进行 Deepdream 后，放大一次，再进行 Deepdream，如此循环几次
img_scene = read_image('./data/uestc.jpg')
img_Scene = img_scene.copy()
img = tf.keras.applications.inception_v3.preprocess_input(img_scene)
img = tf.convert_to_tensor(img)
import time
startTime = time.time()
OCTAVE_SCALE = 1.30
img = tf.keras.applications.inception_v3.preprocess_input(img_scene)
img = tf.convert_to_tensor(img)
initial_shape = tf.shape(img)[:-1]
for octave in range(-3, 2):
new_size = tf.cast(tf.convert_to_tensor(initial_shape),
tf.float32) * (OCTAVE_SCALE ** octave)
img = tf.image.resize(img, tf.cast(new_size, tf.int32))
img = render_deepdream(dream_model, img, steps=50, step_size=0.01)
img = tf.image.resize(img, initial_shape)
img = normalize_image(img)
show_image(img)
endTime = time.time()
print('Total time:', startTime - endTime)
Step 10/ 50, loss 0.1253846138715744
Step 20/ 50, loss 0.28544867038726807
Step 30/ 50, loss 0.3829663395881653
Step 40/ 50, loss 0.4530993700027466
Step 50/ 50, loss 0.5086963176727295
Step 10/ 50, loss 0.31151697039604187
Step 20/ 50, loss 0.4383772909641266
Step 30/ 50, loss 0.5194989442825317
Step 40/ 50, loss 0.5801485180854797
Step 50/ 50, loss 0.6286039352416992
Step 10/ 50, loss 0.35074129700660706
Step 20/ 50, loss 0.4853087365627289
Step 30/ 50, loss 0.5703152418136597
Step 40/ 50, loss 0.6327226161956787
Step 50/ 50, loss 0.6832340955734253
Step 10/ 50, loss 0.37902626395225525
Step 20/ 50, loss 0.5169548392295837
Step 30/ 50, loss 0.6002033948898315
Step 40/ 50, loss 0.6597338914871216
Step 50/ 50, loss 0.7072175741195679
Step 10/ 50, loss 0.39113712310791016
Step 20/ 50, loss 0.5287901163101196
Step 30/ 50, loss 0.6115886569023132
Step 40/ 50, loss 0.6704986691474915
Step 50/ 50, loss 0.7165963053703308
Total time: -69.93595671653748
背景图像过程优化2
在背景图像过程优化1基础上，增加了块状计算方式，这样再大的图像也能进行分块运算，最后再合成
img_scene = read_image('./data/test.jpg')
img_Scene = img_scene.copy()
img = tf.keras.applications.inception_v3.preprocess_input(img_scene)
img = tf.convert_to_tensor(img)
相比于 背景图像过程优化1，此处重新定义了 calc_loss，并重命名为 calc_loss_optimization，其目的是增加了 tf.image.resize 操作，在我们按块操作时，最后剩余的图像尺寸可能不够进行提取模型的一系列操作，所以我们对其进行放大，为最大限度减小误差，我们放大的尺寸与块状尺寸相同，如果不进行此操作，在不同的 dream_model 层，不同的 octave 与 octave_scale，可能出现报错为:
Check failed: cudnnSetTensorNdDescriptor(handle_.get(), elem_type, nd, dims.data(), strides.data()) == CUDNN_STATUS_SUCCESS (3 vs. 0)batch_descriptor: {count: 1 feature_map_count: 192 spatial: 0 61 value_min: 0.000000 value_max: 0.000000 layout: BatchYXDepth}
def calc_loss_optimization(img, model, size):
img = tf.image.resize(img, tf.cast([size, size], tf.int32))
img = tf.expand_dims(img, axis=0)
layer_activations = model(img)
losses = []
for act in layer_activations:
loss = tf.math.reduce_mean(act)
losses.append(loss)
return tf.reduce_sum(losses)
def random_roll(img, maxroll=512):
shift = tf.random.uniform(shape=[2], minval=-maxroll, maxval=maxroll, dtype=tf.int32)
shift_down, shift_right = shift[0], shift[1]
img_rolled = tf.roll(tf.roll(img, shift_right, axis=1), shift_down, axis=0)
return shift_down, shift_right, img_rolled
def get_tiled_gradients(model, img, tile_size=512):
shift_down, shift_right, img_rolled = random_roll(img, tile_size)
gradients = tf.zeros_like(img_rolled)
xs = tf.range(0, img_rolled.shape[0], tile_size)
ys = tf.range(0, img_rolled.shape[1], tile_size)
for x in xs:
for y in ys:
with tf.GradientTape() as tape:
tape.watch(img_rolled)
img_tile = img_rolled[x:x + tile_size, y:y + tile_size]
loss = calc_loss_optimization(img_tile, model, tile_size)
gradients = gradients + tape.gradient(loss, img_rolled)
gradients = tf.roll(tf.roll(gradients, -shift_right, axis=1), -shift_down, axis=0)
gradients /= tf.math.reduce_std(gradients) + 1e-8
return gradients
def render_deepdream_with_octaves(model, img, steps_per_octave=100, step_size=0.01,
octaves=range(-3, -1), octave_scale=1.3):
initial_shape = img.shape[:-1]
for octave in octaves:
new_size = tf.cast(tf.convert_to_tensor(initial_shape), tf.float32) * (octave_scale ** octave)
img = tf.image.resize(img, tf.cast(new_size, tf.int32))
for step in range(steps_per_octave):
gradients = get_tiled_gradients(model, img)
img = img + gradients * step_size
img = tf.clip_by_value(img, -1, 1)
if ((step + 1) % 10 == 0):
print('Octave {}, Step {}'.format(octave, step + 1))
img = tf.image.resize(img, initial_shape)
result = normalize_image(img)
return result
import time
startTime = time.time()
print('DeepDream is deepdreaming......')
img = render_deepdream_with_octaves(dream_model, img, steps_per_octave=50,
step_size=0.01, octaves=range(-3, 2), octave_scale=1.3)
show_image(img)
endTime = time.time()
print('Total time:', endTime - startTime)
file_name = './data/uestc_deepdream.jpg'
save_image(img, file_name)
print('Deepdream is saved!')
DeepDream is deepdreaming......
Octave -3, Step 10
Octave -3, Step 20
Octave -3, Step 30
Octave -3, Step 40
Octave -3, Step 50
Octave -2, Step 10
Octave -2, Step 20
Octave -2, Step 30
Octave -2, Step 40
Octave -2, Step 50
Octave -1, Step 10
Octave -1, Step 20
Octave -1, Step 30
Octave -1, Step 40
Octave -1, Step 50
Octave 0, Step 10
Octave 0, Step 20
Octave 0, Step 30
Octave 0, Step 40
Octave 0, Step 50
Octave 1, Step 10
Octave 1, Step 20
Octave 1, Step 30
Octave 1, Step 40
Octave 1, Step 50
Total time: 115.36939454078674
Deepdream is saved!