TensorBoard visualization
tensorflow is not only a computational graph software, it also includes the tensorboard visualization tool, which is installed by default when tensorflow is installed. The usage method is very simple. Use writer = tf.summary.FileWriter('./graph', sess.graph) to A file writer can be created, ./graph is the storage directory, and sess.graph represents the read graph structure.
We can write a simple little program
import tensorflow as tf
a = tf.constant(2)
b = tf.constant(3)
x = tf.add(a, b)
with tf.Session() as sess:
writer = tf.summary.FileWriter('./graphs', sess.graph)
print(sess.run(x))
writer.close() # close the writer when you’re done using it
Constant types
A constant can be created in the following way
tf.constant(value, dtype=None, shape=None, name='Const', verify_shape=False)
For example, create one-dimensional vectors and matrices, and then multiply them together
a = tf.constant([2, 2], name='a')
b = tf.constant([[0, 1], [2, 3]], name='b')
x = tf.multiply(a, b, name='dot_production')
with tf.Session() as sess:
print(sess.run(x))
>> [[0, 2]
[4, 6]]
This is similar to the one in numpy, and there are also constant creation of some special values.
tf.zeros(shape, dtype=tf.float32, name=None)
tf.zeros_like(input_tensor, dtype=None, name=None, optimize=True)
tf.ones(shape, dtype=tf.float32, name=None)
tf.ones_like(input_tensor, dtype=None, name=None, optimize=True)
tf.fill(dims, value, name=None)
tf.fill([2, 3], 8)
>> [[8, 8, 8], [8, 8, 8]]
There are also sequence creation similar to numpy
tf.linspace(start, stop, num, name=None)
tf.linspace(10.0, 13.0, 4)
>> [10.0, 11.0, 12.0, 13.0]
tf.range(start, limit=None, delta=1, dtype=None, name='range')
tf.range(3, limit=18, delta=3)
>> [3, 6, 9, 12, 15]
The biggest difference between this and numpy is that it cannot iterate, that is
for _ in tf.range(4): # TypeError
In addition, some random numbers can be generated
tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)
tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None,
name=None)
tf.random_uniform(shape, minval=0, maxval=None, dtype=tf.float32, seed=None,
name=None)
tf.random_shuffle(value, seed=None, name=None)
tf.random_crop(value, size, seed=None, name=None)
tf.multinomial(logits, num_samples, seed=None, name=None)
tf.random_gamma(shape, alpha, beta=None, dtype=tf.float32, seed=None, name=None)
In addition, the data types of tensorflow and numpy can be used in general, that is to say
tf.ones([2, 2], np.float32)
>> [[1.0, 1.0], [1.0, 1.0]]
It is best not to use the data types that come with python, and be careful when using numpy data types, because the data types of tensorflow and numpy may no longer be compatible in the future.
Variable (Variable)
What's wrong with using constants? Constants will exist in the definition of the calculation graph. If there are too many constants, it will make loading the calculation graph very slow, and the value of the constant cannot be changed, so variables are introduced.
a = tf.Variable(2, name='scalar')
b = tf.Variable([2, 3], name='vector')
c = tf.Variable([[0, 1], [2, 3]], name='matrix')
w = tf.Variable(tf.zeros([784, 10]), name='weight')
Variables have the following operations
x = tf.Variable()
x.initializer # 初始化
x.eval() # 读取里面的值
x.assign() # 分配值给这个变量
Note that a variable must be initialized before it can be used. Initialization can be seen as a variable assignment operation. The easiest way to initialize is to initialize all variables at once
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
You can also initialize some variables
init_ab = tf.variable_initializer([a, b], name='init_ab')
with tf.Session() as sess:
sess.run(init_ab)
or initialize a variable
w = tf.Variable(tf.zeros([784, 10]))
with tf.Session() as sess:
sess.run(w.initializer)
If we want to get the value of a variable, there are two ways
w = tf.Variable(tf.truncated_normal([10, 10], name='normal'))
with tf.Session() as sess:
sess.run(w.initializer)
print(w.eval()) # 方法一
print(sess.run(w)) # 方法二
Check out this little program below
w = tf.Variable(10)
w.assign(100)
with tf.Session() as sess:
sess.run(w.initializer)
print(w.eval())
>> 10
The above degree will get 10, this is because although we define the assign operation, tensorflow performs the operation in the session, so we need to perform the assign operation.
w = tf.Variable(10)
assign_op = w.assign(100)
with tf.Session() as sess:
sess.run(w.initializer)
sess.run(assign_op)
print(w.eval())
>> 100
In addition, each session of tensorflow is independent of each other, we can look at the following example
W = tf.Variable(10)
sess1 = tf.Session()
sess2 = tf.Session()
sess1.run(W.initializer)
sess2.run(W.initializer)
print(sess1.run(W.assign_add(10))) # >> 20
print(sess2.run(W.assign_sub(2))) # >> 8
print(sess1.run(W.assign_add(100))) # >> 120
print(sess2.run(W.assign_sub(50))) # >> -42
sess1.close()
sess2.close()
You can also define a variable based on a variable
w = tf.Variable(tf.truncated_normal([700, 10]))
u = tf.Variable(w * 2)
Placeholders
tensorflow中一般有两步，第一步是定义图，第二步是在session中进行图中的计算。对于图中我们暂时不知道值的量，我们可以定义为占位符，之后再用`feed_dict`去赋值。
定义占位符的方式非常简单
tf.placeholder(dtype, shape=None, name=None)
dtype是必须要指定的参数，shape如果是None，说明任何大小的tensor都能够接受，使用shape=None很容易定义好图，但是在debug的时候这将成为噩梦，所以最好是指定好shape。
我们可以给出下面的小例子。
a = tf.placeholder(tf.float32, shape=[3])
b = tf.constant([5, 5, 5], tf.float32)
c = a + b
with tf.Session() as sess:
print(sess.run(c, feed_dict={a: [1, 2, 3]}))
除此之外，也可以给tensorflow中的运算进行feed操作，如下
a = tf.add(2, 3)
b = tf.multiply(a, 3)
with tf.Session() as sess:
print(sess.run(b, feed_dict={a: 2}))
>> 6
lazy loading
lazy loading是指你推迟变量的创建直到你必须要使用他的时候。下面我们看看一般的loading和lazy loading的区别。
# normal loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
z = tf.add(x, y)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for _ in range(10):
sess.run(z)
# lazy loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for _ in range(10):
sess.run(tf.add(x, y))
normal loading 会在图中创建x和y变量，同时创建x+y的运算，而lazy loading只会创建x和y两个变量。这不是一个bug，那么问题在哪里呢？
normal loading在session中不管做多少次x+y，只需要执行z定义的加法操作就可以了，而lazy loading在session中每进行一次x+y，就会在图中创建一个加法操作，如果进行1000次x+y的运算，normal loading的计算图没有任何变化，而lazy loading的计算图会多1000个节点，每个节点都表示x+y的操作。
看到了吗，这就是lazy loading造成的问题，这会严重影响图的读入速度。