TensorFlow learning process record (2) -- basic use (1) -- building graph

This chapter is based on the Chinese documentation of TensorFlow.
I also follow the blog to learn step by step and make my own understanding.

1. Build the graph

The documentation states that the first step in building a graph is to create a source op, which does not require any input and whose output can also be used to perform operations on other ops.

In the Python library, the return value of the op constructor represents the output of the constructed op, and these return values ​​can be passed to other op constructors as input.
The TensorFlow Python library has a default graph, and the op constructor can be used for Add nodes. This default graph is sufficient for many programs

The documentation gives the code:

import tensorflow as tf

# 创建一个常量 op, 产生一个 1x2 矩阵. 这个 op 被作为一个节点
# 加到默认图中.
#
# 构造器的返回值代表该常量 op 的返回值.
matrix1 = tf.constant([[3., 3.]])

# 创建另外一个常量 op, 产生一个 2x1 矩阵.
matrix2 = tf.constant([[2.],[2.]])

# 创建一个矩阵乘法 matmul op , 把 'matrix1' 和 'matrix2' 作为输入.
# 返回值 'product' 代表矩阵乘法的结果.
product = tf.matmul(matrix1, matrix2)

The default graph now has three nodes, two constant() ops, and one matmul() op.

1.1 constant

The constant node represents a constant, and the function signature is:

def constant(value, dtype=None, shape=None, name="Const", verify_shape=False)

After creating a constant tensor through constant. The resulting tensor is dtypea value of type, specified by the arguments valueand shape(optionally).
The parameter dtypecan be a constant value or dtypea list of values ​​of type. If valuea list, the length of the list must be less than or equal to shapethe number of elements contained by the argument (if specified shape). In cases where the list length is less than shapethe number of elements specified by , the last element in the list will be used to fill the remaining entries.
Parameters shapeare optional. If present, specifies the dimension of the resulting tensor. valueThe shape to use if not present .
If no argument is specified dtype, the valuetype is inferred from the type.

# 一维tensor：用列表填充
tensor = tf.constant([1, 2, 3, 4, 5, 6, 7]) => [1 2 3 4 5 6 7]

# 二维tensor:用-1填充
tensor = tf.constant(-1.0, shape=[2, 3]) => [[-1. -1. -1.]
                                            [-1. -1. -1.]]

parameter:

value	dtype	shape	name	verify_shape
a dtypeconstant (or list) value of type	the element type of the resulting tensor	optional dimension of the resulting tensor	the name of the tensor (optional)	Boolean, specifying whether the tensor supports shape

result	abnormal
a constant tensor	TypeError: if shape is not supported or incorrect

Therefore, in the code given in the previous document

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.], [2.]])

produces a 1x2 matrix and a 2x1 matrix respectively
Then , in

product = tf.matmul(matrix1, matrix2)

, creates a matrix multiplication matmul , taking 'matrix1' and 'matrix2' as input

1.2 matmul

The function signature of matmul is:

def matmul(a,
           b,
           transpose_a=False,
           transpose_b=False,
           adjoint_a=False,
           adjoint_b=False,
           a_is_sparse=False,
           b_is_sparse=False,
           name=None):

The op multiplies the proof 'a' by the matrix 'b', and the output 'a' * 'b'
after any exchange, the output must be a tensor of rank >= 2, where the inner two dimensions specify valid matrix multiplication parameters . and matches any external dimension.
The two matrices must be of the same type. The supported types are: float16, float32, float64, int32, complex64, the complex128
matrix can be converted or conjugated, these settings are False by default, which can be achieved by setting transpose_a, transpose_b, adjoint_a, adjoint_b to True.
If one or both matrices contain some 0s, then setting the corresponding a_is_sparse or (and) b_is_sparse to True can make the multiplication more efficient.
This optimization only works for ordinary matrices (rank 2 tensors) of data type "bfloat16" or "float32".
for example:

# 2维 tensor 'a'
# [[1, 2, 3],
# [4, 5, 6]]
a = tf.constant([1, 2, 3, 4, 5, 6], shape=[2, 3])

# 二维 tensor `b`
# [[ 7,  8],
#  [ 9, 10],
#  [11, 12]]
b = tf.constant([7, 8, 9, 10, 11, 12], shape=[3, 2])

# `a` * `b`
# [[ 58,  64],
#  [139, 154]]
c = tf.matmul(a, b)

# 三维 tensor `a`
  # [[[ 1,  2,  3],
  #   [ 4,  5,  6]],
  #  [[ 7,  8,  9],
  #   [10, 11, 12]]]
  a = tf.constant(np.arange(1, 13, dtype=np.int32),
                  shape=[2, 2, 3])
# 三维 tensor `b`
  # [[[13, 14],
  #   [15, 16],
  #   [17, 18]],
  #  [[19, 20],
  #   [21, 22],
  #   [23, 24]]]
  b = tf.constant(np.arange(13, 25, dtype=np.int32),
                  shape=[2, 3, 2])
# `a` * `b`
  # [[[ 94, 100],
  #   [229, 244]],
  #  [[508, 532],
  #   [697, 730]]]
  c = tf.matmul(a, b)

Since python3.5, the @ operator has been supported. In TensorFlow, it is a tf.matmul()function, so the following two lines are equivalent:

  d = a @ b @ [[10.], [11.]]
  d = tf.matmul(tf.matmul(a, b), [[10.], [11.]])

parameter:

a and b	transpose_a(b)	adjoint_a(b)	a(b)_is_sparse	name
float16.. and other types (described earlier) and rank greater than 1Tensor	matrix a or b permuted before multiplication	matrix a or b is conjugated before multiplication	a or b is a sparse matrix	give the op a name

result	abnormal
Output a matrix of the same type as a and b, and the product of a and b matrices.	ValueError: if transpose_a, adjoint_a, or transpose_b and adjoint_b are all set to True

So in the documentation:

product = tf.matmul(matrix1, matrix2)

product represents the product of the two matrices matrix1 and matrix2

1.3 Start the default map

In order to actually do the matrix multiplication and get the result of the matrix multiplication, you have to start the graph in the session.

sess = tf.Session()
result = sess.run(product)
# 整个执行过程是自动化的, 会话负责传递 op 所需的全部输入. op 通常是并发执行的
# 函数调用 'run(product)' 触发了图中三个 op (两个常量 op 和一个矩阵乘法 op) 的执行.

1.4 Output results

Finally print the result result:

print(result)  # ==> [[ 12.]]

1.5 Close the session

The Session object needs to be closed after use to release resources. In addition to explicitly calling close, you can also use the "with" block to automatically complete the closing action:

with tf.Session() as sess:
    result = sess.run([product])
    print result

In implementation, TensorFlow converts graph definitions into distributed execution operations to take full advantage of available computing resources (such as CPU or GPU). Generally, you do not need to explicitly specify whether to use CPU or GPU, TensorFlow can automatically detect. If it detects GPU, TensorFlow will try to use the first GPU found to perform operations.
If there is more than one GPU available on the machine, other GPUs except the first are not involved in the calculation by default. In order for TensorFlow to use these GPUs, You must explicitly assign ops to them for execution. The with…Device statement is used to assign a specific CPU or GPU to execute an operation:

with tf.Session() as sess:
    with tf.device("/gpu:1"):
        matrix1 = tf.constant([[3., 3.]])
        matrix2 = tf.constant([[2.],[2.]])
        product = tf.matmul(matrix1, matrix2)
        ...

"/cpu:0": the machine's CPU.
"/gpu:0": the machine's first GPU, if any.
"/gpu:1": the machine's second GPU, and so on.