Chapter 2 of "Practice Deep Learning" (TF2.0 Edition)

Chapter two

import tensorflow as tf
print(tf.__version__)

Check the version of TF

2.2 Data manipulation

2.2.1 Create tensor

1. Create tensor

x = tf.constant(range(12))#constant 代表是常量的tensor
print(x.shape)
x
print(x)
len(x)#可以用len函数获取长度

(12,)
<tf.Tensor: id=0, shape=(12,), dtype=int32, numpy=array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])>
12

2. Use the reshape function
   Use the reshape function to change the shape of the row vector x to (3, 4), which is a matrix with 3 rows and 4 columns, and denote it as X

 X = tf.reshape(x,(3,4))#第一个参数为tensor，第二个参数为()元组

The above x.reshape((3, 4)) can also be written as x.reshape((-1, 4)) or x.reshape((3, -1)). Since the number of elements of x is known, -1 here can be inferred from the number of elements and the size of other dimensions.

3. Create a tensor with each element being 0 and shape (2, 3, 4)

tf.zeros((2,3,4))

4. Create a tensor with 1 element

tf.ones((3,4))

5. Specify the value of each element in the tensor that needs to be created through the Python list (list)

Y = tf.constant([[2,1,4,3],[1,2,3,4],[4,3,2,1]])
Y

<tf.Tensor: id=9, shape=(3, 4), dtype=int32, numpy=
array([[2, 1, 4, 3],
       [1, 2, 3, 4],
       [4, 3, 2, 1]])>

6. We need to randomly generate the value of each element in the tensor. Next we create a tensor with shape (3, 4). Each element of it is randomly sampled from a normal distribution with a mean of 0 and a standard deviation of 1.

x=tf.random.normal(shape=[3,4], mean=0, stddev=1)
print(x)

tf.Tensor(
[[-1.153268    0.6116716   0.9703915  -2.7604232 ]
 [ 0.48349026 -0.14327626  2.940394    0.98280823]
 [ 0.11714476 -1.9485139  -0.46181852 -0.23992358]], 
 shape=(3, 4), dtype=float32)

2.2.2 Operation

1. Multiply and divide

X + Y  #这里的X,Y有相同的形状，对应元素可以相加
X * Y#对应的元素相乘
X / Y

2. Exponential calculation

Y = tf.cast(Y, tf.float32)#这里先转换tensor的类型

x=tf.reshape(tf.constant(range(12)),(3,4))
print(x)
x = tf.cast(x, tf.float32)
print(tf.exp(x))

tf.Tensor(
[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]], shape=(3, 4), dtype=int32)
tf.Tensor(
[[1.0000000e+00 2.7182817e+00 7.3890562e+00 2.0085537e+01]
 [5.4598152e+01 1.4841316e+02 4.0342880e+02 1.0966332e+03]
 [2.9809580e+03 8.1030840e+03 2.2026467e+04 5.9874145e+04]], shape=(3, 4), dtype=float32)

Note here that cast type conversion must be required here

3. Matrix multiplication, matmul function to do matrix multiplication

Y = tf.cast(Y, tf.int32)
tf.matmul(X, tf.transpose(Y))

4. Matrix transpose

tf.transpose(Y)

5. tensor stitching

tf.concat([X,Y],axis = 0)#行为axis=0，结果shape （6,4）
tf.concat([X,Y],axis = 1)#列为axis=1，结果shape （3,8）

6. If the judgment value is equal, take X == Y as an example. If the condition that X and Y are at the same position is judged to be true (the value is equal), then the value of the new tensor at the same position is 1; otherwise, it is 0.

tf.equal(X,Y)

<tf.Tensor: id=31, shape=(3, 4), dtype=bool, numpy=
array([[False,  True, False,  True],
       [False, False, False, False],
       [False, False, False, False]])>

7. Sum

tf.reduce_sum(X)

2.2.3 Broadcast mechanism

Perform element-wise operations on two tensors with the same shape. When performing element-wise operations on two tensors with different shapes, a broadcasting mechanism may be triggered: first copy the elements appropriately to make the two tensors have the same shape, and then perform element-wise operations.

2.2.4 Index

In tensor, the index (index) represents the position of the element. The index of tensor starts from 0 and increases one by one. For example, a matrix with 3 rows and 2 columns has row indexes 0, 1, and 2, and column indexes 0 and 1, respectively.

X[1:3]
#X为3行4列，这句话等于X[1:3,:],依据左闭右开指定范围的惯例，它截取了矩阵X中行索引为1和2的两行。

<tf.Tensor: id=50, shape=(2, 4), dtype=float32, numpy=
array([[ 4.,  5.,  6.,  7.],
       [ 8.,  9., 10., 11.]], dtype=float32)>

1. Assignment
   Specify the position of a single element that needs to be accessed in the tensor, such as the index of the row and column in the matrix, and re-assign the element.

X = tf.Variable(X)#从常量转换为变量
X[1,2].assign(9)

<tf.Variable 'UnreadVariable' shape=(3, 4) dtype=float32, numpy=
array([[ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  9.,  7.],
       [ 8.,  9., 10., 11.]], dtype=float32)>

2.2.5 Operational memory overhead

Tensor and NumPy mutual transformation

import numpy as np

P = np.ones((2,3))
D = tf.constant(P)

The following is the tensor of TF

<tf.Tensor: id=115, shape=(2, 3), dtype=float64, numpy=
array([[1., 1., 1.],
       [1., 1., 1.]])>

After converting back, the
following is np tensor

np.array(D)

2.3 Automatic gradient

A function request on the gradient column vectors of xx.

x = tf.reshape(tf.Variable(range(4), dtype=tf.float32),(4,1))

with tf.GradientTape() as t:
    t.watch(x)
    y = 2 * tf.matmul(tf.transpose(x), x)

dy_dx = t.gradient(y, x)
dy_dx

<tf.Tensor: id=30, shape=(4, 1), dtype=float32, numpy=
array([[ 0.],
       [ 4.],
       [ 8.],
       [12.]], dtype=float32)>

with tf.GradientTape(persistent=True) as g:
    g.watch(x)
    y = x * x
    z = y * y
    dz_dx = g.gradient(z, x)  # 108.0 (4*x^3 at x = 3)
    dy_dx = g.gradient(y, x)  # 6.0
dz_dx,dy_dx

WARNING:tensorflow:Calling GradientTape.gradient on a persistent tape inside its context is significantly less efficient than calling it outside the context (it causes the gradient ops to be recorded on the tape, leading to increased CPU and memory usage). Only call GradientTape.gradient inside the context if you actually want to trace the gradient in order to compute higher order derivatives.
WARNING:tensorflow:Calling GradientTape.gradient on a persistent tape inside its context is significantly less efficient than calling it outside the context (it causes the gradient ops to be recorded on the tape, leading to increased CPU and memory usage). Only call GradientTape.gradient inside the context if you actually want to trace the gradient in order to compute higher order derivatives.

(<tf.Tensor: id=41, shape=(4, 1), dtype=float32, numpy=
 array([[  0.],
        [  4.],
        [ 32.],
        [108.]], dtype=float32)>,
 <tf.Tensor: id=47, shape=(4, 1), dtype=float32, numpy=
 array([[0.],
        [2.],
        [4.],
        [6.]], dtype=float32)>)

with tf.GradientTape(persistent=True) as g:
    g.watch(x)
    y = x * x
    z = y * y
    dz_dx = g.gradient(z, x)  # 108.0 (4*x^3 at x = 3)
    dy_dx = g.gradient(y, x)  # 6.0
dz_dx,dy_dx

WARNING:tensorflow:Calling GradientTape.gradient on a persistent tape inside its context is significantly less efficient than calling it outside the context (it causes the gradient ops to be recorded on the tape, leading to increased CPU and memory usage). Only call GradientTape.gradient inside the context if you actually want to trace the gradient in order to compute higher order derivatives.
WARNING:tensorflow:Calling GradientTape.gradient on a persistent tape inside its context is significantly less efficient than calling it outside the context (it causes the gradient ops to be recorded on the tape, leading to increased CPU and memory usage). Only call GradientTape.gradient inside the context if you actually want to trace the gradient in order to compute higher order derivatives.

(<tf.Tensor: id=41, shape=(4, 1), dtype=float32, numpy=
 array([[  0.],
        [  4.],
        [ 32.],
        [108.]], dtype=float32)>,
 <tf.Tensor: id=47, shape=(4, 1), dtype=float32, numpy=
 array([[0.],
        [2.],
        [4.],
        [6.]], dtype=float32)>)

2.4

When you want to know the specific usage of a certain function or class, you can use the help function. Let us take the ones function as an example to check its usage. For more detailed information, you can select the API version consistent with the tensorflow version in your environment to query through the Tensorflow API document version selection page.

help(tf.ones)

When we want to know which functions and classes are available in a module, we can use the dir function. Below we print all the members or attributes in the dtypes and random modules.

When we want to know which functions and classes are available in a module, we can use the dir function. Below we print all the members or attributes in the dtypes and random modules.

dir(tf.dtypes)

['DType',
 'QUANTIZED_DTYPES',
 '__builtins__',
 '__cached__',
 '__doc__',
 '__file__',
 '__loader__',
 '__name__',
 '__package__',
 '__path__',
 '__spec__',
 '_sys',
 'as_dtype',
 'bfloat16',
 'bool',
 'cast',
 'complex',
 'complex128',
 'complex64',
 'double',
 'float16',
 'float32',
 'float64',
 'half',
 'int16',
 'int32',
 'int64',
 'int8',
 'qint16',
 'qint32',
 'qint8',
 'quint16',
 'quint8',
 'resource',
 'saturate_cast',
 'string',
 'uint16',
 'uint32',
 'uint64',
 'uint8',
 'variant']