1. Basic concepts of TensorFlow
The three most basic concepts in tensorflow are calculation graph ( tf.Graph ), tensor ( tf.Tensor ) and session (tf.Session ).
Computation graph is tensorFlow's computation model. All TensorFlow programs are expressed in the form of computational graphs. Each node on the calculation graph is an operation, and the edges on the calculation graph represent the data transfer relationship between operations. The computing graph also saves information about the device running each operation (for example, whether it is running on a CPU or a GPU) and the dependencies between operations. Computational graphs provide the ability to manage different collections, and TensorFlow automatically maintains a different default collection.
Tensor is the data model of TensorFlow, and the input and output of operations in TensorFlow are all tensors. A tensor does not store any data itself, it is just a reference to the result of an operation. TensorFlow programs can be better organized through tensors.
A session is a TensorFlow computing model that manages the system resources owned by a TensorFlow program, and all operations are performed through a session.
**
2. The process of training a neural network can be divided into the following three steps:**
1.定义神经网络的结构和前向传播的输出结果
2. 定义损失函数以及选择反向传播优化的算法
3. 生成会话（ tf.Session ）并且在训练集数据上反复运行反向传播优化算法
No matter how the structure of the neural network changes, these 3 steps are invariant.
**
3. Neural network structure design and training optimization methods
**
In terms of the structure of the neural network, on the one hand, deep learning needs to use the activation function (Activation Function) to realize the delinearization of the neural network model, and on the other hand, it needs to use one or more hidden layers to make the structure of the neural network deeper to solve complex issues.
When training a neural network, you can use a learning rate setting with exponential decay, use regularization to avoid overfitting, and use a moving average model to make the final model more robust.