01-TensorFlow2.0 basis

01-TensorFlow basis

What is Tensorflow

Google's open-source software library

• Take data flow diagram, a numerical calculation
• Support for multiple platforms - GPU, CPU, mobile device
• Initially used for deep learning, becoming more and more common

Tensorflow data structure

Data flow diagram #

1. Input-output relationship between the nodes, the transport line tensor: Line. Tensor: tensor - refers to data

2. Node: operation (op): The special arithmetic operation node, all operations are an op, data processing

• Just use tensorflow of API functions are defined in OP
• Node is assigned to run on a variety of computing devices
  

1. graph: the overall program structure of FIG.

• Is essentially a memory location allocated, there is a default map, all tensor op and memory addresses are the same.
• FIG different memory addresses are not the same, the process of calculating the interfering

1. session: session: FIG computation program (running only a map, can be specified in the session to run FIG graph = g)

• Structure diagram of
• Allocation of computing resources
• Control over resources (variables, queues, thread)

The characteristics Tensorflow

• High degree of flexibility, ease of calling the function, you can also write your own package
• True portability, you can simply run on different devices
• Product research and combination
• Automatic differentiated, mainly for the reverse propagation calculations
• Multi-language support, C ++, Java, JS, R
• Performance optimization

The front end system Tensorflow

1. FIG mechanism defined procedure: front-end system
2. Backend system: calculation map structure

Tensorflow version changes

Tensorflow1.0 Key Features

• XLA: Accelerate Linear Algebra
  • Enhance the training speed 58 times
  • You can be run on the mobile device
• It cited higher-level API
  • tf.layers/ tf.metrics / tf.losses/ tf.keras
• Tensorflow debugger
• Support docker mirroring, introduced tensorflow serving services
  

Tensorflow 2.0 Key Features

• Tf.keras and eager mode using a simple model building
• Robust cross-platform deployment model
• Strong research experiments
• 清除了不推荐使用和重复的API
  

Tensorflow2.0 简化模型开发流程

1. 使用tf.data加载数据
2. 使用tf.keras 构建模型，也可以使用premade estimator 验证模型
   • 使用tensorflow hub进行迁移学习
   • 注： 迁移学习 - 使用一个前人预先训练好的，应用在其他领域的网络作为模型训练的起点，站在前人基础上更进一步，不必重新发明轮子。
3. 使用eager mode 进行运行和调试
4. 使用分发策略进行分布式训练
5. 导出到SavedModel
6. 使用Tensorflow Serve, Tensorflow Lite, Tensorflow.js

Tensorflow 强大的跨平台能力

• Tensorflow 服务
  • 直接通过HTTP/ TEST 或 GTPC/协议缓冲区
• Tensorflow Lite - Android, iOS 和嵌入式
• Tensorflow.js - Javascript 部署
• 其他语言

Tensorflow vs. Pytorch

入门时间（易用性）

• Tensorflow 1.*
  • 静态图 ，构建完之后不可以更改, 效率高
  • 额外概念， 会话，变量，占位符
  • 写样本代码
• Tensorflow 2.0
  • 动态图， 构建完之后可以更改， 效率不高，调试容易
  • Eager mode 直接集成在python中
• Pytorch
  • 动态图
  • numpy扩展，集成在python

"""
不同方式求解 1 + 1/2 + 1/2^2 + 1/2^3 + ...... + 1/2^50
"""

# 1. tensorflow 1.*求解
import tensorflow as tf
print(tf.__version__)
x = tf.Variable(0.)
y = tf.Variable(1.)

add_op = x.assign(x + y)
div_op = y.assign(y / 2)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer()) 
    for iteration in range(50):
        sess.run(add_op)
        sess.run(div_op)
    print(x.eval())


# 2. pytorch 求解
import torch
print(torch.__version__)

x = torch.Tensor([0.])
y = torch.Tensor([1.])
for iteration in range(50):
    x = x + y
    y = y / 2
print(x)


# 3. tensorflow 2.0 求解
import tensorflow as tf
print(tf.__version__)
x = tf.constant(0.)
y = tf.constant(1.)
for iteration in range(50):
    x = x + y
    y = y / 2
print(x.numpy())


# 4. 纯python求解
x = 0
y = 1
for iteration in range(50):
    x = x + y
    y = y / 2
print(x)  # 精度有点不一样

图创建和调试

• Tensorflow 1.*
  • 静态图，难以调试， 需要使用tfdbg
• Tensorflow 2.0 与 pytorch
  • 动态图，python自带的调试工具

全面性

• python缺少少量的功能，使用频次很低
  • 沿维翻转张量 （np.flip, np.flipud, np.fliplr)
  • 检查无穷与非数值张量（np.is_nan, np.is_inf)
  • 快速傅里叶变换 （np.fft)

序列化和部署

• Tensorflow 支持更加广泛，多语言，跨平台
• pytorch 支持比较简单