【TensoFlow】 专用名词与相关概念(Proper Noun & relative concepts)

Developer Guide

Using a TensorFlow Lite model in your mobile app requires multiple considerations: you must choose a pre-trained or custom model, convert the model to a TensorFLow Lite format, and finally, integrate the model in your app.

1. Choose a model

Depending on the use case, you can choose one of the popular open-sourced models, such as InceptionV3 or MobileNets, and re-train these models with a custom data set or even build your own custom model.

Use a pre-trained model

MobileNets is a family of mobile-first computer vision models for TensorFlow designed to effectively maximize accuracy, while taking into consideration the restricted resources for on-device or embedded applications. MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints for a variety of uses. They can be used for classification, detection, embeddings, and segmentation—similar to other popular large scale models, such asInception. Google provides 16 pre-trained ImageNet classification checkpoints for MobileNets that can be used in mobile projects of all sizes.

Inception-v3 is an image recognition model that achieves fairly high accuracy recognizing general objects with 1000 classes, for example, "Zebra", "Dalmatian", and "Dishwasher". The model extracts general features from input images using a convolutional neural network and classifies them based on those features with fully-connected and softmax layers.

On Device Smart Reply is an on-device model that provides one-touch replies for incoming text messages by suggesting contextually relevant messages. The model is built specifically for memory constrained devices, such as watches and phones, and has been successfully used in Smart Replies on Android Wear. Currently, this model is Android-specific.

These pre-trained models are available for download

Re-train Inception-V3 or MobileNet for a custom data set

These pre-trained models were trained on the ImageNet data set which contains 1000 predefined classes. If these classes are not sufficient for your use case, the model will need to be re-trained. This technique is called transfer learningand starts with a model that has been already trained on a problem, then retrains the model on a similar problem. Deep learning from scratch can take days, but transfer learning is fairly quick. In order to do this, you need to generate a custom data set labeled with the relevant classes.

The TensorFlow for Poets codelab walks through the re-training process step-by-step. The code supports both floating point and quantized inference.

Train a custom model

A developer may choose to train a custom model using Tensorflow (see the Tutorials for examples of building and training models). If you have already written a model, the first step is to export this to a tf.GraphDef file. This is required because some formats do not store the model structure outside the code, and we must communicate with other parts of the framework. See Exporting the Inference Graph to create .pb file for the custom model.

TensorFlow Lite currently supports a subset of TensorFlow operators. Refer to the TensorFlow Lite & TensorFlow Compatibility Guide for supported operators and their usage. This set of operators will continue to grow in future Tensorflow Lite releases.

2. Convert the model format

The model generated (or downloaded) in the previous step is a standard Tensorflow model and you should now have a .pb or .pbtxt tf.GraphDef file. Models generated with transfer learning (re-training) or custom models must be converted—but, we must first freeze the graph to convert the model to the Tensorflow Lite format. This process uses several model formats:

扫描二维码关注公众号，回复： 404481 查看本文章

• tf.GraphDef (.pb) —A protobuf that represents the TensorFlow training or computation graph. It contains operators, tensors, and variables definitions.（包括了运算、向量、变量定义。是图和数据 .pb=meta+data，将变量及其取值以常量constant的形式存储，同时去掉一些不必要的节点）
• CheckPoint (.ckpt) —Serialized variables from a TensorFlow graph. Since this does not contain a graph structure, it cannot be interpreted by itself.( 将 计算图结构meta 与 变量取值data 分开)
• FrozenGraphDef —A subclass of GraphDef that does not contain variables. A GraphDef can be converted to a FrozenGraphDef by taking a CheckPoint and a GraphDef, and converting each variable into a constant using the value retrieved from the CheckPoint.
• SavedModel —A GraphDef and CheckPoint with a signature that labels input and output arguments to a model. A GraphDef and CheckPoint can be extracted from a SavedModel.
• TensorFlow Lite model (.tflite) —A serialized FlatBuffer that contains TensorFlow Lite operators and tensors for the TensorFlow Lite interpreter, similar to a FrozenGraphDef.

To use the GraphDef .pb file with TensorFlow Lite, you must have checkpoints that contain trained weight parameters. The .pb file only contains the structure of the graph. The process of merging the checkpoint values with the graph structure is called freezing the graph.

tensorflow：what is the relationship between ckpt file and ckpt meta and ckpt index and .pb file

https://stackoverflow.com/questions/44516609/tensorflow-what-is-the-relationship-between-ckpt-file-and-ckpt-meta-and-ckp/44521818#44521818?newreg=dc8b4493f7484f0694cbeed604f654e8

• the .ckpt file is the old version output of saver.save(sess), which is the equivalent of your .ckpt-data (see below)

• the "checkpoint" file is only here to tell some TF functions which is the latest checkpoint file.

• .ckpt-meta contains the metagraph（元计算图）, i.e. the structure of your computation graph, without the values of the variables (basically what you can see in tensorboard/graph).

• .ckpt-data contains the values for all the variables, without the structure. To restore a model in python, you'll usually use the meta and data files with (but you can also use the .pb file):

  saver = tf.train.import_meta_graph(path_to_ckpt_meta)
  saver.restore(sess, path_to_ckpt_data)

• I don't know exactly for .ckpt-index, I guess it's some kind of index needed internally to map the two previous files correctly. Anyway it's not really necessary usually, you can restore a model with only .ckpt-meta and .ckpt-data.

• the .pb file can save your whole graph (meta + data). To load and use (but not train) a graph in c++ you'll usually use it, created with freeze_graph, which creates the .pb file from the meta and data. Be careful, (at least in previous TF versions and for some people) the py function provided by freeze_graph did not work properly, so you'd have to use the script version. Tensorflow also provides a tf.train.Saver.to_proto() method, but I don't know what it does exactly.

There are a lot of questions here about how to save and restore a graph. See the answer here for instance, but be careful that the two cited tutorials, though really helpful, are far from perfect, and a lot of people still seem to struggle to import a model in c++.

EDIT: it looks like you can also use the .ckpt files in c++ now, so I guess you don't necessarily need the .pb file any more.

Multiply-Accumulates (MACs) which measures the number of fused Multiplication and Addition operations. （融合乘加运算）

Graphvs Computional Graph

Graph or Computional Graph is the core concept of tensorflw to present compution. When you use tensorflow, you firstly create you own Computation Graph and pass the Graph to tensorflow. How to do that? As you may know, tensorflow support many front programming languages, like Python, C++, Java and Go and the core language is C++, how the other languages transform the Graph to C++? They use a tool called protobuf which can generate specific language stubs, that's where the GraphDef come from. GraphDef is a serialized version of Graph. 

which one should I have to run a graph loaded from protobuf file (.pb)

You should read you *pb file using GraphDef and bind the GraphDef to a (default) Graph, then use a session to run the Graph for computation, like the following code:

import tensorflow as tf
from tensorflow.python.platform import gfile
with tf.Session() as sess:
    model_filename ='PATH_TO_PB.pb'
    with gfile.FastGFile(model_filename, 'rb') as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        g_in = tf.import_graph_def(graph_def)
LOGDIR='/logs/tests/1/'
train_writer = tf.summary.FileWriter(LOGDIR)
train_writer.add_graph(sess.graph)