Eager Execution

　　Eager Execution Tensorflow the programming environment is a command, the operation returns a specific value.

　　To start Eager Execution, please tf.enable_eager_execution () to the beginning of the program or answer the console.

　　Eager Execution will enable behavior change TensorFlow operation, and now they will immediately assess the value and return to the Python . tf.Tensor objects references a specific value, instead of a handle symbolic computation graph nodes. Since the calculation of FIG run later in the session need to build, so to print () or debugger easy to check the result. Assessment, inspection sheets and output values will not be interrupted gradient calculation process.

　　tf.Tensor numpy ndarray and can be interchangeable.

Dynamic flow control:

　　In the implementation of the model, all functions of the host language are available.

Construction of model

　　When used in conjunction with the TensorFlow Eager and Eager Execution, you can write their own layer or layer provided tf.keras.layers package.

　　Although any Python object presentation layer, but provides a convenient TensorFlow base class tf.karas.layers.Layer. You can implement your own layer through his:

class MysimpleLayer(tf.keras.layers.Layer):
    def __init__(self,output_units):
        super(MysimpleLayer,self).__init__()
        self.output_units=output_units
    def build(self,input_shape):
        #权重形状
        self.kernel=self.add_variable("kernel",[input_shape[-1],self.output_units])
    def call(self,input):
        #输出
        return tf.matmul(input,self.kernel)

　　Tf.keras.layers.Dense layer is preferably used (instead of the above MysimpleLayer), having a functional superset (which can add bias).

　　When the layer is combined into a model, the model may be used tf.keras.Sequential represent a linear stacked layers. He is ideal for the basic model:

= tf.keras.Sequential Model ([tf.keras.layer.Dense (10, = input_shape (784 ,)), 
tf.keras.layers.Dense ( 10)]) # must specify the shape of the first layer

　　Or, through inheritance tf.keras.Model model organized into categories. This layer is itself a container layer, allowing tf.keras.Model objects contain other objects tf.keras.Model.

class MniSTModel(tf.keras.Model):
    def __init__(self):
        super(MniSTModel,self).__init__()
        self.dese1=tf.keras.layers.Dense(units=10)
        self.dense2=tf.keras.layers.Dense(units=10)
    def call(self,input):
        result=self.dense1(input)
        result=self.dense2(result)
        result=self.dense2(result)#复用dense2权重
        return result

model=MniSTModel()

Gradient calculation

　　Automatic differentiation for the realization of machine learning algorithms (such as back-propagation neural network) is useful. During Eager Execution, using a tracking operation in order to calculate the gradient tf.GradientTape later.

　　tf.GradientTape is an optional feature that provides optimal performance when no tracking. Because of all the different operations may occur during each call, so all the previous records to "tape" in the propagation will be. To calculate the gradient, reverse playback tape, and then abandoned. Specific tf.gradientTape can only calculate a gradient; subsequent calls will throw a runtime error.

w=tf.Variable([[1.0]])
with tf.GradientTape() as tape:
    loss=w*w

grad=tape.gradient(loss,w)
print(grad)#=>tf.Tensor([[2.]],shape=(1,1),dtype=float32)

Trainer

　　The following example will create a multi-layer model that would classify an array of standard MNIST handwriting. It demonstrates the construction and optimization layer API trainable in FIG Eager Execution environment.

#下载mnist数据集
(mnist_images,mnist_labels),_=tf.keras.datasets.mnist.load_data()

dataset=tf.data.Dataset.from_tensor_silices((tf.cast(mnist_images[...,tf.newaxis]/255,tf.float32),tf.cast(mnist_labels,tf.int64)))
dataset=dataset.shuffle(1000).batch(32)

#构建模型
mnist_model=tf.keras.Sequential([tf.keras.layers.Conv2D(16,[3,3],activation='relu'),
    tf.keras.layers.Conv2D(16,[3,3],activation='relu'),
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(10)
])

　　Even without training, you can also call the model Eager Execution and check the output:

for images,labels in dataset.take(1):
    print('logits:",mnist_model(images[0:1]).numpy())

　　　Although keras model has a built-training cycle (using the fit method), but sometimes require more customized settings. The following is an example of the training cycle eager achieved with:

optimizer=tf.train.AdamOptimizer()
loss_history=[]

for (batch,(images,labels)) in enumerate(dataset.take(400)):
    if batch %80==0:
        print()
    print('.',end=' ')
    with tf.GradientTape() as tape:
        logits=mnist_model(images,training=True)
        loss_value=tf.losses.sparse_sorftmax_cross_entropy(labels,logits)
loss_history.append(loss_value.numpy())
grads=tape.gradient(loss_value,mnist_model.variables)
optimizer.apply_gradients(zip(grads,mnist_model.variables),global_step=tf.train.get_or_create_global_step())

import matplotlib.pyplot as plt
plt.plot(loss_history)
plt.xlabel('Batch #')
plt.ylabel('loss [entropy]')
Text(0,0.5,'loss [entropy]')

Variables and optimizer

　　tf.Variable objects stored in the variable tf.Tensor value during a training visit in order to more easily achieve automatic differentiation. Parameters of the model may be encapsulated as a variable in the class.

　　By using the model parameters can be better packaging and tf.GradientTape tf.Variable binding. For example, automatic differentiation above example can be rewritten as:

class Model(tf.keras.Model):
    def __init__(self):
        super(Model,self).__init__()
        self.W=tf.Variable(5.,name='weight')
        self.B=tf.Variable(10.,name='bias')
    def call(self,inputs):
        return inputs*self.W+self.B

#创建小型数据集
NUM_EXAMPLES=2000
training_inputs=tf.random_normal([NUM_EXAMPLES])
noise=tf.random_normal([NUM_EXAMPLES])
training_outputs=training_inputs*3+2+noise

#损失函数
def loss(model,inputs,targets):
    error=model(inputs)-targets
    return tf.reduce_mean(tf.square(error))

def grad(model,inputs,targets):
    with tf.GradientTape() as tape:
        loss_value=loss(model,inputs,targets)
    return tape.gradient(loss_value,[model.W,model.B])

#1.建立model实例  2.计算损失函数对W、B的微分  3.更新参数
model=Model()
optimizer=tf.train.GradientDescentOptimizer(learning_rate=0.01)

print('Initial loss:{:.3f}'.format(loss(model,training_inputs,training_outputs)))

#训练xunhuan
for i in range(300):
    grads=grad(model,training_inputs,training_outputs)
    optimizer.apply_gradients(zip(grads,[model.W,model.B]),global_step=tf.train.get_or_create_global_step())
    if i%20==0:
        print('Loss at step {:03d}:{:.3f}'.format(i,loss(model,training_inputs,training_outputs)))
print('Final loss:{:.3f}'.format(loss(model,training_inputs,training_outputs)))
print('W={},B={}'.format(model.W.numpy(),model.B.numpy()))