paddle (c)

First, the data source

InMemoryDataset,QueueDataset

Load data and buffer data before training. Such is created by DatasetFactory.

import paddle.fluid as fluid
dataset = fluid.DatasetFactory().create_dataset("InMemoryDataset")
filelist = ["a.txt", "b.txt"]
dataset.set_filelist(filelist)
dataset.load_into_memory()

View Code

PyReader

Create a reader object data input in python. The threads in python prefetched data, and inserting the asynchronous queue. When calling Executor.run, data queue will be automatically extracted.

feed_list (list (Variable) | tuple (Variable)) - feed variable list () created by fluid.layers.data.
capacity (int) - PyReader object maintains the internal size of the queue capacity. The unit is batch number. When the reader reads faster, set a large capacity value.
use_double_buffer (bool) - whether to use double_buffer_reader. If use_double_buffer = True, PyReader asynchronously to the next batch pre-read data, the speed data reading process, but will also take up a small amount of CPU / GPU memory, i.e., a batch input data storage space.
iterable (bool) - DataLoader the created object whether iteration.
return_list (bool) - whether or not each device on the data returned in list form. Valid only in iterable = True mode. If return_list = False, returns data on each device are str -> LoDTensor mapping table, key mapping table in which the name of each input variable. If return_list = True, then each device on the return data are list (LoDTensor). Recommended return_list = False under static map mode, return_list = True in the dynamic graphic mode.

import paddle
import paddle.fluid as fluid
import numpy as np

BATCH_SIZE = 10

def generator():
  for i in range(5):
     yield np.random.uniform(low=0, high=255, size=[784, 784]),

image = fluid.layers.data(name='image', shape=[784, 784], dtype='float32')
reader = fluid.io.PyReader(feed_list=[image], capacity=4, iterable=False)
reader.decorate_sample_list_generator(
  paddle.batch(generator, batch_size=BATCH_SIZE))

executor = fluid.Executor(fluid.CPUPlace())
executor.run(fluid.default_startup_program())
for i in range(3):
  reader.start()
  while True:
      try:
          executor.run(feed=None)
      except fluid.core.EOFException:
          reader.reset()
          break

View Code

Loading bypaddle.fluid.io下的buffered，cache，chain用于批量导入数据

Under paddle.fluid.io first, map_readers, xmap_readers for transformed data.

Second, the neural network parameters

A neural network initialization parameter fluid.ParamAttr

BilinearInitializer: This parameter initialization function interface for transposition function convolution, the sampled input. By any integer factor of the user is an enlarged shape (B, C, H, W ) in feature map.
ConstantInitializer: The interface is constant initialization function for the weights initialized by the value initializes the value of the input variable entered;
force_init_on_cpu:
init_on_cpu
NormalInitializer: random normal (Gaussian) distribution initialization function
NumpyArrayInitializer: the OP using Numpy array initialized parameter variables.
UniformInitializer uniformly distributed random initializer
XavierInitializer

import paddle.fluid as fluid
import math
factor = 2
C = 2
H = W = 32
w_attr = fluid.ParamAttr(
    learning_rate=0.,
    regularizer=fluid.regularizer.L2Decay(0.),
    initializer=fluid.initializer.BilinearInitializer())
x = fluid.layers.data(name="data", shape=[4, H, W],
                      dtype="float32")
conv_up = fluid.layers.conv2d_transpose(
    input=x,
    num_filters=C,
    output_size=None,
    filter_size=2 * factor - factor % 2,
    padding=int(math.ceil((factor - 1) / 2.)),
    stride=factor,
    groups=C,
    param_attr=w_attr,
    bias_attr=False)

View Code

Third, the scope

paddle.fluid.executor.scope_guard(scope)

The interface switching scope (scope) through the python with statement. Scope records the mapping relationship between variable name and variable (Variable), a programming language similar to braces. If you do not call this interface, all variables and variable names will be recorded in the default global scope. When a user needs to create a variable with the same name, if undesired parameter names mapping relationship is covered by the interface switching is required scope. After switching through with statement, with statement block all variables created will be assigned to the new scope.

global_scope: get global / default scope instance. Many API uses the default global_scope

import paddle.fluid as fluid
import numpy

new_scope = fluid.Scope()
with fluid.scope_guard(new_scope):
     fluid.global_scope().var("data").get_tensor().set(numpy.ones((1, 2)), fluid.CPUPlace())
data = numpy.array(new_scope.find_var("data").get_tensor())
print(data)

View Code

Fourth, the dynamic graph mechanism

PaddlePaddle is a dynamic model of DyGraph FIG execution mechanism, the results may be performed immediately, without having to build the entire FIG. At the same time, and in the past a static perform calculations picture above, under DyGraph mode all your operations can get immediate execution results without having to wait for computation graph constructed all executed, which would allow you to build deep learning tasks under PaddlePaddle more intuitive and debugging model, while also reducing the amount of code for building static calculation chart so that you write, the process of debugging the network becomes more convenient.

PaddlePaddle DyGraph is a more flexible and easy to use mode, provides:

More flexible and convenient code organization structure: using python models designed to perform control flow and object-oriented
easier debugging features: Direct method of printing using python instant print the desired results, so that the model results to check the running easy to test changes
and static executes generic model code: code can use the same model more convenient DyGraph debugging, execution, and also support the implementation of the use of the original still picture mode

fluid.dygraph include lower

Conv2D,Conv3D,Conv2DTranspose,Conv3DTranspose,FC,Pool2D

load_dygraph,save_graph

Layer object for storing a dynamic layer

import paddle.fluid as fluid
import numpy as np

with fluid.dygraph.guard():
    value = np.arange(26).reshape(2, 13).astype("float32")
    a = fluid.dygraph.to_variable(value)
    linear = fluid.Linear(13, 5, dtype="float32")
    adam = fluid.optimizer.Adam(learning_rate=0.01,
                                parameter_list=linear.parameters())
    out = linear(a)
    out.backward()
    adam.minimize(out)
    linear.clear_gradients()

import paddle.fluid as fluid
with fluid.dygraph.guard():
    emb = fluid.dygraph.Embedding([10, 10])
    state_dict = emb.state_dict()
    fluid.save_dygraph(state_dict, "paddle_dy")

View Code

create_parameter，full_name，sublayers，named_parameters，add_sublayer

Five models

glu

Linear gating means Gated Linear Units (GLU) from the split, sigmoid and elementwise_mul composition. Particular, along a given dimension the same size as the input split into two portions, $a$

G L U (a, b) = a ⨂ σ (b)

scaled_dot_product_attention

该接口实现了的基于点积（并进行了缩放）的多头注意力（Multi-Head Attention）机制。attention可以表述为将一个查询（query）和一组键值对（key-value pair）映射为一个输出；Multi-Head Attention则是使用多路进行attention，而且对attention的输入进行了线性变换。公式如下：

scaled_dot_product_attention

Guess you like