# Copy
tf.contrib.layers.embed_sequence
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/layers/embed_sequence
Description: embedding operations performed on the sequence data, input [batch_size, sequence_length] of Tensor, returned [batch_size, sequence_length, embed_dim] the tensor.
example:
features = [[1,2,3],[4,5,6]]
outputs = tf.contrib.layers.embed_sequence(features, vocab_size, embed_dim)
# If embed_dim = 4, the result is output
[
[[0.1,0.2,0.3,0.1],[0.2,0.5,0.7,0.2],[0.1,0.6,0.1,0.2]],
[[0.6,0.2,0.8,0.2],[0.5,0.6,0.9,0.2],[0.3,0.9,0.2,0.2]]
]
tf.strided_slice
Links: https://www.tensorflow.org/api_docs/python/tf/strided_slice
Description: Performs the slicing operation on incoming tensor, tensor return after slicing. The main parameters input_, start, end, strides, strides on behalf of slicing step.
example:
# 'input' is [[[1, 1, 1], [2, 2, 2]],
# [[3, 3, 3], [4, 4, 4]],
# [[5, 5, 5], [6, 6, 6]]]
tf.strided_slice(input, [1, 0, 0], [2, 1, 3], [1, 1, 1]) ==> [[[3, 3, 3]]]
# Above line of code [1,0,0] represent the three dimensions of the original array slice start position, [2,1,3] represents the end position.
[1,1,1] Representative step sections, represented in three dimensions are 1 sections step. Our raw input data into 3 x 2 x 3,
We obtained parameter, a dimension on the first slice start = 1, end = 2,
The second dimension start = 0, end = 1, a third dimension start = 0, end = 3.
Our view from the inside dimension, third dimension has three elements of the original data, slicing start = 0, end = 3, stride = 1, the representative elements of the third dimension to retain all of us.
Similarly, in the second dimension, start = 0, end = 1, stride = 1, leaving only the representative of a first dimension on the second slice, so we only [[[1,1,1] ], [[3,3,3]], [[5,5,5]]].
Then we look at the first dimension, start = 1, end = 2, stride = 1 take only representative of the second slice, so to give [[[3,3,3]]. The following two examples empathy.
tf.strided_slice(input, [1, 0, 0], [2, 2, 3], [1, 1, 1])
==> [[[3, 3, 3],
[4, 4, 4]]]
tf.strided_slice(input, [1, -1, 0], [2, -3, 3], [1, -1, 1])
==>[[[4, 4, 4],
[3, 3, 3]]]
tf.contrib.rnn.MultiRNNCell
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/MultiRNNCell
Description: RNN unit for stacking in sequence. It accepts a parameter is composed of RNN cell list.
example:
# Rnn_size representative of a number of cells in the number of hidden nodes, layer_nums representative of stacked rnn cell rnn
lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)
composed_cell = tf.contrib.rnn.MultiRNNCell([lstm for _ in range(num_layers)])
# Above such an approach can be run in tensorflow1.0 but in tensorflow1.1 version, the above configuration does not allow lstm multiplexing unit to regenerate a new object, so in the source code, the nested function definition of a function cell, so as to ensure that every time a new object instance.
def get_lstm(rnn_size):
lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)
return lstm
composed_cell = tf.contrib.rnn.MultiRNNCell([get_lstm(rnn_size) for _ in range(num_layers)])
tf.nn.dynamic_rnn
Links: https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn
Description: Construction RNN, accept dynamic input sequence. Tensor RNN return output and a final state. Rnn in that the difference dynamic_rnn, dynamic_rnn for different batch, may receive different sequence_length, for example, a first batch is [batch_size, 10], the second batch is [batch_size, 20]. The rnn only receive fixed-length sequence_length.
example:
output, state = tf.nn.dynamic_rnn(cell, inputs)
tf.tile
Links: https://www.tensorflow.org/api_docs/python/tf/tile
Description: The tensor input may be reproduced, copied after the return tensor. The main parameters are input and multiples.
example:
# Fake code
input = [a, b, c, d]
output = tf.tile(input, 2)
# output = [a, b, c, d, a, b, c, d]
input = [[1,2,3], [4,5,6]]
output = tf.tile(input, [2, 3])
# output = [[1,2,3,1,2,3,1,2,3],
[4,5,6,4,5,6,4,5,6],
[1,2,3,1,2,3,1,2,3],
[4,5,6,4,5,6,4,5,6]]
tf.fill
Links: https://www.tensorflow.org/api_docs/python/tf/fill
Description: main parameters and dims value, constructed by filling a value of shape Tensor dims.
example:
tf.fill([2,3],9) => [[9,9,9],[9,9,9]]
tf.contrib.seq2seq.TrainingHelper
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/TrainingHelper
Description: Decoder end of training to function. This function does not output stage t-1 t as input stages, but the true value of the target is directly input to the RNN. The main parameters are inputs and sequence_length. Back helper objects, as a function of the parameter BasicDecoder.
example:
training_helper = tf.contrib.seq2seq.TrainingHelper(inputs=decoder_embed_input,
sequence_length=target_sequence_length,
time_major=False)
tf.contrib.seq2seq.BasicDecoder
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/BasicDecoder
Description: generating a basic target decoder
example:
# Cell layer is RNN, training_helper TrainingHelper generated by objects,
encoder_state tensor RNN the initial state,
output_layer represents the output layer, it is an object tf.layers.Layer.
training_decoder = tf.contrib.seq2seq.BasicDecoder(cell,
training_helper,
encoder_state,
output_layer)
tf.contrib.seq2seq.dynamic_decode
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/dynamic_decode
Description: Performs dynamic decoding of the decoder. The maximum sequence length is defined by maximum_iterations parameters.
tf.contrib.seq2seq.GreedyEmbeddingHelper
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/GreedyEmbeddingHelper
Description: It TrainingHelper difference is that it will be before embedding t-1 input to output in RNN.
tf.sequence_mask
Links: https://www.tensorflow.org/api_docs/python/tf/sequence_mask
Description: The tensor conducted mask, returns True and False composed of tensor
example:
# Fake code
tf.sequence_mask([1,3,2],5) =>
[[True, False, False, False, False],
[True, True, True, False, False],
[True, True, False, False, False]]
# Where dtype default is tf.bool, use tf.float32 in our code, which is calculated for the later generation weight loss.
tf.contrib.seq2seq.sequence_loss
Links: https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/sequence_loss
Description: computing a weighted cross entropy sequence logits.
example:
# Training_logits is the result of the output layer, targets a target, masks we use tf.sequence_mask result of the calculation, where as the weight, that is to say we will not <PAD> into the calculation when calculating the cross entropy.
cost = tf.contrib.seq2seq.sequence_loss(
training_logits,
targets,
masks)