目录
Building Graphs 构建TensorFlow的数据流图.......................................... 21
Core graph data structures 核心图数据库 ...................................... 22
class tf.Graph ..................................................................................................... 22
class tf.Operation .............................................................................................. 39 class
tf.Tensor ..................................................................................................... 45
Tensor types ........................................................................................................ 51 class
tf.DType ....................................................................................................... 51
tf.as_dtype(type_value) .................................................................................. 56
Utility functions 实用函数.......................................................................................................... 56
tf.device(dev) ....................................................................................................... 56 tf.name_scope(name)............................................................................................ 57
tf.control_dependencies(control_inputs) ............................................ 58
tf.convert_to_tensor(value, dtype=None, name=None, as_ref=False)
tf.convert_to_tensor_or_indexed_slices(value, dtype=None, name=None, as_ref=False)
tf.get_default_graph() .................................................................................... 60 tf.reset_default_graph() ................................................................................ 61
tf.import_graph_def(graph_def, input_map=None,
return_elements=None, name=None, op_dict=None) ........................... 61 tf.load_op_library(library_filename) ................................................... 63
Graph collections .............................................................................................................. 64
tf.add_to_collection(name, value) .......................................................... 64
tf.get_collection(key, scope=None) ........................................................ 64 class
tf.GraphKeys .............................................................................................. 65
Defining new operations .................................................................................................. 66 class tf.RegisterGradient ............................................................................. 66
tf.NoGradient(op_type) .................................................................................... 67 class
tf.RegisterShape .................................................................................... 68 class
tf.TensorShape ......................................................................................... 69 class
tf.Dimension .............................................................................................. 77
tf.op_scope(values, name, default_name=None) ................................ 79
tf.get_seed(op_seed) ......................................................................................... 80
For libraries building on TensorFlow .............................................................................. 81
tf.register_tensor_conversion_function(base_type,
conversion_func, priority=100) ................................................................. 81
Other Functions and Classes.......................................................................................... 82 class
tf.bytes ....................................................................................................... 82
Constants, Sequences, and Random Values .................................................................. 82
Constant Value Tensors .................................................................................................. 83
tf.zeros(shape, dtype=tf.float32, name=None) ................................ 83 tf.zeros_like(tensor, dtype=None, name=None) ................................ 84 tf.ones(shape, dtype=tf.float32, name=None) .................................. 85 tf.ones_like(tensor, dtype=None, name=None) .................................. 86 tf.fill(dims, value, name=None) ............................................................... 87
tf.constant(value, dtype=None, shape=None, name='Const') ... 87
Sequences ......................................................................................................................... 89
tf.linspace(start, stop, num, name=None) ......................................... 89
tf.range(start, limit=None, delta=1, name='range') ................. 90
Random Tensors .............................................................................................................. 91 Examples: ...................................................................................................................... 91
tf.random_normal(shape, mean=0.0, stddev=1.0,
dtype=tf.float32, seed=None, name=None) ............................................ 92
tf.truncated_normal(shape, mean=0.0, stddev=1.0,
dtype=tf.float32, seed=None, name=None) ............................................ 93
tf.random_uniform(shape, minval=0, maxval=None,
dtype=tf.float32, seed=None, name=None) ............................................ 94 tf.random_shuffle(value, seed=None, name=None) ........................... 95 tf.random_crop(value, size, seed=None, name=None) .................... 96 tf.set_random_seed(seed) ................................................................................ 97
Variables ................................................................................................................................ 99
Variables .......................................................................................................................... 100 class tf.Variable .............................................................................................. 100 Variable helper functions ............................................................................................... 111 tf.all_variables() ............................................................................................ 112 tf.trainable_variables() .............................................................................. 112 tf.moving_average_variables() .................................................................. 113 tf.initialize_all_variables() .................................................................. 113
tf.initialize_variables(var_list, name='init') ......................... 113 tf.assert_variables_initialized(var_list=None) ......................... 114
Saving and Restoring Variables ................................................................................... 115 class tf.train.Saver ....................................................................................... 115
tf.train.latest_checkpoint(checkpoint_dir,
latest_filename=None) ..................................................................................... 123
tf.train.get_checkpoint_state(checkpoint_dir,
latest_filename=None) ..................................................................................... 124
tf.train.update_checkpoint_state(save_dir, model_checkpoint_path, all_model_checkpoint_paths=None,
latest_filename=None) ..................................................................................... 125
Sharing Variables ........................................................................................................... 125
tf.get_variable(name, shape=None, dtype=tf.float32, initializer=None, trainable=True, collections=None) ............. 125 tf.get_variable_scope() ................................................................................ 127
tf.make_template(name_, func_, **kwargs) ....................................... 127
tf.variable_op_scope(values, name, default_name,
initializer=None) .............................................................................................. 130
tf.variable_scope(name_or_scope, reuse=None,
initializer=None) .............................................................................................. 131
tf.constant_initializer(value=0.0, dtype=tf.float32) ........... 133 tf.random_normal_initializer(mean=0.0, stddev=1.0, seed=None, dtype=tf.float32) .................................................................... 133
tf.truncated_normal_initializer(mean=0.0, stddev=1.0,
seed=None, dtype=tf.float32) .................................................................... 134
tf.random_uniform_initializer(minval=0.0, maxval=1.0,
seed=None, dtype=tf.float32) .................................................................... 135
tf.uniform_unit_scaling_initializer(factor=1.0, seed=None,
dtype=tf.float32) .............................................................................................. 136
tf.zeros_initializer(shape, dtype=tf.float32) ........................... 137
Sparse Variable Updates ............................................................................................... 137
tf.scatter_update(ref, indices, updates, use_locking=None,
name=None) ............................................................................................................... 138
tf.scatter_add(ref, indices, updates, use_locking=None,
name=None) ............................................................................................................... 140
tf.scatter_sub(ref, indices, updates, use_locking=None,
name=None) ............................................................................................................... 142
tf.sparse_mask(a, mask_indices, name=None)................................... 144 class tf.IndexedSlices .................................................................................. 145
Tensor Transformations ..................................................................................................... 148
Casting ............................................................................................................................. 149
tf.string_to_number(string_tensor, out_type=None, name=None) ................................... 149
tf.to_double(x, name='ToDouble') .......................................................... 150
tf.to_float(x, name='ToFloat') ............................................................... 150
tf.to_bfloat16(x, name='ToBFloat16') ................................................. 151
tf.to_int32(x, name='ToInt32') ............................................................... 152
tf.to_int64(x, name='ToInt64') ............................................................... 152
tf.cast(x, dtype, name=None) .................................................................... 153
Shapes and Shaping ...................................................................................................... 154
tf.shape(input, name=None) ......................................................................... 154
tf.size(input, name=None) ........................................................................... 155
tf.rank(input, name=None) ........................................................................... 155
tf.reshape(tensor, shape, name=None) ................................................. 156
tf.squeeze(input, squeeze_dims=None, name=None) ....................... 158
tf.expand_dims(input, dim, name=None) .............................................. 159
Slicing and Joining .......................................................................................................... 160
tf.slice(input_, begin, size, name=None) ....................................... 160
tf.split(split_dim, num_split, value, name='split') ............. 162
tf.tile(input, multiples, name=None) ................................................. 163
tf.pad(input, paddings, name=None) ...................................................... 163
tf.concat(concat_dim, values, name='concat') .............................. 165
tf.pack(values, name='pack') .................................................................... 166
tf.unpack(value, num=None, name='unpack') ..................................... 167
tf.reverse_sequence(input, seq_lengths, seq_dim,
batch_dim=None, name=None) ......................................................................... 168
tf.reverse(tensor, dims, name=None) ................................................... 170
tf.transpose(a, perm=None, name='transpose') .............................. 171
tf.space_to_depth(input, block_size, name=None) ....................... 173
tf.depth_to_space(input, block_size, name=None) ....................... 175
tf.gather(params, indices, validate_indices=None,
name=None) ............................................................................................................... 177
tf.dynamic_partition(data, partitions, num_partitions,
name=None) ............................................................................................................... 178
tf.dynamic_stitch(indices, data, name=None) ................................ 181
tf.boolean_mask(tensor, mask, name='boolean_mask') ............... 182
Other Functions and Classes........................................................................................ 184
tf.shape_n(input, name=None) .................................................................... 184
tf.unique_with_counts(x, name=None) ................................................... 184
Math ...................................................................................................................................... 185
Arithmetic Operators ...................................................................................................... 188
tf.add(x, y, name=None) ................................................................................ 188
tf.sub(x, y, name=None) ................................................................................ 189
tf.mul(x, y, name=None) ................................................................................ 190
tf.div(x, y, name=None) ................................................................................ 190
tf.truediv(x, y, name=None) ...................................................................... 191
tf.floordiv(x, y, name=None) .................................................................... 192
tf.mod(x, y, name=None) ................................................................................ 193
tf.cross(a, b, name=None) ........................................................................... 193
Basic Math Functions ..................................................................................................... 194
tf.add_n(inputs, name=None) ...................................................................... 194
tf.abs(x, name=None) ....................................................................................... 195
tf.neg(x, name=None) ....................................................................................... 196
tf.sign(x, name=None) ..................................................................................... 196 tf.inv(x, name=None) ....................................................................................... 197 tf.square(x, name=None) ................................................................................ 197 tf.round(x, name=None) .................................................................................. 198 tf.sqrt(x, name=None) ..................................................................................... 199 tf.rsqrt(x, name=None) .................................................................................. 199 tf.pow(x, y, name=None) ................................................................................ 200 tf.exp(x, name=None) ....................................................................................... 201 tf.log(x, name=None) ....................................................................................... 201 tf.ceil(x, name=None) ..................................................................................... 202 tf.floor(x, name=None) .................................................................................. 202 tf.maximum(x, y, name=None) ...................................................................... 203 tf.minimum(x, y, name=None) ...................................................................... 203 tf.cos(x, name=None) ....................................................................................... 204 tf.sin(x, name=None) ....................................................................................... 204 tf.lgamma(x, name=None) ................................................................................ 205 tf.erf(x, name=None) ....................................................................................... 206
tf.erfc(x, name=None) ..................................................................................... 206
Matrix Math Functions .................................................................................................... 207 tf.diag(diagonal, name=None) .................................................................... 207
tf.transpose(a, perm=None, name='transpose') .............................. 208
tf.matmul(a, b, transpose_a=False, transpose_b=False,
a_is_sparse=False, b_is_sparse=False, name=None) .................... 209
tf.batch_matmul(x, y, adj_x=None, adj_y=None, name=None) . 210 tf.matrix_determinant(input, name=None) .......................................... 212 tf.batch_matrix_determinant(input, name=None) ........................... 212 tf.matrix_inverse(input, name=None) ................................................... 213 tf.batch_matrix_inverse(input, name=None) ..................................... 214 tf.cholesky(input, name=None) .................................................................. 214 tf.batch_cholesky(input, name=None) ................................................... 215 tf.self_adjoint_eig(input, name=None) .............................................. 216 tf.batch_self_adjoint_eig(input, name=None) ................................ 217 tf.matrix_solve(matrix, rhs, name=None) .......................................... 217 tf.batch_matrix_solve(matrix, rhs, name=None) ........................... 218 tf.matrix_triangular_solve(matrix, rhs, lower=None,
name=None) ............................................................................................................... 219
tf.batch_matrix_triangular_solve(matrix, rhs, lower=None,
name=None) ............................................................................................................... 220
tf.matrix_solve_ls(matrix, rhs, l2_regularizer=0.0,
fast=True, name=None) ..................................................................................... 221
tf.batch_matrix_solve_ls(matrix, rhs, l2_regularizer=0.0,
fast=True, name=None) ..................................................................................... 223
Complex Number Functions.......................................................................................... 225
tf.complex(real, imag, name=None) ........................................................ 225
tf.complex_abs(x, name=None) .................................................................... 226
tf.conj(in_, name=None) ................................................................................ 226
tf.imag(in_, name=None) ................................................................................ 227
tf.real(in_, name=None) ................................................................................ 228
tf.fft2d(in_, name=None) .............................................................................. 229
tf.ifft2d(in_, name=None) ........................................................................... 229
Reduction ......................................................................................................................... 230
tf.reduce_sum(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 230
tf.reduce_prod(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 231
tf.reduce_min(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 232
tf.reduce_max(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 233
tf.reduce_mean(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 234
tf.reduce_all(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 235
tf.reduce_any(input_tensor, reduction_indices=None,
keep_dims=False, name=None) ...................................................................... 236
tf.accumulate_n(inputs, shape=None, tensor_dtype=None,
name=None) ............................................................................................................... 237
Segmentation .................................................................................................................. 238
tf.segment_sum(data, segment_ids, name=None) .............................. 239
tf.segment_prod(data, segment_ids, name=None) ........................... 240
tf.segment_min(data, segment_ids, name=None) .............................. 242
tf.segment_max(data, segment_ids, name=None) .............................. 243
tf.segment_mean(data, segment_ids, name=None) ........................... 245
tf.unsorted_segment_sum(data, segment_ids, num_segments,
name=None) ............................................................................................................... 246
tf.sparse_segment_sum(data, indices, segment_ids,
name=None) ............................................................................................................... 248
tf.sparse_segment_mean(data, indices, segment_ids,
name=None) ............................................................................................................... 249
tf.sparse_segment_sqrt_n(data, indices, segment_ids,
name=None) ............................................................................................................... 250
Sequence Comparison and Indexing .......................................................................... 251
tf.argmin(input, dimension, name=None) ............................................ 251
tf.argmax(input, dimension, name=None) ............................................ 252
tf.listdiff(x, y, name=None) .................................................................... 253
tf.where(input, name=None) ......................................................................... 254
tf.unique(x, name=None) ................................................................................ 255
tf.edit_distance(hypothesis, truth, normalize=True,
name='edit_distance') ..................................................................................... 256
tf.invert_permutation(x, name=None) ................................................... 258
Other Functions and Classes........................................................................................ 259
tf.scalar_mul(scalar, x) .............................................................................. 259
tf.sparse_segment_sqrt_n_grad(grad, indices, segment_ids,
output_dim0, name=None) ................................................................................ 259
Control Flow......................................................................................................................... 260
Control Flow Operations ................................................................................................ 261
tf.identity(input, name=None) .................................................................. 261
tf.tuple(tensors, name=None, control_inputs=None) .................. 262
tf.group(*inputs, **kwargs) ...................................................................... 263 tf.no_op(name=None).......................................................................................... 264
tf.count_up_to(ref, limit, name=None) .............................................. 264
tf.cond(pred, fn1, fn2, name=None) ...................................................... 265
Logical Operators ........................................................................................................... 266
tf.logical_and(x, y, name=None) ............................................................. 266
tf.logical_not(x, name=None) .................................................................... 267
tf.logical_or(x, y, name=None) ............................................................... 267
tf.logical_xor(x, y, name='LogicalXor') .......................................... 268
Comparison Operators ................................................................................................... 268
tf.equal(x, y, name=None) ........................................................................... 268
tf.not_equal(x, y, name=None) .................................................................. 269
tf.less(x, y, name=None) .............................................................................. 269
tf.less_equal(x, y, name=None) ............................................................... 270
tf.greater(x, y, name=None) ...................................................................... 271
tf.greater_equal(x, y, name=None) ........................................................ 271
tf.select(condition, t, e, name=None) .............................................. 272
tf.where(input, name=None) ......................................................................... 273
Debugging Operations ................................................................................................... 274
tf.is_finite(x, name=None) ......................................................................... 275
tf.is_inf(x, name=None) ................................................................................ 275
tf.is_nan(x, name=None) ................................................................................ 276
tf.verify_tensor_all_finite(t, msg, name=None) .........................
tf.check_numerics(tensor, message, name=None) ........................... 277 tf.add_check_numerics_ops() ...................................................................... 277
tf.Assert(condition, data, summarize=None, name=None) ........ 278
tf.Print(input_, data, message=None, first_n=None,
summarize=None, name=None) ......................................................................... 278
Images .................................................................................................................................. 279
Encoding and Decoding ................................................................................................. 281
tf.image.decode_jpeg(contents, channels=None, ratio=None, fancy_upscaling=None, try_recover_truncated=None,
acceptable_fraction=None, name=None) ................................................. 282
tf.image.encode_jpeg(image, format=None, quality=None, progressive=None, optimize_size=None, chroma_downsampling=None, density_unit=None, x_density=None, y_density=None, xmp_metadata=None,
name=None) ............................................................................................................... 283
tf.image.decode_png(contents, channels=None, dtype=None,
name=None) ............................................................................................................... 285
tf.image.encode_png(image, compression=None, name=None) ... 286
Resizing ............................................................................................................................ 287
tf.image.resize_images(images, new_height, new_width,
method=0, align_corners=False) ............................................................... 287
tf.image.resize_area(images, size, align_corners=None,
name=None) ............................................................................................................... 289
tf.image.resize_bicubic(images, size, align_corners=None,
name=None) ............................................................................................................... 290
tf.image.resize_bilinear(images, size, align_corners=None,
name=None) ............................................................................................................... 291
tf.image.resize_nearest_neighbor(images, size,
align_corners=None, name=None) ............................................................... 291
Cropping ........................................................................................................................... 292
tf.image.resize_image_with_crop_or_pad(image,
target_height, target_width) .................................................................... 292
tf.image.pad_to_bounding_box(image, offset_height,
offset_width, target_height, target_width)................................... 293
tf.image.crop_to_bounding_box(image, offset_height,
offset_width, target_height, target_width)................................... 294
tf.image.extract_glimpse(input, size, offsets, centered=None, normalized=None, uniform_noise=None,
name=None) ............................................................................................................... 296
Flipping and Transposing .............................................................................................. 297 tf.image.flip_up_down(image) .................................................................... 297
tf.image.random_flip_up_down(image, seed=None) ......................... 298 tf.image.flip_left_right(image) ............................................................. 299
tf.image.random_flip_left_right(image, seed=None) .................. 299 tf.image.transpose_image(image) ............................................................. 300
Converting Between Colorspaces. ............................................................................... 301 tf.image.rgb_to_grayscale(images) ........................................................ 302 tf.image.grayscale_to_rgb(images) ........................................................ 302
tf.image.hsv_to_rgb(images, name=None) ............................................ 303
tf.image.rgb_to_hsv(images, name=None) ............................................ 303
tf.image.convert_image_dtype(image, dtype, saturate=False,
name=None) ............................................................................................................... 304
Image Adjustments ......................................................................................................... 305
tf.image.adjust_brightness(image, delta) ....................................... 306
tf.image.random_brightness(image, max_delta, seed=None) ... 306
tf.image.adjust_contrast(images, contrast_factor) .................. 307
tf.image.random_contrast(image, lower, upper, seed=None) . 308
tf.image.adjust_hue(image, delta, name=None) .............................. 309
tf.image.random_hue(image, max_delta, seed=None) .................... 310
tf.image.adjust_saturation(image, saturation_factor, name=None)........................... 311
tf.image.random_saturation(image, lower, upper, seed=None)...... 312
tf.image.per_image_whitening(image) ................................................... 313
Working with Bounding Boxes ...................................................................................... 313
tf.image.draw_bounding_boxes(images, boxes, name=None) ...... 314 tf.image.sample_distorted_bounding_box(image_size, bounding_boxes, seed=None, seed2=None, min_object_covered=None, aspect_ratio_range=None, area_range=None, max_attempts=None,
use_image_if_no_bounding_boxes=None, name=None) ....................... 315
Other Functions and Classes........................................................................................ 317
tf.image.saturate_cast(image, dtype) ................................................. 318
Sparse Tensors ................................................................................................................... 318
Sparse Tensor Representation ..................................................................................... 319 class tf.SparseTensor ..................................................................................... 319
class tf.SparseTensorValue ......................................................................... 322
Sparse to Dense Conversion ........................................................................................ 323
tf.sparse_to_dense(sparse_indices, output_shape, sparse_values, default_value=0, validate_indices=True,
name=None) ............................................................................................................... 323
tf.sparse_tensor_to_dense(sp_input, default_value=0, validate_indices=True, name=None) ........................................................ 324
tf.sparse_to_indicator(sp_input, vocab_size, name=None) ... 326
Manipulation .................................................................................................................... 327
tf.sparse_concat(concat_dim, sp_inputs, name=None) ............... 327
tf.sparse_reorder(sp_input, name=None) ............................................ 329
tf.sparse_split(split_dim, num_split, sp_input, name=None).... 330
tf.sparse_retain(sp_input, to_retain) .............................................. 331
tf.sparse_fill_empty_rows(sp_input, default_value,
name=None) ............................................................................................................... 332
Inputs and Readers ............................................................................................................ 334
Placeholders .................................................................................................................... 335
tf.placeholder(dtype, shape=None, name=None) .............................. 335
Readers ............................................................................................................................ 336 class tf.ReaderBase.......................................................................................... 336 class tf.TextLineReader ................................................................................ 341 class tf.WholeFileReader .............................................................................. 345 class tf.IdentityReader ................................................................................ 349 class tf.TFRecordReader ................................................................................ 353 class tf.FixedLengthRecordReader .......................................................... 357
Converting ........................................................................................................................ 361
tf.decode_csv(records, record_defaults, field_delim=None,
name=None) ............................................................................................................... 361
tf.decode_raw(bytes, out_type, little_endian=None,
name=None) ............................................................................................................... 362
Example protocol buffer ............................................................................................. 363 class tf.VarLenFeature .................................................................................. 363 class tf.FixedLenFeature .............................................................................. 363 class tf.FixedLenSequenceFeature .......................................................... 364
tf.parse_example(serialized, features, name=None,
example_names=None).......................................................................................... 365
tf.parse_single_example(serialized, features, name=None,
example_names=None).......................................................................................... 369
tf.decode_json_example(json_examples, name=None) .................... 370
Queues ............................................................................................................................. 371 class tf.QueueBase ............................................................................................ 371 class tf.FIFOQueue ............................................................................................ 377
class tf.RandomShuffleQueue ...................................................................... 378
Dealing with the filesystem ............................................................................................ 380
tf.matching_files(pattern, name=None) .............................................. 380
tf.read_file(filename, name=None) ........................................................ 380
Input pipeline ................................................................................................................... 381
Beginning of an input pipeline ................................................................................... 381
tf.train.match_filenames_once(pattern, name=None) .................. 381
tf.train.limit_epochs(tensor, num_epochs=None, name=None)382
tf.train.range_input_producer(limit, num_epochs=None,
shuffle=True, seed=None, capacity=32, name=None) .................... 383
tf.train.slice_input_producer(tensor_list, num_epochs=None,
shuffle=True, seed=None, capacity=32, name=None) .................... 383
tf.train.string_input_producer(string_tensor, num_epochs=None, shuffle=True, seed=None, capacity=32,
name=None) ............................................................................................................... 385
Batching at the end of an input pipeline .................................................................. 386
tf.train.batch(tensor_list, batch_size, num_threads=1, capacity=32, enqueue_many=False, shapes=None, name=None) . 386
tf.train.batch_join(tensor_list_list, batch_size, capacity=32, enqueue_many=False, shapes=None, name=None) . 388 tf.train.shuffle_batch(tensor_list, batch_size, capacity, min_after_dequeue, num_threads=1, seed=None,
enqueue_many=False, shapes=None, name=None) ................................ 390
tf.train.shuffle_batch_join(tensor_list_list, batch_size, capacity, min_after_dequeue, seed=None, enqueue_many=False, shapes=None, name=None) ................................................................................ 393
Data IO (Python functions) ................................................................................................ 395
Data IO (Python Functions) ........................................................................................... 395 class tf.python_io.TFRecordWriter ........................................................ 395 tf.python_io.tf_record_iterator(path) .............................................. 396
TFRecords Format Details ........................................................................................ 397
Neural Network ................................................................................................................... 397
Activation Functions ....................................................................................................... 399
tf.nn.relu(features, name=None) ............................................................. 400
tf.nn.relu6(features, name=None) .......................................................... 400
tf.nn.elu(features, name=None) ............................................................... 401
tf.nn.softplus(features, name=None) ................................................... 401
tf.nn.softsign(features, name=None) ................................................... 402
tf.nn.dropout(x, keep_prob, noise_shape=None, seed=None,
name=None) ............................................................................................................... 402
tf.nn.bias_add(value, bias, name=None) ............................................ 404
tf.sigmoid(x, name=None) .............................................................................. 404
tf.tanh(x, name=None) ..................................................................................... 405
Convolution ...................................................................................................................... 406
tf.nn.conv2d(input, filter, strides, padding,
use_cudnn_on_gpu=None, name=None) ........................................................ 408
tf.nn.depthwise_conv2d(input, filter, strides, padding,
name=None) ............................................................................................................... 409
tf.nn.separable_conv2d(input, depthwise_filter,
pointwise_filter, strides, padding, name=None) ......................... 411
tf.nn.conv2d_transpose(value, filter, output_shape,
strides, padding='SAME', name=None) ................................................... 412
Pooling .............................................................................................................................. 413
tf.nn.avg_pool(value, ksize, strides, padding, name=None)414
tf.nn.max_pool(value, ksize, strides, padding, name=None)415
tf.nn.max_pool_with_argmax(input, ksize, strides, padding,
Targmax=None, name=None) .............................................................................. 415
Normalization ................................................................................................................... 417
tf.nn.l2_normalize(x, dim, epsilon=1e-12, name=None) ........... 417
tf.nn.local_response_normalization(input, depth_radius=None, bias=None, alpha=None, beta=None,
name=None) ............................................................................................................... 418
tf.nn.moments(x, axes, name=None, keep_dims=False) ............... 419
Losses .............................................................................................................................. 419
tf.nn.l2_loss(t, name=None) ...................................................................... 420
Classification ................................................................................................................... 420
tf.nn.sigmoid_cross_entropy_with_logits(logits, targets,
name=None) ............................................................................................................... 420
tf.nn.softmax(logits, name=None) .......................................................... 421
tf.nn.softmax_cross_entropy_with_logits(logits, labels,
name=None) ............................................................................................................... 422
tf.nn.sparse_softmax_cross_entropy_with_logits(logits,
labels, name=None) ............................................................................................ 423
Embeddings ..................................................................................................................... 424
tf.nn.embedding_lookup(params, ids,
partition_strategy='mod', name=None, validate_indices=True)
....................................................................................................................................... 425
Evaluation ........................................................................................................................ 426
tf.nn.top_k(input, k=1, sorted=True, name=None) ....................... 427
tf.nn.in_top_k(predictions, targets, k, name=None) ............... 427
Candidate Sampling ....................................................................................................... 429
Sampled Loss Functions ........................................................................................... 429
tf.nn.nce_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=False, partition_strategy='mod',
name='nce_loss') ................................................................................................. 429
tf.nn.sampled_softmax_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, partition_strategy='mod',
name='sampled_softmax_loss') .................................................................... 431
Candidate Samplers ................................................................................................... 433
tf.nn.uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None) ........ 433 tf.nn.log_uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None) ........ 435 tf.nn.learned_unigram_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None,
name=None) ............................................................................................................... 437
tf.nn.fixed_unigram_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, vocab_file='', distortion=1.0, num_reserved_ids=0, num_shards=1, shard=0,
unigrams=(), seed=None, name=None) ...................................................... 438
Miscellaneous candidate sampling utilities ............................................................. 441
tf.nn.compute_accidental_hits(true_classes, sampled_candidates, num_true, seed=None, name=None) ............. 441
Running Graphs .................................................................................................................. 443
Session management .................................................................................................... 443 class tf.Session ................................................................................................. 444
class tf.InteractiveSession ...................................................................... 449
tf.get_default_session() .............................................................................. 451
Error classes .................................................................................................................... 451 class tf.OpError ................................................................................................. 451
class tf.errors.CancelledError ............................................................... 453 class tf.errors.UnknownError .................................................................... 453 class tf.errors.InvalidArgumentError ................................................. 454 class tf.errors.DeadlineExceededError .............................................. 455 class tf.errors.NotFoundError .................................................................. 455 class tf.errors.AlreadyExistsError ...................................................... 455 class tf.errors.PermissionDeniedError .............................................. 456 class tf.errors.UnauthenticatedError ................................................. 456 class tf.errors.ResourceExhaustedError ............................................ 457 class tf.errors.FailedPreconditionError .......................................... 457 class tf.errors.AbortedError .................................................................... 458 class tf.errors.OutOfRangeError ............................................................. 458 class tf.errors.UnimplementedError ...................................................... 459 class tf.errors.InternalError .................................................................. 459 class tf.errors.UnavailableError .......................................................... 460 class tf.errors.DataLossError .................................................................. 460 Training................................................................................................................................. 460
Optimizers ........................................................................................................................ 462 class tf.train.Optimizer .............................................................................. 462
Usage ........................................................................................................................... 463
Processing gradients before applying them. .......................................................... 463
Gating Gradients ......................................................................................................... 468 Slots .............................................................................................................................. 469
class tf.train.GradientDescentOptimizer .......................................... 470 class tf.train.AdagradOptimizer ............................................................. 471 class tf.train.MomentumOptimizer .......................................................... 472 class tf.train.AdamOptimizer .................................................................... 472 class tf.train.FtrlOptimizer .................................................................... 474 class tf.train.RMSPropOptimizer ............................................................. 475
Gradient Computation .................................................................................................... 476
tf.gradients(ys, xs, grad_ys=None, name='gradients', colocate_gradients_with_ops=False, gate_gradients=False,
aggregation_method=None) .............................................................................. 476 class tf.AggregationMethod ......................................................................... 478 tf.stop_gradient(input, name=None) ...................................................... 478
Gradient Clipping ............................................................................................................ 479
tf.clip_by_value(t, clip_value_min, clip_value_max,
name=None) ............................................................................................................... 480
tf.clip_by_norm(t, clip_norm, name=None) ....................................... 481
tf.clip_by_average_norm(t, clip_norm, name=None) .................... 482 tf.clip_by_global_norm(t_list, clip_norm, use_norm=None,
name=None) ............................................................................................................... 482
tf.global_norm(t_list, name=None) ........................................................ 484
Decaying the learning rate ............................................................................................ 485
tf.train.exponential_decay(learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None) ........... 485
Moving Averages ............................................................................................................ 486
class tf.train.ExponentialMovingAverage .......................................... 487
Coordinator and QueueRunner .................................................................................... 493 class tf.train.Coordinator ......................................................................... 493 class tf.train.QueueRunner ......................................................................... 498
tf.train.add_queue_runner(qr, collection='queue_runners')503 tf.train.start_queue_runners(sess=None, coord=None, daemon=True, start=True, collection='queue_runners') ........... 504
Summary Operations ..................................................................................................... 504
tf.scalar_summary(tags, values, collections=None,
name=None) ............................................................................................................... 505
tf.image_summary(tag, tensor, max_images=3,
collections=None, name=None) .................................................................... 505
tf.histogram_summary(tag, values, collections=None,
name=None) ............................................................................................................... 507
tf.nn.zero_fraction(value, name=None) .............................................. 508
tf.merge_summary(inputs, collections=None, name=None) ........ 509
tf.merge_all_summaries(key='summaries') .......................................... 509
Adding Summaries to Event Files ................................................................................ 510 class tf.train.SummaryWriter .................................................................... 510 tf.train.summary_iterator(path) ............................................................. 514
Training utilities ............................................................................................................... 515
tf.train.global_step(sess, global_step_tensor) ......................... 515
tf.train.write_graph(graph_def, logdir, name, as_text=True)
....................................................................................................................................... 516
Other Functions and Classes........................................................................................ 516 class tf.train.LooperThread ...................................................................... 517
tf.train.export_meta_graph(filename=None, meta_info_def=None, graph_def=None, saver_def=None,
collection_list=None, as_text=False) ................................................. 522
tf.train.generate_checkpoint_state_proto(save_dir, model_checkpoint_path, all_model_checkpoint_paths=None) ... 523 tf.train.import_meta_graph(meta_graph_or_file) ......................... 523
Wraps python functions ..................................................................................................... 524
Script Language Operators. .......................................................................................... 524
Other Functions and Classes........................................................................................ 525
tf.py_func(func, inp, Tout, name=None) ............................................ 525 Testing .................................................................................................................................. 525
Unit tests .......................................................................................................................... 526 tf.test.main() ..................................................................................................... 526
Utilities .............................................................................................................................. 527
tf.test.assert_equal_graph_def(actual, expected) .................... 527 tf.test.get_temp_dir() .................................................................................. 527
tf.test.is_built_with_cuda() .................................................................... 528
Gradient checking ........................................................................................................... 528
tf.test.compute_gradient(x, x_shape, y, y_shape, x_init_value=None, delta=0.001, init_targets=None) ............... 528 tf.test.compute_gradient_error(x, x_shape, y, y_shape, x_init_value=None, delta=0.001, init_targets=None) ............... 529
Layers (contrib) ................................................................................................................... 530
Higher level ops for building neural network layers. .................................................. 531
tf.contrib.layers.convolution2d(x, num_output_channels, kernel_size, activation_fn=None, stride=(1, 1), padding='SAME', weight_init=_initializer, bias_init=_initializer, name=None, weight_collections=None, bias_collections=None, output_collections=None,
weight_regularizer=None, bias_regularizer=None) ....................... 531
tf.contrib.layers.fully_connected(x, num_output_units, activation_fn=None, weight_init=_initializer, bias_init=_initializer, name=None, weight_collections=('weights',), bias_collections=('biases',), output_collections=('activations',),
weight_regularizer=None, bias_regularizer=None) ....................... 534
Regularizers..................................................................................................................... 536
tf.contrib.layers.l1_regularizer(scale) .......................................... 536 tf.contrib.layers.l2_regularizer(scale) .......................................... 537
Initializers ......................................................................................................................... 538
tf.contrib.layers.xavier_initializer(uniform=True,
seed=None, dtype=tf.float32) .................................................................... 538
tf.contrib.layers.xavier_initializer_conv2d(uniform=True,
seed=None, dtype=tf.float32) .................................................................... 539 Summaries ....................................................................................................................... 540
tf.contrib.layers.summarize_activation(op)................................... 540 tf.contrib.layers.summarize_tensor(tensor)................................... 540
tf.contrib.layers.summarize_tensors(tensors,
summarizer=summarize_tensor) .................................................................... 541
tf.contrib.layers.summarize_collection(collection,
name_filter=None, summarizer=summarize_tensor) ......................... 541
tf.contrib.layers.summarize_activations(name_filter=None,
summarizer=summarize_activation) .......................................................... 542
Other Functions and Classes........................................................................................ 542
tf.contrib.layers.assert_same_float_dtype(tensors=None,
dtype=None) ............................................................................................................. 542
Utilities (contrib) .................................................................................................................. 543
Miscellaneous Utility Functions .................................................................................... 543
tf.contrib.util.constant_value(tensor) ............................................ 543
tf.contrib.util.make_tensor_proto(values, dtype=None,
shape=None) ............................................................................................................. 544