tf.control_dependencies()函数解析

这两天看batch normalization的代码时，碰到一个函数tf.control_dependencies()，特此记录。

with tf.control_dependencies([a, b]):
  # 只有在a和b执行完后，c和d才会被执行
  # 意思就是c，d操作依赖a，b操作
  c = ...
  d = ...

此函数指定某些操作执行的依赖关系，tf.control_dependencies(control_inputs)返回的是一个控制依赖的上下文管理器，使用with关键字可以让在这个上下文环境中的操作都在control_inputs之后执行。

1.可以嵌套control_dependencies 使用

with tf.control_dependencies([a, b]):
  # Ops constructed here run after `a` and `b`.
  with tf.control_dependencies([c, d]):
    # Ops constructed here run after `a`, `b`, `c`, and `d`.

2.可以传入None 来消除依赖：

with tf.control_dependencies([a, b]):
  # Ops constructed here run after `a` and `b`.
  with tf.control_dependencies(None):
    # Ops constructed here run normally, not waiting for either `a` or `b`.
    with tf.control_dependencies([c, d]):
      # Ops constructed here run after `c` and `d`, also not waiting
      # for either `a` or `b`.

3.控制依赖只对那些在上下文环境中建立的操作有效，仅仅在context中使用一个操作或张量是没用的

# WRONG
def my_func(pred, tensor):
  t = tf.matmul(tensor, tensor)
  with tf.control_dependencies([pred]):
    # The matmul op is created outside the context, so no control dependency will be added.
    return t

# RIGHT
def my_func(pred, tensor):
  with tf.control_dependencies([pred]):
    # The matmul op is created in the context, so a control dependency will be added.
    return tf.matmul(tensor, tensor)