TensorFlow study notes (1): variable and get_variable, name_scope() and variable_scope()

Variable

There are two ops about variables in tensorflow, tf.Variable() and tf.get_variable(). The difference between these two is described below.
When using tf.Variable, if a naming conflict is detected, the system will handle it by itself. When using tf.get_variable(), the system will not handle the conflict, but will report an error

import tensorflow as tf
w_1 = tf.Variable(3,name="w_1")
w_2 = tf.Variable(1,name="w_1")
print w_1.name
print w_2.name
#output
#w_1:0
#w_1_1:0

import tensorflow as tf

w_1 = tf.get_variable(name="w_1",initializer=1)
w_2 = tf.get_variable(name="w_1",initializer=2)
#error message
#ValueError: Variable w_1 already exists, disallowed. Did
#you mean to set reuse=True in VarScope?

Based on the characteristics of these two functions, when we need to share variables, we need to use them tf.get_variable(). In other cases, the two are used the same

• tf.get_variable() and tf.Variable() are the two main ways to create variables in TensorFlow;
• If you use tf.get_variable() and tf.Variable() in the context of tf.name_scope(), the main difference is that
• • The variable name created by tf.get_variable() is not affected by name_scope;
  • For variables created by tf.get_variable(), the value of the name attribute cannot be the same; when tf.Variable() creates a variable, the value of the name attribute is allowed to be repeated (the alias mechanism will be automatically introduced in the underlying implementation.

import tensorflow as tf

with tf.variable_scope("scope1"):
    w1 = tf.get_variable("w1", shape=[])
    w2 = tf.Variable(0.0, name="w2")
with tf.variable_scope("scope1", reuse=True):
    w1_p = tf.get_variable("w1", shape=[])
    w2_p = tf.Variable(1.0, name="w2")

assert w1 == w1_p
assert w2 != w2_p

The behavior of the get_variable() function depends on the state of reuse:

• case1: reuse is set to False, creating and returning a new variable:

  with tf.variable_scope('foo'):
      v = tf.get_variable('v', [1])
  assert v.name == 'foo/v:0

  
  

• case2: reuse is set to True, will search in the stored variable with the given name:

  with tf.variable_scope('foo'):
      v = tf.get_variable('v', [1])
  with tf.variable_scope('foo', reuse=True):
      v1 = tf.get_variable('v')
  assert v1 == v

  
  

Seeing this, you can understand the true face of the parameter reuse mentioned on the official website. Since tf.Variable() every time a new object is created, all reuse=True has nothing to do with it. For get_variable() , if a variable object has already been created, return that object, and if no variable object was created, create a new one.

variable_scope()

A double-nested namespace:

with tf.variable_scope('foo'):
    with tf.variable_scope('bar'):
        v = tf.get_variable('v', [1])
assert v.name == 'foo/bar/v:0'

with tf.name_scope('foo'):
    with tf.variable_scope('bar'):
        v = tf.get_variable('v', [1])
assert v.name == 'bar/v:0'