tensorflow中name_scope和variable_scope变量的使用

1. variable_scope的使用

首先，使用variable_scope可以很方便的管理get_varibale。

如何确定 get_variable 的 prefixed name?

1.1 variable scope是可以嵌套的:

import tensorflow as tf

with tf.variable_scope("tet1"):
    var3 = tf.get_variable("var3",shape=[2],dtype=tf.float32)
    print (var3.name)
    with tf.variable_scope("tet2"):
        var4 = tf.get_variable("var4",shape=[2],dtype=tf.float32)
        print (var4.name)

输出：

tet1/var3:0
tet1/tet2/var4:0

可以看到变量嵌套命名，类似于查找文件时一个完整的路径。

1.2 get_varibale.name 以创建变量的 scope 作为名字的prefix

示例1：

import tensorflow as tf

def te2():
    with tf.variable_scope("te2"):
        var2 = tf.get_variable("var2",shape=[2], dtype=tf.float32)
        print ('var2.name:',var2.name)
        def te1():
            with tf.variable_scope("te1"):
                var1 = tf.get_variable("var1", shape=[2], dtype=tf.float32)
            return var1
        return te1() #在scope te2 内调用的

res = te2()
print ('res.name:',res.name)

输出：

var2.name: te2/var2:0
res.name: te2/te1/var1:0

示例2：

import tensorflow as tf

def te2():
    with tf.variable_scope("te2"):
        var2 = tf.get_variable("var2",shape=[2], dtype=tf.float32)
        print ('var2.name:',var2.name)
        def te1():
            with tf.variable_scope("te1"):
                var1 = tf.get_variable("var1", shape=[2], dtype=tf.float32)
            return var1
    return te1() #在scope te2 外调用的

res = te2()
print ('res.name:',res.name)

输出：

var2.name: te2/var2:0
res.name: te1/var1:0

1.3 tf.variable_scope(“name”) 与 tf.variable_scope(scope)的区别

import tensorflow as tf

with tf.variable_scope("scope"):
    print (tf.get_variable("w",shape=[1]))#这个变量的name是 scope/w
    with tf.variable_scope("scope"):
       # 这个变量的name是 scope/scope/w,这两个变量的名字是不一样的，所以不会产生冲突
       print( tf.get_variable("w", shape=[1]) )

print('........................................')

with tf.variable_scope("yin"):
    print(tf.get_variable("w",shape=[1]))
    scope = tf.get_variable_scope()#这个变量的name是 yin/w
    print('scope:',scope)
    with tf.variable_scope(scope):#这种方式设置的scope，是用的外部的scope
        print(tf.get_variable("w", shape=[1]))#这个变量的name也是 yin/w,两个变量的名字一样，会报错

<tf.Variable 'scope/w:0' shape=(1,) dtype=float32_ref>
<tf.Variable 'scope/scope/w:0' shape=(1,) dtype=float32_ref>
........................................
<tf.Variable 'yin/w:0' shape=(1,) dtype=float32_ref>
scope: <tensorflow.python.ops.variable_scope.VariableScope object at 0x12060cd68>
Traceback (most recent call last):

  File "<ipython-input-1-cb0c968e524c>", line 1, in <module>
    runfile('/Users/lilong/Desktop/tensorflow_test/tensorflow_8/variable_use.py', wdir='/Users/lilong/Desktop/tensorflow_test/tensorflow_8')
    ....
    ....
    ValueError: Variable yin/w already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:
    ...

1.4 共享变量

共享变量的前提是，变量的名字是一样的，变量的名字是由变量名和其scope前缀一起构成， tf.get_variable_scope().reuse_variables() 是允许共享当前scope下的所有变量。reused_variables可以看同一个节点。

tf.get_variable_scope() :获取当前scope
tf.get_variable_scope().reuse_variables():共享变量

import tensorflow as tf

with tf.variable_scope("level1"):
    print(tf.get_variable("w",shape=[1]))
    scope = tf.get_variable_scope()
    print('scope:',scope)
    with tf.variable_scope("level2"):
        print(tf.get_variable("w", shape=[1]))

print('...................')

with tf.variable_scope("level1", reuse=True): #即使嵌套的variable_scope也会被reuse
    print(tf.get_variable("w",shape=[1]))
    scope = tf.get_variable_scope()
    print('scope:',scope)
    with tf.variable_scope("level2"):
        print(tf.get_variable("w", shape=[1]))

输出：

<tf.Variable 'level1/w:0' shape=(1,) dtype=float32_ref>
scope: <tensorflow.python.ops.variable_scope.VariableScope object at 0x12b470b00>
<tf.Variable 'level1/level2/w:0' shape=(1,) dtype=float32_ref>
...................
<tf.Variable 'level1/w:0' shape=(1,) dtype=float32_ref>
scope: <tensorflow.python.ops.variable_scope.VariableScope object at 0x12b470940>
<tf.Variable 'level1/level2/w:0' shape=(1,) dtype=float32_ref>

1.5 tf.variable_scope有时也会处理命名冲突

import tensorflow as tf

def test(name=None):
    # 可以看出，如果只是使用default_name这个属性来创建variable_scope的时候，会处理命名冲突
    with tf.variable_scope(name, default_name="scope") as scope:
        w = tf.get_variable("w", shape=[2, 10])

test()
test()
ws = tf.trainable_variables()
for w in ws:
    print(w.name)

输出：

scope/w:0
scope_1/w:0

2 name_scope的使用

2.1 在tensorflow中,有两个scope, 一个是name_scope一个是variable_scope，下面通过简单的示例说明：

示例1：

import tensorflow as tf

with tf.name_scope("hello") as name_scope:
    arr1 = tf.get_variable("arr1", shape=[2,10],dtype=tf.float32)
    print ('h1:',name_scope)
    print ('h2:',arr1.name)
    print ("scope_name:%s " % tf.get_variable_scope().original_name_scope)

运行：

h1: hello/
h2: arr1:0
scope_name:

示例2：

import tensorflow as tf;

with tf.variable_scope("hello") as variable_scope:
    arr1 = tf.get_variable("arr1", shape=[2, 10], dtype=tf.float32)

    print ('p1:',variable_scope)      # 输出变量命令空间对象
    print ('p2:',variable_scope.name) # 打印出变量空间名字
    print ('p3:',arr1.name)           # 输出变量名：hello/arr1:0
    print ('p4:',tf.get_variable_scope().original_name_scope) # 获得原始命名空间
    print ('p6:',tf.get_variable_scope()) # 获取的就是variable_scope对象

    with tf.variable_scope("xixi") as v_scope2:
        print ('p7:',tf.get_variable_scope().original_name_scope)
        print ('p8:',tf.get_variable_scope()) # 获取的就是v _scope2，这是另一个命名空间

输出：

p1: <tensorflow.python.ops.variable_scope.VariableScope object at 0x12ccd4b38>
p2: hello
p3: hello/arr1:0
p4: hello/
p6: <tensorflow.python.ops.variable_scope.VariableScope object at 0x12ccd4b38>
p7: hello/xixi/
p8: <tensorflow.python.ops.variable_scope.VariableScope object at 0x12ccd4dd8>

示例3：

import tensorflow as tf

with tf.name_scope("name1"):
    with tf.variable_scope("var1"):
        w = tf.get_variable("w",shape=[2])
        res = tf.add(w,[3])

print (w.name)
print (res.name)

输出：

var1/w:0
name1/var1/Add:0

variable_scope和name_scope都会给op的name加上前缀
这实际上是因为创建 variable_scope 时内部会创建一个同名的 name_scope
对比三个个程序可以看出:

name_scope对 get_variable()创建的变量的名字不会有任何影响,而创建的op会被加上前缀.
tf.get_variable_scope() 返回的只是 variable_scope,不管 name_scope. 所以以后在使用tf.get_variable_scope().reuse_variables() 时可以不用管name_scope

2.2 name_scope作用

name_scope 是给op_name加前缀, variable_scope是给get_variable()创建的变量的名字加前缀

典型的 TensorFlow 可以有数以千计的节点，如此多而难以一下全部看到，甚至无法使用标准图表工具来展示。为简单起见，我们为op/tensor名划定范围，并且可视化把该信息用于在图表中的节点上定义一个层级，默认情况下，只有顶层节点会显示。
下面这个例子使用tf.name_scope在hidden命名域下定义了三个操作:

import tensorflow as tf

with tf.name_scope('hidden') as scope:
  a = tf.constant(5, name='alpha')
  W = tf.Variable(tf.random_uniform([1, 2], -1.0, 1.0), name='weights')
  b = tf.Variable(tf.zeros([1]), name='biases')
  print a.name
  print W.name
  print b.name

输出：

hidden/alpha 
hidden/weights 
hidden/biases

2.3 tf.name_scope(None) 有清除name scope的作用

import tensorflow as tf
with tf.name_scope("hehe"):
    w1 = tf.Variable(1.0)
    with tf.name_scope(None):
        w2 = tf.Variable(2.0)
print(w1.name)
print(w2.name)

#hehe/Variable:0
#Variable:0

小小总结：
1. 使用tf.Variable()的时候，tf.name_scope()和tf.variable_scope() 都会给 Variable 和 op 的 name属性加上前缀。
2. 使用tf.get_variable()的时候，tf.name_scope()就不会给 tf.get_variable()创建出来的Variable加前缀。但是 tf.Variable() 创建出来的就会受到 name_scope 的影响

参考：
https://blog.csdn.net/u012436149/article/details/53018924/
https://blog.csdn.net/u012436149/article/details/53081454