Clash Royale CLAN TAG#URR8PPP

How to put tf.layers variables in tf.name_scope/tf.variable_scope?

Okay, so I have been messing around with Tensorflow a lot recently and have started using the higher level Layers API. But it has been posing a problem for me.

The following code produces a correct(ish) graph for a convolutional block:

def conv_layer(self, inputs, filter_size = 3, num_filters = 256, name = None):
scope_name = name
if name == None:
scope_name = "conv_layer"

with tf.name_scope(scope_name):
conv = tf.contrib.layers.conv2d(inputs, num_filters, filter_size, activation_fn = None)
batch_norm = tf.contrib.layers.batch_norm(conv)
act = tf.nn.leaky_relu(batch_norm)

return act


The problem is that the tf.layers API makes some ugly variables that do not actually stay within the name_scope. Here is the Tensorboard view so you can see what I mean.

Is there anyway to get those variables to go into the scope? This is a big problem when it comes to visualizing the graph because I plan this network to much larger. (As you can see to the right, this is already a big problem, I have to remove those from the main graph manually every time I boot up Tensorboard.)

Thanks in advanced :)

EDIT - SOLUTION / WORK AROUND

Changing each instance of name_scope with variable_scope the problem has been omitted. However, I had to assign each variable_scope with a unique ID and set reuse = False.

name_scope

variable_scope

variable_scope

reuse = False

def conv_layer(self, inputs, filter_size = 3, num_filters = 256, name = None):
scope_name = name
if name == None:
scope_name = "conv_layer_" + str(self.conv_id)
self.conv_id += 1

with tf.variable_scope(scope_name, reuse = False):
conv = tf.contrib.layers.conv2d(inputs, num_filters, filter_size, activation_fn = None)
batch_norm = tf.contrib.layers.batch_norm(conv)
act = tf.nn.leaky_relu(batch_norm)

return act


As you can see, the variables are nicely hidden away in the correct blocks :)

1 Answer
1

You can try using tf.variable_scope instead. tf.name_scope is ignored by variables created via tf.get_variable() which is usually used by tf.layers functions. This is in contrast to variables created via tf.Variable.

tf.variable_scope

tf.name_scope

tf.get_variable()

tf.layers

tf.Variable

See this question for an (albeit somewhat outdated) explanation of the differences.

name_scope

variable_scope

ValueError: Variable conv_layer/Conv/weights already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?

reuse

variable_scope

Variable conv_layer/Conv/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

variable_scope

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.