[FIXED] Reset tensorflow Optimizer

Issue

I am loading from a saved model and I would like to be able to reset a tensorflow optimizer such as an Adam Optimizer. Ideally something like:

sess.run([tf.initialize_variables(Adamopt)])

or

sess.run([Adamopt.reset])

I have tried looking for an answer but have yet to find any way to do it. Here's what I've found which don't address the issue: https://github.com/tensorflow/tensorflow/issues/634

In TensorFlow is there any way to just initialize uninitialised variables?

Tensorflow: Using Adam optimizer

I basically just want a way to reset the "slot" variables in the Adam Optimizer.

Thanks

Solution

The simplest way I found was to give the optimizer its own variable scope and then run

optimizer_scope = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES,
                                 "scope/prefix/for/optimizer")
sess.run(tf.initialize_variables(optimizer_scope))

idea from freeze weights

Answered By - Steven