neuralgym.callbacks¶
Callback Class¶
-
class
neuralgym.callbacks.
Callback
(cb_loc)¶ Callback class.
Callbacks are functions that execute automatically during training/evaluation process (primary trainer). For examples, saving/loading models, scheduling learning rate, updating network parameters (assigning target network in reinforcement learning), summary learning processes, saving images, secondary trainer (generative adversarial network), etc.
Currently there are three types of callbacks:
and five types of locations to call (supported in primary trainer):
- train_start
- train_end
- step_start
- step_end
- exception
-
run
()¶ Abstract method for executing the callback.
Note: ops should be defined in __init__, otherwise when callbacks are called multiple times, the ops in graph will continue increasing.
-
class
neuralgym.callbacks.
PeriodicCallback
(cb_loc, pstep, func=None, **kwargs)¶ PeriodicCallback executes periodically.
PeriodicalCallback is executed at:
- at the step start, and
- at the step end
of training every p steps periodically.
Parameters: - cb_loc – callback location
- pstep (int) – run function every pstep
- func (function) – function to call
- **kwargs – kwargs for function
-
class
neuralgym.callbacks.
OnceCallback
(cb_loc, func=None, **kwargs)¶ OnceCallback only executes once.
OnceCallback is executed:
- at the train start,
- at the train end, and
- when exception occurs
during training process.
-
class
neuralgym.callbacks.
ScheduledCallback
(cb_loc, schedule)¶ ScheduledCallback executes according to its schedule.
ScheduledCallback is executed:
- at the step start, and
- at the step end
according to recorded step in schedule.
Parameters: - cb_loc – callback location
- schedule (dict) – a dict, with step as its key, funcs as its value: e.g. {1: func1, 80: func2}
Callbacks¶
HyperParamScheduler¶
-
class
neuralgym.callbacks.
HyperParamScheduler
(param_name, schedule, scope=None, cb_loc=<CallbackLoc.step_end: 3>)¶ Set hyper parameters according to schedule.
This callback sets hyper parameters with numpy using tf.assign according to schedule.
Examples:
HyperParamScheduler( 'lr', { 1: 1e-2, 150: 1e-3, 225: 4e-4, 300: 1e-4, }, scope=None, )
WeightsViewer¶
-
class
neuralgym.callbacks.
WeightsViewer
(counts=True, size=True, verbose=True, hist_summary=True)¶ WeightsViewer logs names and size of all weights.
Parameters:
ModelSaver¶
ModelRestorer¶
-
class
neuralgym.callbacks.
ModelRestorer
(saver, dump_prefix=None, ckpt_file=None, optimistic=False)¶ Restore model from file either with dump_prefix or ckpt_file.
Parameters:
NPZModelLoader¶
-
class
neuralgym.callbacks.
NPZModelLoader
(npz_file, weights=None, variable_scope=<tensorflow.python.ops.variable_scope.VariableScope object>)¶ NPZModelLoader loads a model with weights in npz file.
Parameters: - npz_file (str) – name of npz_file
- weights – if provided, only load names in weights from npz file
- variable_scope – if provided, load all weights in this scope, otherwise load from default variable scope.
Examples:
# TODO
ModelSync¶
-
class
neuralgym.callbacks.
ModelSync
(pstep, from_namescope, to_namescope, step_start=False)¶ ModelSync.
Currently it only supports sync trainable variables from one namescope to another namescope, which is enough for reinforcement learning.
Parameters: Examples:
# TODO
-
run
(sess, step)¶ Run model sync
-