DifferenceMonitor

class DifferenceMonitor(reducer: Reducer, attr: str, module: Module = None, train_update: bool = True, eval_update: bool = True, prepend: bool = False, filter_: Callable[[Any, Any], bool] | None = None, map_: Callable[[Any, Any], tuple[Tensor, ...]] | None = None, op_: Callable[[Tensor, Tensor], Tensor] | None = None)

Bases: Monitor

Records the difference of an attribute in a Module before and after its forward call.

Parameters:

• reducer (Reducer) – underlying means for reducing samples over time and storing them.

• attr (str) – attribute or nested attribute to target.

• module (Module, optional) – module to register as the target for monitoring, can be modified after construction. Defaults to None.

• train_update (bool, optional) – if this monitor should be called when the module being monitored is in train mode. Defaults to True.

• eval_update (bool, optional) – if this monitor should be called when the module being monitored is in eval mode. Defaults to True.

• prepend (bool, optional) – if this monitor should be called before other registered forward prehooks or posthooks. Defaults to False.

• filter (Callable[[Any, Any], bool] | None, optional) – test if the input should be passed to the reducer, ignores None values when None. Defaults to None.

• map (Callable[[Any, Any], tuple[torch.Tensor, ...]] | None, optional) – modifies the input before being passed to the reducer, post-forward value minus pre-forward value wrapped in a tuple if None. Defaults to None.

• op (Callable[[torch.Tensor, torch.Tensor], torch.Tensor] | None, optional) – operation to calculate the difference between post-forward and pre-forward, only used when map_ is None, subtraction when None. Defaults to None.

Note

The nested attribute should be specified with dot notation. For instance, if the observed module has an attribute a which in turn has an attribute b that should be monitored, then attr should be ‘a.b’`. Even with nested attributes, the monitor’s hook will be tied to the module with which it is registered.

Note

The left-hand argument of filter_, map_, and op_ is the attribute after the forward() call of module is run, and the right-hand argument is before it is run.

By default, filter_ will only reject an input if both the pre and post states are None. By default, map_ will use op_ to compare the pre-forward value from the post-forward value. If either is None (but not both), map_ will assume the None value was composed of all-zeros.

clear(**kwargs) → None

Clears monitor state and reinitializes the reducer’s state.

classmethod partialconstructor(reducer: Reducer, train_update: bool = True, eval_update: bool = True, prepend: bool = False, filter_: Callable[[Any, Any], bool] | None = None, map_: Callable[[Any, Any], tuple[Tensor, ...]] | None = None, op_: Callable[[Tensor, Tensor], Tensor] | None = None) → MonitorConstructor

Returns a function with a common signature for monitor construction.

Parameters:

• reducer (Reducer) – underlying means for reducing samples over time and storing them.

• train_update (bool, optional) – if this monitor should be called when the module being monitored is in train mode. Defaults to True.

• eval_update (bool, optional) – if this monitor should be called when the module being monitored is in eval mode. Defaults to True.

• prepend (bool, optional) – if this monitor should be called before other registered forward prehooks or posthooks. Defaults to False.

• filter (Callable[[Any, Any], bool] | None, optional) – test if the input should be passed to the reducer, ignores None values when None. Defaults to None.

• map (Callable[[Any, Any], tuple[torch.Tensor, ...]] | None, optional) – modifies the input before being passed to the reducer, post-forward value minus pre-forward value wrapped in a tuple if None. Defaults to None.

• op (Callable[[torch.Tensor, torch.Tensor], torch.Tensor] | None, optional) – operation to calculate the difference between post-forward and pre-forward, only used when map_ is None, subtraction when None. Defaults to None.

Returns:

partial constructor for monitor.

Return type:

MonitorConstructor