Normalization¶

class Normalization(module: Module, attr: str, order: int | float, scale: float | complex, dim: int | tuple[int, ...] | None, epsilon: float = 1e-12, *, train_update: bool = True, eval_update: bool = True, as_prehook: bool = False, prepend: bool = False, always_call: bool = False)[source]¶

Normalizes attribute of registered module on call.

Parameters:

module (nn.Module) – module to which the hook should be registered.
attr (str) – fully-qualified string name of attribute to normalize.
order (int | float) – order of \(p\)-norm by which to normalize.
scale (float | complex) – desired \(p\)-norm of elements along specified dimensions.
dim (int | tuple[int] | None) – dimension(s) along which to normalize, all dimensions if None.
epsilon (float, optional) – value added to the denominator in case of zero-valued norms. Defaults to 1e-12.
train_update (bool, optional) – if normalization should be performed when hooked module is in train mode. Defaults to True.
eval_update (bool, optional) – if normalization should be performed when hooked module is in eval mode. Defaults to True.
as_prehook (bool, optional) – if normalization should occur before forward() is. Defaults to False.
prepend (bool, optional) – if normalization should occur before other hooks previously registered to the hooked module. Defaults to False.
always_call (bool, optional) – if the hook should be run even if an exception occurs, only applies when as_prehook is False. Defaults to False.

hook(module: Module) → None[source]¶

Function to be called on the registered module’s call.