elektronn3.training.padam module

class elektronn3.training.padam.Padam(*args: Any, **kwargs: Any)[source]

Bases: torch.optim.

Implements Partially adaptive momentum estimation (Padam) algorithm.

Parameters
  • params (iterable) – iterable of parameters to optimize or dicts defining parameter groups

  • lr (float, optional) – learning rate (default: 1e-1)

  • betas (Tuple[float, float], optional) – coefficients used for computing running averages of gradient and its square (default: (0.9, 0.999))

  • eps (float, optional) – term added to the denominator to improve numerical stability (default: 1e-8)

  • weight_decay (float, optional) – weight decay (L2 penalty) (default: 0)

  • partial (float, optional) – partially adaptive parameter

step(closure=None)[source]

Performs a single optimization step.

Parameters

closure (callable, optional) – A closure that reevaluates the model and returns the loss.