mindspore.nn.CosineDecayLR
- class mindspore.nn.CosineDecayLR(min_lr, max_lr, decay_steps)[source]
Calculates learning rate based on cosine decay function.
For current step, the formula of computing decayed learning rate is:
\[\begin{split}decayed\_learning\_rate = &min\_lr + 0.5 * (max\_lr - min\_lr) *\\ &(1 + cos(\frac{current\_step}{decay\_steps}\pi))\end{split}\]- Parameters
- Inputs:
global_step (Tensor) - The current step number.
- Outputs:
Tensor. The learning rate value for the current step with shape \(()\).
- Raises
TypeError – If min_lr or max_lr is not a float.
TypeError – If decay_steps is not an int.
ValueError – If min_lr is less than 0 or decay_steps is less than 1.
ValueError – If max_lr is less than or equal to 0.
- Supported Platforms:
Ascend
GPU
Examples
>>> import mindspore >>> from mindspore import Tensor, nn >>> >>> min_lr = 0.01 >>> max_lr = 0.1 >>> decay_steps = 4 >>> global_steps = Tensor(2, mindspore.int32) >>> cosine_decay_lr = nn.CosineDecayLR(min_lr, max_lr, decay_steps) >>> lr = cosine_decay_lr(global_steps) >>> net = nn.Dense(2, 3) >>> optim = nn.SGD(net.trainable_params(), learning_rate=lr)