mindspore.experimental.optim.lr_scheduler.ReduceLROnPlateau
- class mindspore.experimental.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.1, patience=10, threshold=1e-4, threshold_mode='rel', cooldown=0, min_lr=0, eps=1e-8)[source]
Reduce learning rate when a metric has stopped improving. Models often benefit from reducing the learning rate by a factor of 2-10 once learning stagnates. The scheduler reads the metrics metrics during execution and adjusts the learning rate via the step method if the metrics do not improve within patience cycles.
Warning
This is an experimental lr scheduler module that is subject to change. This module must be used with optimizers in Experimental Optimizer .
- Parameters
optimizer (
mindspore.experimental.optim.Optimizer) – Wrapped optimizer.mode (str, optional) – Trigger mode that triggers a reduction in learning rate when the monitoring metrics are at their min / max point. In min mode, lr will be reduced when the quantity monitored has stopped decreasing; in max mode it will be reduced when the quantity monitored has stopped increasing. Default:
'min'.factor (float, optional) – Factor by which the learning rate will be reduced. Default:
0.1.patience (int, optional) – Number of epochs with no improvement after which learning rate will be reduced. For example, if patience = 2, then we will ignore the first 2 epochs with no improvement, and will only decrease the LR after the 3rd epoch if the loss still hasn't improved then. Default:
10.threshold (float, optional) – Threshold for measuring the new optimum, to only focus on significant changes. Default:
1e-4.threshold_mode (str, optional) –
A mode for measuring indicators of change for the better. One of rel, abs. Default:
'rel'.Assume that best represents the best value of the current performance metric.
In
'rel'mode, the indicator is compared to a threshold in proportional form:When mode is
'max', the indicator is considered better if it exceeds best * ( 1 + threshold ).When mode is
'min', the indicator is considered better if it is lower than best * ( 1 - threshold ).
In
'abs'mode, the indicator is compared to threshold in absolute value form:When mode is
'max', the indicator is considered better if it exceeds best + threshold.When mode is
'min', the indicator is considered better if it is lower than best - threshold.
cooldown (int, optional) – Number of epochs to wait before resuming normal operation after lr has been reduced. Default:
0.min_lr (Union(float, list), optional) – A scalar or a list of scalars. A lower bound on the learning rate of all param groups or each group respectively. Default:
0.eps (float, optional) – Minimal decay applied to lr. If the difference between new and old lr is smaller than eps, the update is ignored. Default:
1e-8.
- Raises
ValueError – factor is greater than or equal to 1.0.
TypeError – optimizer is not an Optimizer.
ValueError – When min_lr is a list or tuple, the length of it is not equal to the number of param groups.
ValueError – mode is neither
'min'nor'max'.ValueError – threshold_mode is neither
'rel'nor'abs'.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore.experimental import optim >>> from mindspore import nn >>> net = nn.Dense(3, 2) >>> optimizer = optim.Adam(net.trainable_params(), 0.1) >>> scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min', patience=0) >>> metrics = [1, 1.5, 1.8, 0.4, 0.5] >>> for i in range(5): ... scheduler.step(metrics[i]) ... current_lr = scheduler.get_last_lr() ... print(current_lr) [Tensor(shape=[], dtype=Float32, value= 0.1)] [Tensor(shape=[], dtype=Float32, value= 0.01)] [Tensor(shape=[], dtype=Float32, value= 0.001)] [Tensor(shape=[], dtype=Float32, value= 0.001)] [Tensor(shape=[], dtype=Float32, value= 0.0001)]