Comparing the function differences between torch.optim.lr_scheduler.StepLR and torch.optim.lr_scheduler.MultiStepLR
torch.optim.lr_scheduler.StepLR
torch.optim.lr_scheduler.StepLR(
optimizer,
step_size,
gamma=0.1,
last_epoch=-1,
verbose=False
)
For more information, see torch.optim.lr_scheduler.StepLR.
torch.optim.lr_scheduler.MultiStepLR
torch.optim.lr_scheduler.MultiStepLR(
optimizer,
milestones,
gamma=0.1,
last_epoch=-1,
verbose=False
)
For more information, see torch.optim.lr_scheduler.MultiStepLR.
mindspore.nn.piecewise_constant_lr
mindspore.nn.piecewise_constant_lr(
milestone,
learning_rates
)
For more information, see mindspore.nn.piecewise_constant_lr.
Differences
PyTorch (torch.optim.lr_scheduler.StepLR): Setting the learning rate in segments. torch.optim.lr_scheduler.StepLR
multiplies the learning rate by gamma every fixed step_size, by passing in step_size. When verbose
is True, the relevant information is printed for each update.
MindSpore (mindspore.nn.piecewise_constant_lr): Pass in the list of step values of milestones and the corresponding list of learning rate setting values, reach the step values and the learning rate takes the corresponding values. A list of learning rates is eventually returned as input to the optimizer.
Categories |
Subcategories |
PyTorch |
MindSpore |
Differences |
---|---|---|---|---|
Parameter |
Parameter 1 |
optimizer |
Optimizer for PyTorch applications. MindSpore does not have this Parameter |
|
Parameter 2 |
step_size |
milestone |
MindSpore segmentation updates the step list of learning rates, and PyTorch uses a fixed step value. |
|
Parameter 3 |
gamma |
Parameters for PyTorch decay learning rate. MindSpore does not have this Parameter. |
||
Parameter 4 |
last_epoch |
MindSpore does not have this Parameter. |
||
Parameter 5 |
verbose |
PyTorch |
||
Parameter 6 |
learning_rates |
List of MindSpore settings for learning rate |
PyTorch (torch.optim.lr_scheduler.MultiStepLR): torch.optim.lr_scheduler.MultiStepLR
reaches the step value by passing in a list of step values for milestones, and the learning rate is multiplied by gamma. When used, the optimizer is used as input and the step
method is called during the training process to update the values. verbose
prints information about each update when it is True.
MindSpore (mindspore.nn.piecewise_constant_lr): Pass in the list of step values for milestones and the corresponding list of learning rate setting values, reach the step values and the learning rate takes the corresponding values. A list of learning rates is eventually returned as input to the optimizer.
Categories |
Subcategories |
PyTorch |
MindSpore |
Differences |
---|---|---|---|---|
Parameter |
Parameter 1 |
optimizer |
Optimizer for PyTorch applications. MindSpore does not have this Parameter |
|
Parameter 2 |
milestones |
milestone |
Segmentation updates the step list of learning rates, with the same functions and different parameter names |
|
Parameter 3 |
gamma |
Parameters for PyTorch decay learning rate. MindSpore does not have this Parameter. |
||
Parameter 4 |
last_epoch |
MindSpore does not have this Parameter. |
||
Parameter 5 |
verbose |
PyTorch |
||
Parameter 6 |
learning_rates |
List of MindSpore settings for learning rate |
Code Example
from mindspore import nn
# In MindSpore:
milestone = [2, 5, 10]
learning_rates = [0.1, 0.05, 0.01]
output = nn.piecewise_constant_lr(milestone, learning_rates)
print(output)
# Out:
# [0.1, 0.1, 0.05, 0.05, 0.05, 0.01, 0.01, 0.01, 0.01, 0.01]
# In torch:
import numpy as np
import torch
from torch import optim
model = torch.nn.Sequential(torch.nn.Linear(20, 1))
optimizer = optim.SGD(model.parameters(), 0.1)
# Use step_lr
step_lr = optim.lr_scheduler.StepLR(optimizer, step_size=2, gamma=0.9)
# Use multi_step_lr
multi_step_lr = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[30, 80], gamma=0.9)