Comparing the function differences between torch.optim.lr_scheduler.StepLR and torch.optim.lr_scheduler.MultiStepLR

torch.optim.lr_scheduler.StepLR

torch.optim.lr_scheduler.StepLR(
    optimizer,
    step_size,
    gamma=0.1,
    last_epoch=-1,
    verbose=False
)

For more information, see torch.optim.lr_scheduler.StepLR.

torch.optim.lr_scheduler.MultiStepLR

torch.optim.lr_scheduler.MultiStepLR(
    optimizer,
    milestones,
    gamma=0.1,
    last_epoch=-1,
    verbose=False
)

For more information, see torch.optim.lr_scheduler.MultiStepLR.

mindspore.nn.piecewise_constant_lr

mindspore.nn.piecewise_constant_lr(
    milestone,
    learning_rates
)

For more information, see mindspore.nn.piecewise_constant_lr.

Differences

PyTorch (torch.optim.lr_scheduler.StepLR): Setting the learning rate in segments. torch.optim.lr_scheduler.StepLR multiplies the learning rate by gamma every fixed step_size, by passing in step_size. When verbose is True, the relevant information is printed for each update.

MindSpore (mindspore.nn.piecewise_constant_lr): Pass in the list of step values of milestones and the corresponding list of learning rate setting values, reach the step values and the learning rate takes the corresponding values. A list of learning rates is eventually returned as input to the optimizer.

Categories	Subcategories	PyTorch	MindSpore	Differences
Parameter	Parameter 1	optimizer		Optimizer for PyTorch applications. MindSpore does not have this Parameter
	Parameter 2	step_size	milestone	MindSpore segmentation updates the step list of learning rates, and PyTorch uses a fixed step value.
	Parameter 3	gamma		Parameters for PyTorch decay learning rate. MindSpore does not have this Parameter.
	Parameter 4	last_epoch		MindSpore does not have this Parameter.
	Parameter 5		verbose	PyTorch verbose prints information about each update when it is True.
	Parameter 6		learning_rates	List of MindSpore settings for learning rate

PyTorch (torch.optim.lr_scheduler.MultiStepLR): torch.optim.lr_scheduler.MultiStepLR reaches the step value by passing in a list of step values for milestones, and the learning rate is multiplied by gamma. When used, the optimizer is used as input and the step method is called during the training process to update the values. verbose prints information about each update when it is True.

MindSpore (mindspore.nn.piecewise_constant_lr): Pass in the list of step values for milestones and the corresponding list of learning rate setting values, reach the step values and the learning rate takes the corresponding values. A list of learning rates is eventually returned as input to the optimizer.

Categories	Subcategories	PyTorch	MindSpore	Differences
Parameter	Parameter 1	optimizer		Optimizer for PyTorch applications. MindSpore does not have this Parameter
	Parameter 2	milestones	milestone	Segmentation updates the step list of learning rates, with the same functions and different parameter names
	Parameter 3	gamma		Parameters for PyTorch decay learning rate. MindSpore does not have this Parameter.
	Parameter 4	last_epoch		MindSpore does not have this Parameter.
	Parameter 5		verbose	PyTorch verbose prints information about each update when it is True. MindSpore does not have this Parameter.
	Parameter 6		learning_rates	List of MindSpore settings for learning rate

Code Example

from mindspore import nn

# In MindSpore：
milestone = [2, 5, 10]
learning_rates = [0.1, 0.05, 0.01]
output = nn.piecewise_constant_lr(milestone, learning_rates)
print(output)
# Out：
# [0.1, 0.1, 0.05, 0.05, 0.05, 0.01, 0.01, 0.01, 0.01, 0.01]


# In torch:
import numpy as np
import torch
from torch import optim

model = torch.nn.Sequential(torch.nn.Linear(20, 1))
optimizer = optim.SGD(model.parameters(), 0.1)

# step_lr
step_lr = optim.lr_scheduler.StepLR(optimizer, step_size=2, gamma=0.9)
# multi_step_lr
multi_step_lr = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[30, 80], gamma=0.9)