Comparing the function difference with torch.nn.Module.parameters
torch.nn.Module.parameters
torch.nn.Module.parameters(recurse=True)
For more information, see torch.nn.Module.parameters.
mindspore.nn.Cell.get_parameters
mindspore.nn.Cell.get_parameters(expand=True)
For more information, see mindspore.nn.Cell.get_parameters.
Differences
In PyTorch, the network has three concepts: parameter
, buffer
, and state
, where state
is the collection of parameter
and buffer
. parameter
can use the requires_grad
attribute to distinguish whether the parameter
in the network needs to be optimized; buffer
is mostly defined as an invariant in the network, for example, when defining the network, the running_mean
and running_var
in BN will be automatically register as buffer; users can also register parameter
and buffer
through related interfaces.
-torch.nn.Module.parameters
: Get the parameter
in the network, and return a generator.
-torch.nn.Module.named_parameters
: Get the name of parameter
and parameter
itself in the network, and return a generator.
In MindSpore, there is only the concept of parameter
currently. The requires_grad
attribute is used to distinguish whether the parameter
in the network needs to be optimized. For example, when defining the network, the moving_mean
and moving_var
in BN will be defined as parameter
with attribute requires_grad=False
.
-mindspore.nn.Cell.get_parameters
: Get the parameter
in the network, and return a generator.
-mindspore.nn.Cell.trainable_params
: The function returns a list of all trainable parameters(with attribute requires_grad=True
).
Due to the difference in concept definitions, although both torch.nn.Module.parameters
and mindspore.nn.Cell.get_parameters
get the parameter
in the network, the returned content is slightly different: for example, moving_mean
and moving_variance
in BN are registered as buffer
in PyTorch, so they will not be returned by torch.nn.Module.parameters
interface, but they will be returned by mindspore.nn.Cell.get_parameters
because they are defined as parameter
in MindSpore.
Code Example
from mindspore import nn
class ConvBN(nn.Cell):
def __init__(self):
super(ConvBN, self).__init__()
self.conv = nn.Conv2d(3, 64, 3)
self.bn = nn.BatchNorm2d(64)
def construct(self, x):
x = self.conv(x)
x = self.bn(x)
return x
class MyNet(nn.Cell):
def __init__(self):
super(MyNet, self).__init__()
self.build_block = nn.SequentialCell(ConvBN(), nn.ReLU())
def construct(self, x):
return self.build_block(x)
# The following implements mindspore.nn.Cell.get_parameters() with MindSpore.
net = MyNet()
print(type(net.get_parameters()), "\n")
for params in net.get_parameters():
print("Name: ", params.name)
print("params: ", params)
# Out:
<class 'generator'>
Name: build_block.0.conv.weight
params: Parameter (name=build_block.0.conv.weight, shape=(64, 3, 3, 3), dtype=Float32, requires_grad=True)
Name: build_block.0.bn.moving_mean
params: Parameter (name=build_block.0.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False)
Name: build_block.0.bn.moving_variance
params: Parameter (name=build_block.0.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)
Name: build_block.0.bn.gamma
params: Parameter (name=build_block.0.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True)
Name: build_block.0.bn.beta
params: Parameter (name=build_block.0.bn.beta, shape=(64,), dtype=Float32, requires_grad=True)
import torch.nn as nn
class ConvBN(nn.Module):
def __init__(self):
super(ConvBN, self).__init__()
self.conv = nn.Conv2d(3, 64, 3)
self.bn = nn.BatchNorm2d(64)
def forward(self, x):
x = self.conv(x)
x = self.bn(x)
return x
class MyNet(nn.Module):
def __init__(self):
super(MyNet, self).__init__()
self.build_block = nn.Sequential(ConvBN(), nn.ReLU())
def construct(self, x):
return self.build_block(x)
# The following implements torch.nn.Module.parameters() with torch.
net = MyNet()
print(type(net.parameters()), "\n")
for name, params in net.named_parameters():
print("Name: ", name)
print("params: ", params.size())
# Out:
<class 'generator'>
Name: build_block.0.conv.weight
params: torch.Size([64, 3, 3, 3])
Name: build_block.0.conv.bias
params: torch.Size([64])
Name: build_block.0.bn.weight
params: torch.Size([64])
Name: build_block.0.bn.bias
params: torch.Size([64])