Differences with torch.nn.GroupNorm
torch.nn.GroupNorm
class torch.nn.GroupNorm(
num_groups,
num_channels,
eps=1e-05,
affine=True
)(input) -> Tensor
For more information, see torch.nn.GroupNorm.
mindspore.nn.GroupNorm
class mindspore.nn.GroupNorm(
num_groups,
num_channels,
eps=1e-05,
affine=True,
gamma_init='ones',
beta_init='zeros'
)(x) -> Tensor
For more information, see mindspore.nn.GroupNorm.
Differences
PyTorch: Group normalization is performed on the mini-batch input by dividing the channels into groups and then calculating the mean and variance within each group for normalization.
MindSpore: MindSpore API implements basically the same function as PyTorch. MindSpore can also perform additional initialization of the radiating parameters that need to be learned.
Categories |
Subcategories |
PyTorch |
MindSpore |
Difference |
---|---|---|---|---|
Parameters |
Parameter 1 |
num_groups |
num_groups |
- |
Parameter 2 |
num_channels |
num_channels |
- |
|
Parameter 3 |
eps |
eps |
- |
|
Parameter 4 |
affine |
affine |
- |
|
Parameter 5 |
- |
gamma_init |
Initialize the radial transform parameter gamma used for learning in the formula. The default is ‘ones’, while PyTorch cannot be set additionally, can only be ‘ones’. |
|
Parameter 6 |
- |
beta_init |
Initialize the radial transform parameter beta used for learning in the formula. The default is ‘zeros’, while PyTorch cannot be set additionally, can only be ‘zeros’. |
|
Input |
Single input |
input |
x |
Interface input, same function, different parameter names |
Code Example 1
MindSpore API basically implements the same function as TensorFlow, and MindSpore can also perform additional initialization of the two learning parameters.
# PyTorch
import torch
import numpy as np
from torch import tensor, nn
x = tensor(np.ones([1, 2, 4, 4], np.float32))
net = nn.GroupNorm(2, 2)
output = net(x).detach().numpy()
print(output)
# [[[[0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]
#
# [[0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]]]
# MindSpore
import mindspore as ms
import numpy as np
from mindspore import Tensor, nn
x = Tensor(np.ones([1, 2, 4, 4], np.float32))
net = nn.GroupNorm(2, 2)
output = net(x)
print(output)
# [[[[0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]
#
# [[0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]]]