Differences with torch.nn.Linear
torch.nn.Linear
class torch.nn.Linear(
in_features,
out_features,
bias=True
)(input) -> Tensor
For more information, see torch.nn.Linear.
mindspore.nn.Dense
class mindspore.nn.Dense(
in_channels,
out_channels,
weight_init=None,
bias_init=None,
has_bias=True,
activation=None
)(x) -> Tensor
For more information, see mindspore.nn.Dense.
Differences
Pytorch: Fully connected layer that implements the matrix multiplication operation.
MindSpore: The implementation function of the API in MindSpore is basically the same as that of PyTorch, and it is possible to add activation functions after the fully connected layer.
Weight Initialization Difference
When weight_init
of mindspore.nn.Dense is None
, the weight is initialized using HeUniform. This is the same as PyTorch weight initialization.
When bias_init
of mindspore.nn.Dense is None
, the bias is initialized using Uniform. This is the same as the PyTorch bias initialization.
Categories |
Subcategories |
PyTorch |
MindSpore |
Differences |
---|---|---|---|---|
Parameters |
Parameter 1 |
in_features |
in_channels |
Same function, different parameter names |
Parameter 2 |
out_features |
out_channels |
Same function, different parameter names |
|
Parameter 3 |
bias |
has_bias |
Same function, different parameter names |
|
Parameter 4 |
- |
weight_init |
Initialization method for the weight parameter, which is not available for PyTorch |
|
Parameter 5 |
- |
bias_init |
Initialization method for the bias parameter, which is not available for PyTorch |
|
Parameter 6 |
- |
activation |
Activation function applied to the output of the fully connected layer, which is not available for PyTorch |
|
Input |
Single input |
input |
x |
Same function, only different parameter names |
Code Example
The two APIs achieve the same function and have the same usage.
# PyTorch
import torch
from torch import nn
import numpy as np
net = nn.Linear(3, 4)
x = torch.tensor(np.array([[180, 234, 154], [244, 48, 247]]), dtype=torch.float)
output = net(x)
print(output.detach().numpy().shape)
# (2, 4)
# MindSpore
import mindspore
from mindspore import Tensor, nn
import numpy as np
x = Tensor(np.array([[180, 234, 154], [244, 48, 247]]), mindspore.float32)
net = nn.Dense(3, 4)
output = net(x)
print(output.shape)
# (2, 4)