Function Differences with torch.nn.Linear
torch.nn.Linear
torch.nn.Linear(
in_features,
out_features,
bias=True
)
For more information, see torch.nn.Linear.
mindspore.nn.Dense
class mindspore.nn.Dense(
in_channels,
out_channels,
weight_init='normal',
bias_init='zeros',
has_bias=True,
activation=None
)(input)
For more information, see mindspore.nn.Dense.
Differences
Pytorch: Applies a linear transformation to the incoming data.
MindSpore: Applies a linear transformation to the incoming data, and applies the activation
function before outputting the data.
Code Example
import mindspore as ms
from mindspore import nn
import torch
import numpy as np
# In MindSpore, default weight will be initialized through standard normal distribution.
# Default bias will be initialized by zero.
# Default none activation used.
input_net = ms.Tensor(np.array([[180, 234, 154], [244, 48, 247]]), ms.float32)
net = nn.Dense(3, 4)
output = net(input_net)
print(output.shape)
# Out:
# (2, 4)
# In torch, default weight and bias will be initialized through uniform distribution.
# No parameter to set the activation.
input_net = torch.Tensor(np.array([[180, 234, 154], [244, 48, 247]]))
net = torch.nn.Linear(3, 4)
output = net(input_net)
print(output.shape)
# Out:
# torch.Size([2, 4])