Function Differences with torch.nn.Linear

torch.nn.Linear

torch.nn.Linear(
    in_features,
    out_features,
    bias=True
)

For more information, see torch.nn.Linear.

mindspore.nn.Dense

class mindspore.nn.Dense(
    in_channels,
    out_channels,
    weight_init='normal',
    bias_init='zeros',
    has_bias=True,
    activation=None
)(input)

For more information, see mindspore.nn.Dense.

Differences

Pytorch: Applies a linear transformation to the incoming data.

MindSpore: Applies a linear transformation to the incoming data, and applies the activation function before outputting the data.

Code Example

import mindspore as ms
from mindspore import nn
import torch
import numpy as np

# In MindSpore, default weight will be initialized through standard normal distribution.
# Default bias will be initialized by zero.
# Default none activation used.
input_net = ms.Tensor(np.array([[180, 234, 154], [244, 48, 247]]), ms.float32)
net = nn.Dense(3, 4)
output = net(input_net)
print(output.shape)
# Out：
# (2, 4)

# In torch, default weight and bias will be initialized through uniform distribution.
# No parameter to set the activation.
input_net = torch.Tensor(np.array([[180, 234, 154], [244, 48, 247]]))
net = torch.nn.Linear(3, 4)
output = net(input_net)
print(output.shape)
# Out：
# torch.Size([2, 4])