Differences with torch.nn.ConvTranspose1d
torch.nn.ConvTranspose1d
class torch.nn.ConvTranspose1d(
in_channels,
out_channels,
kernel_size,
stride=1,
padding=0,
output_padding=0,
groups=1,
bias=True,
dilation=1,
padding_mode='zeros'
)(input) -> Tensor
For more information, see torch.nn.ConvTranspose1d.
mindspore.nn.Conv1dTranspose
class mindspore.nn.Conv1dTranspose(
in_channels,
out_channels,
kernel_size,
stride=1,
pad_mode='same',
padding=0,
dilation=1,
group=1,
has_bias=False,
weight_init=None,
bias_init=None
)(x) -> Tensor
For more information, see mindspore.nn.Conv1dTranspose.
Differences
PyTorch: Computing a one-dimensional transposed convolution can be thought of as Conv1d solving for the gradient of the input, also known as deconvolution (which is not actually true deconvolution). The input shape is usually \((N,C_{in}, L_{in})\), where \(N\) is the batch size, \(C\) is the spatial dimension, and \(L\) is the length of the sequence. The output shape is \((N,C_{out},L_{out})\), where \(L_{out}=(L_{in}-1)×stride-2×padding+dilation×(kernel\_size-1)+output\_padding+1\).
MindSpore: MindSpore API implements basically the same function as PyTorch, but with the new "pad_mode" parameter. When "pad_mode" = "pad", it is the same as the PyTorch default, and the weight_init and bias_init parameters can be used to configure the initialization method.
Weight Initialization Difference
When weight_init
of mindspore.nn.Conv1dTranspose is None
, the weight is initialized using HeUniform. This is the same as PyTorch weight initialization.
When bias_init
of mindspore.nn.Conv1dTranspose is None
, the bias is initialized using Uniform. This is the same as the PyTorch bias initialization.
Categories |
Subcategories |
PyTorch |
MindSpore |
Difference |
---|---|---|---|---|
Parameters |
Parameter 1 |
in_channels |
in_channels |
- |
Parameter 2 |
out_channels |
out_channels |
- |
|
Parameter 3 |
kernel_size |
kernel_size |
- |
|
Parameter 4 |
stride |
stride |
- |
|
Parameter 5 |
padding |
padding |
- |
|
Parameter 6 |
output_padding |
- |
Usually used with stride > 1 to adjust output shapes. For example, it is common to set padding to (kernel_size - 1)/2, where setting output_padding = (stride - 1) ensures that input shapes/output shapes = stride. MindSpore does not have this parameter |
|
Parameter 7 |
groups |
group |
Same function, different parameter names |
|
Parameter 8 |
bias |
has_bias |
PyTorch defaults to True, and MindSpore defaults to False |
|
Parameter 9 |
dilation |
dilation |
- |
|
Parameter 10 |
padding_mode |
- |
Numeric padding mode, only supports "zeros" i.e. padding 0. MindSpore does not have this parameter, but pads 0 by default |
|
Parameter 11 |
- |
pad_mode |
Specify the padding mode. Optional values are "same", "valid", "pad". In "same" and "valid" mode, padding must be set to 0, and default is "same". PyTorch does not have this parameter |
|
Parameter 12 |
- |
weight_init |
The initialization method for the weight parameter. The type can be Tensor, str, Initializer or numbers.Number. When using str, the values of "TruncatedNormal", "Normal", "Uniform", "HeUniform" and "XavierUniform" distributions and the constants "One" and "Zero" distributions can be selected. The default is None. PyTorch does not have this parameter |
|
Parameter 13 |
- |
bias_init |
The initialization method for the bias parameter. The initialization method is the same as "weight_init", and the default is None. PyTorch does not have this parameter |
|
Input |
Single input |
input |
x |
Same function, different parameter names |
Code Example 1
Both APIs implement one-dimensional transposed convolutional operations and need to be instantiated first when used. When PyTorch sets output_padding to 0 and MindSpore sets pad_mode to "pad", the output width is \(L_{out}=(L_{in}-1)×stride-2×padding+dilation×( kernel\_size-1)+1\). The weights are initialized to 1 in PyTorch through net.weight.data = torch.ones(), and MindSpore sets the parameter weight_init = "ones" directly.
# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np
k = 4
x_ = np.ones([1, 3, 50])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose1d(3, 64, kernel_size=k, stride=1, padding=0, output_padding=0, bias=False)
net.weight.data = torch.ones(3, 64, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 64, 53)
# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np
k = 4
x_ = np.ones([1, 3, 50])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv1dTranspose(3, 64, kernel_size=k, weight_init='ones', pad_mode='pad')
output = net(x)
print(output.shape)
# (1, 64, 53)
Code Example 2
To make the output the same width as the input after dividing stride, PyTorch sets output_padding = stride - 1 and padding to (kernel_size - 1)/2. MindSpore sets pad_mode = "same" and padding = 0.
# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np
k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose1d(3, 64, kernel_size=k, stride=s, padding=(k-1)//2, output_padding=s-1, bias=False)
net.weight.data = torch.ones(3, 64, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 64, 150)
# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np
k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv1dTranspose(3, 64, kernel_size=k, stride=s, weight_init='ones', pad_mode='same', padding=0)
output = net(x)
print(output.shape)
# (1, 64, 150)
Code Example 3
If no padding is done on the original image, a part of the data may be discarded in the case of stride>1. The output width is \(L_{out}=(L_{in}-1)×stride+dilation×(kernel\_size-1)+1\). Set padding and output_padding to 0 in PyTorch and set pad_mode = "valid" in MindSpore.
# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np
k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose1d(3, 64, kernel_size=k, stride=s, padding=0, output_padding=0, bias=False)
net.weight.data = torch.ones(3, 64, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 64, 152)
# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np
k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv1dTranspose(3, 64, kernel_size=k, stride=s, weight_init='ones', pad_mode='valid', padding=0)
output = net(x)
print(output.shape)
# (1, 64, 152)