Differences with torch.nn.ConvTranspose1d

View Source On Gitee

torch.nn.ConvTranspose1d

class torch.nn.ConvTranspose1d(
    in_channels,
    out_channels,
    kernel_size,
    stride=1,
    padding=0,
    output_padding=0,
    groups=1,
    bias=True,
    dilation=1,
    padding_mode='zeros'
)(input) -> Tensor

For more information, see torch.nn.ConvTranspose1d.

mindspore.nn.Conv1dTranspose

class mindspore.nn.Conv1dTranspose(
    in_channels,
    out_channels,
    kernel_size,
    stride=1,
    pad_mode='same',
    padding=0,
    dilation=1,
    group=1,
    has_bias=False,
    weight_init=None,
    bias_init=None
)(x) -> Tensor

For more information, see mindspore.nn.Conv1dTranspose.

Differences

PyTorch: Computing a one-dimensional transposed convolution can be thought of as Conv1d solving for the gradient of the input, also known as deconvolution (which is not actually true deconvolution). The input shape is usually \((N,C_{in}, L_{in})\), where \(N\) is the batch size, \(C\) is the spatial dimension, and \(L\) is the length of the sequence. The output shape is \((N,C_{out},L_{out})\), where \(L_{out}=(L_{in}-1)×stride-2×padding+dilation×(kernel\_size-1)+output\_padding+1\).

MindSpore: MindSpore API implements basically the same function as PyTorch, but with the new “pad_mode” parameter. When “pad_mode” = “pad”, it is the same as the PyTorch default, and the weight_init and bias_init parameters can be used to configure the initialization method.

Weight Initialization Difference

When weight_init of mindspore.nn.Conv1dTranspose is None , the weight is initialized using HeUniform. This is the same as PyTorch weight initialization.

When bias_init of mindspore.nn.Conv1dTranspose is None , the bias is initialized using Uniform. This is the same as the PyTorch bias initialization.

Categories

Subcategories

PyTorch

MindSpore

Difference

Parameters

Parameter 1

in_channels

in_channels

-

Parameter 2

out_channels

out_channels

-

Parameter 3

kernel_size

kernel_size

-

Parameter 4

stride

stride

-

Parameter 5

padding

padding

-

Parameter 6

output_padding

-

Usually used with stride > 1 to adjust output shapes. For example, it is common to set padding to (kernel_size - 1)/2, where setting output_padding = (stride - 1) ensures that input shapes/output shapes = stride. MindSpore does not have this parameter

Parameter 7

groups

group

Same function, different parameter names

Parameter 8

bias

has_bias

PyTorch defaults to True, and MindSpore defaults to False

Parameter 9

dilation

dilation

-

Parameter 10

padding_mode

-

Numeric padding mode, only supports “zeros” i.e. padding 0. MindSpore does not have this parameter, but pads 0 by default

Parameter 11

-

pad_mode

Specify the padding mode. Optional values are “same”, “valid”, “pad”. In “same” and “valid” mode, padding must be set to 0, and default is “same”. PyTorch does not have this parameter

Parameter 12

-

weight_init

The initialization method for the weight parameter. The type can be Tensor, str, Initializer or numbers.Number. When using str, the values of “TruncatedNormal”, “Normal”, “Uniform”, “HeUniform” and “XavierUniform” distributions and the constants “One” and “Zero” distributions can be selected. The default is None. PyTorch does not have this parameter

Parameter 13

-

bias_init

The initialization method for the bias parameter. The initialization method is the same as “weight_init”, and the default is None. PyTorch does not have this parameter

Input

Single input

input

x

Same function, different parameter names

Code Example 1

Both APIs implement one-dimensional transposed convolutional operations and need to be instantiated first when used. When PyTorch sets output_padding to 0 and MindSpore sets pad_mode to “pad”, the output width is \(L_{out}=(L_{in}-1)×stride-2×padding+dilation×( kernel\_size-1)+1\). The weights are initialized to 1 in PyTorch through net.weight.data = torch.ones(), and MindSpore sets the parameter weight_init = “ones” directly.

# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np

k = 4
x_ = np.ones([1, 3, 50])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose1d(3, 64, kernel_size=k, stride=1, padding=0, output_padding=0, bias=False)
net.weight.data = torch.ones(3, 64, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 64, 53)


# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np

k = 4
x_ = np.ones([1, 3, 50])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv1dTranspose(3, 64, kernel_size=k, weight_init='ones', pad_mode='pad')
output = net(x)
print(output.shape)
# (1, 64, 53)

Code Example 2

To make the output the same width as the input after dividing stride, PyTorch sets output_padding = stride - 1 and padding to (kernel_size - 1)/2. MindSpore sets pad_mode = “same” and padding = 0.

# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose1d(3, 64, kernel_size=k, stride=s, padding=(k-1)//2, output_padding=s-1, bias=False)
net.weight.data = torch.ones(3, 64, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 64, 150)


# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv1dTranspose(3, 64, kernel_size=k, stride=s, weight_init='ones', pad_mode='same', padding=0)
output = net(x)
print(output.shape)
# (1, 64, 150)

Code Example 3

If no padding is done on the original image, a part of the data may be discarded in the case of stride>1. The output width is \(L_{out}=(L_{in}-1)×stride+dilation×(kernel\_size-1)+1\). Set padding and output_padding to 0 in PyTorch and set pad_mode = “valid” in MindSpore.

# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose1d(3, 64, kernel_size=k, stride=s, padding=0, output_padding=0, bias=False)
net.weight.data = torch.ones(3, 64, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 64, 152)


# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 50])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv1dTranspose(3, 64, kernel_size=k, stride=s, weight_init='ones', pad_mode='valid', padding=0)
output = net(x)
print(output.shape)
# (1, 64, 152)