# Differences with torch.nn.ConvTranspose3d

torch.nn.ConvTranspose3d

class torch.nn.ConvTranspose3d(
    in_channels,
    out_channels,
    kernel_size,
    stride=1,
    padding=0,
    output_padding=0,
    groups=1,
    bias=True,
    dilation=1,
    padding_mode='zeros'
)(input) -> Tensor

For more information, see torch.nn.ConvTranspose3d.

mindspore.nn.Conv3dTranspose

class mindspore.nn.Conv3dTranspose(
    in_channels,
    out_channels,
    kernel_size,
    stride=1,
    pad_mode='same',
    padding=0,
    dilation=1,
    group=1,
    output_padding=0,
    has_bias=False,
    weight_init=None,
    bias_init=None,
    data_format='NCDHW'
)(x) -> Tensor

For more information, see mindspore.nn.Conv3dTranspose.

Differences

PyTorch: Computing the 3D transposed convolution can be thought of as Conv3d solving for the gradient of the input, also known as deconvolution (which is not really deconvolution). The input shape is usually \((N,C_{in},D_{in},H_{in},W_{in})\), where \(N\) is the batch size, \(C\) is the spatial dimension, and \(D_{in},H_{in},W_{in}\) are the depth, height and width of the feature layer, respectively. The output shape is \((N,C_{out},D_{out},H_{out},W_{out})\), which is calculated as follows. \(D_{out}=(D_{in}-1)×stride[0]-2×padding[0]+dilation[0]×(kernel\underline{ }size[0]-1)+output\ underline{ }padding[0]+1\) \(H_{out}=(H_{in}−1)×stride[1]−2×padding[1]+dilation[1]×(kernel\underline{ }size[1]−1)+output\underline{ }padding[1]+1\) \(W_{out}=(W_{in}−1)×stride[2]−2×padding[2]+dilation[2]×(kernel\underline{ }size[2]−1)+output\underline{ }padding[2]+1\)

MindSpore: MindSpore API implements essentially the same function as PyTorch, but with the new “pad_mode” parameter. When “pad_mode” = “pad”, it is the same as the PyTorch default, and the weight_init and bias_init parameters can be used to configure the initialization method.

Weight Initialization Difference

When weight_init of mindspore.nn.Conv3dTranspose is None , the weight is initialized using HeUniform. This is the same as PyTorch weight initialization.

When bias_init of mindspore.nn.Conv3dTranspose is None , the bias is initialized using Uniform. This is the same as the PyTorch bias initialization.

Categories	Subcategories	PyTorch	MindSpore	Difference
Parameters	Parameter 1	in_channels	in_channels	-
	Parameter 2	out_channels	out_channels	-
	Parameter 3	kernel_size	kernel_size	-
	Parameter 4	stride	stride	-
	Parameter 5	padding	padding	The function is the same. PyTorch can only pad the same values on each side of the three dimensions, which can be tuples of length 3. MindSpore can set the number of padding for the front, tail, top, bottom, left and right respectively, and can be a tuples of length 6.
	Parameter 6	output_padding	output_padding	-
	Parameter 7	groups	group	Same function, different parameter names
	Parameter 8	bias	has_bias	PyTorch defaults to True, and MindSpore defaults to False
	Parameter 9	dilation	dilation	-
	Parameter 10	padding_mode	-	Numeric padding mode, only support “zeros” that is, padding 0. MindSpore does not have this parameter, but the default padding 0
	Parameter 11	-	pad_mode	Specify the padding mode. Optional values are “same”, “valid”, “pad”, in “same” and “valid” mode. padding must be set to 0, and default is “same”. PyTorch does not have this parameter.
	Parameter 12	-	weight_init	The initialization method for the weight parameter. The type can be Tensor, str, Initializer or numbers.Number. When using str, the values of “TruncatedNormal”, “Normal”, “Uniform”, “HeUniform” and “XavierUniform” distributions and the constants “One” and “Zero” distributions can be selected. The default is None. PyTorch does not have this parameter.
	Parameter 13	-	bias_init	The initialization method for the bias parameter. Optional parameter is the same as “weight_init”, and default is None. PyTorch does not have this parameter.
	Parameter 14	-	data_format	Optional value for the data format. Currently only “NCDHW” is supported, in the same order as the default in PyTorch. PyTorch does not have this parameter.
Input	Single input	input	x	Same function, different parameter names

Code Example 1

Both APIs implement 3D transposed convolutional operations and need to be instantiated first when used. To make the output the same width as the input after dividing stride, PyTorch sets output_padding = stride - 1 and padding to (kernel_size - 1)/2. MindSpore sets pad_mode = “same” and padding = 0.

# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 4, 9, 16])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose3d(3, 32, kernel_size=k, stride=s, padding=(k-1)//2, output_padding=s-1, bias=False)
net.weight.data = torch.ones(3, 32, k, k, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 32, 12, 27, 48)


# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 4, 9, 16])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv3dTranspose(3, 32, kernel_size=k, stride=s, weight_init='ones', pad_mode='same')
output = net(x)
print(output.shape)
# (1, 32, 12, 27, 48)

Code Example 2

Both APIs implement 3D transposed convolutional operations and need to be instantiated first when used. If you do not do any padding on the original image, you may discard part of the data if stride > 1. Set padding and output_padding to 0 in PyTorch, and set pad_mode = “valid” in MindSpore, while padding = 0.

# PyTorch
import torch
from torch import tensor
import torch.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 4, 9, 16])
x = tensor(x_, dtype=torch.float32)
net = nn.ConvTranspose3d(3, 32, kernel_size=k, stride=s, bias=False)
net.weight.data = torch.ones(3, 32, k, k, k)
output = net(x).detach().numpy()
print(output.shape)
# (1, 32, 14, 29, 50)


# MindSpore
import mindspore as ms
import mindspore.nn as nn
import numpy as np

k = 5
s = 3
x_ = np.ones([1, 3, 4, 9, 16])
x = ms.Tensor(x_, ms.float32)
net = nn.Conv3dTranspose(3, 32, kernel_size=k, stride=s, weight_init='ones', pad_mode='valid')
output = net(x)
print(output.shape)
# (1, 32, 14, 29, 50)