mindspore.ops.deformable_conv2d

mindspore.ops.deformable_conv2d(x, weight, offsets, kernel_size, strides, padding, bias=None, dilations=(1, 1, 1, 1), groups=1, deformable_groups=1, modulated=True)[source]

Given 4D tensor inputs x, weight and offsets, compute a 2D deformable convolution. The deformable convolution operation can be expressed as follow:

Deformable Convolution v1:

\[y(p)=\sum_{k=1}^{K}w_{k}\cdot x(p+p_{k}+\Delta{p_{k}})\]

Deformable Convolution v2:

\[y(p)=\sum_{k=1}^{K}w_{k}\cdot x(p+p_{k}+\Delta{p_{k}})\cdot \Delta{m_{k}}\]

Where \(\Delta{p_{k}}\) and \(\Delta{m_{k}}\) are the learnable offset and modulation scalar for the k-th location. For details, please refer to Deformable ConvNets v2: More Deformable, Better Results and Deformable Convolutional Networks.

Parameters

  • x (Tensor) – A 4D tensor of input image. With the format “NCHW”, the shape is \((N, C_{in}, H_{in}, W_{in})\). Dtype: float16 or float32.

  • weight (Tensor) – A 4D tensor of learnable filters. Must have the same type as x. The shape is \((C_{out}, C_{in} / groups, H_{f}, W_{f})\).

  • offsets (Tensor) – A 4D tensor of x-y coordinates offset and mask. With the format “NCHW”, the shape is \((batch, 3 * deformable\_groups * H_{f} * W_{f}, H_{out}, W_{out})\). Note the C dimension is stored in the order of (offset_x, offset_y, mask). Must have the same type as x.

  • kernel_size (tuple[int]) – A tuple of 2 integers. The size of kernel.

  • strides (tuple[int]) – A tuple of 4 integers. The stride of the sliding window for each dimension of input. The dimension order is interpreted according to the data format of x. The N and C dimensions must be set to 1.

  • padding (tuple[int]) – A tuple of 4 integers. The number of pixels to add to each (top, bottom, left, right) side of the input.

  • bias (Tensor, optional) – An 1D tensor of additive biases to the filter outputs. The shape is \((C_{out})\). Defaults to None.

  • dilations (tuple[int], optional) – A tuple of 4 integers. The dilation factor for each dimension of input. The dimension order is interpreted according to the data format of x. The N and C dimensions must be set to 1. Defaults to (1, 1, 1, 1).

  • groups (int, optional) – An integer of type int32. The number of blocked connections from input channels to output channels. In_channels and out_channels must both be divisible by groups. Defaults to 1.

  • deformable_groups (int, optional) – An integer of type int32. The number of deformable group partitions. In_channels must be divisible by deformable_groups. Defaults to 1.

  • modulated (bool, optional) – Specifies version of DeformableConv2D, True means v2, False means v1, currently only supports v2. Defaults to True.

Returns

Tensor, A 4D Tensor of output feature map. With the same type as x. With the format “NCHW”, the shape is \((N, C_{out}, H_{out}, W_{out})\).

\[\begin{split}\begin{array}{ll} \\ H_{out} = \left \lfloor{\frac{H_{in} + padding[0] + padding[1] - (H_{f} - 1) \times \text{dilations[2]} - 1 }{\text{stride[0]}} + 1} \right \rfloor \\ W_{out} = \left \lfloor{\frac{W_{in} + padding[2] + padding[3] - (W_{f} - 1) \times \text{dilations[3]} - 1 }{\text{stride[1]}} + 1} \right \rfloor \\ \end{array}\end{split}\]

Raises

  • TypeError – If strides, padding, kernel_size or dilations is not a tuple with integer elements.

  • TypeError – If modulated is not a bool.

  • ValueError – If the tuple size of strides, padding, kernel_size or dilations is not expected.

  • ValueError – The N or C dimensions of ‘strides’ or dilations is not set to 1.

  • ValueError – If modulated is not set to True.

Note

  • This is an experimental interface that is subject to change or deletion.

  • For Ascend platform, only AI-CORE kernel is implemented, which has the following limitations:

    • \(C_{in}\) cannot be divisible by 8 is not supported, e.g. x is \((N, 2, H_{in}, W_{in})\).

    • deformable_groups must equal to 1.

    • offsets value is float which does not contain a decimal part is not supported, e.g. offsets is assigned with “numpy.ones()”.

    • kernel_size should meet the requirement:\(3 * kernel\_size[0] * kernel\_size[1] > 8\).

Supported Platforms:

Ascend GPU CPU

Examples

>>> x = Tensor(np.ones((4, 3, 10, 10)), mstype.float32)
>>> kh, kw = 3, 3
>>> weight = Tensor(np.ones((5, 3, kh, kw)), mstype.float32)
>>> offsets = Tensor(np.ones((4, 3 * kh * kw, 8, 8)), mstype.float32)
>>> output = ops.deformable_conv2d(x, weight, offsets, (kh, kw), (1, 1, 1, 1), (0, 0, 0, 0))
>>> print(output.shape)
(4, 5, 8, 8)