mindspore.nn.GroupNorm

class mindspore.nn.GroupNorm(num_groups, num_channels, eps=1e-05, affine=True, gamma_init='ones', beta_init='zeros')[source]

Group Normalization over a mini-batch of inputs.

Group Normalization is widely used in recurrent neural networks. It applies normalization on a mini-batch of inputs for each single training case as described in the paper Group Normalization. Group Normalization divides the channels into groups and computes within each group the mean and variance for normalization, and it performs very stable over a wide range of batch size. It can be described using the following formula:

\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]
Parameters

  • num_groups (int) – The number of groups to be divided along the channel dimension.

  • num_channels (int) – The number of input channels.

  • eps (float) – A value added to the denominator for numerical stability. Default: 1e-05 .

  • affine (bool) – A bool value, this layer will have learnable affine parameters when set to true . Default: True .

  • gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including 'zeros' , 'ones' , 'xavier_uniform' , 'he_uniform' , etc. Default: 'ones' . If gamma_init is a Tensor, the shape must be \((num\_channels)\).

  • beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including 'zeros' , 'ones' , 'xavier_uniform' , 'he_uniform' , etc. Default: 'zeros' . If beta_init is a Tensor, the shape must be \((num\_channels)\).

Inputs:

  • x (Tensor) - The input feature with shape \((N, C, H, W)\) .

Outputs:

Tensor, the normalized and scaled offset tensor, has the same shape and data type as the x.

Raises

  • TypeError – If num_groups or num_channels is not an int.

  • TypeError – If eps is not a float.

  • TypeError – If affine is not a bool.

  • ValueError – If num_groups or num_channels is less than 1.

  • ValueError – If num_channels is not divided by num_groups.

Supported Platforms:

Ascend GPU CPU

Examples

>>> import mindspore as ms
>>> import numpy as np
>>> group_norm_op = ms.nn.GroupNorm(2, 2)
>>> x = ms.Tensor(np.ones([1, 2, 4, 4], np.float32))
>>> output = group_norm_op(x)
>>> print(output)
[[[[0. 0. 0. 0.]
   [0. 0. 0. 0.]
   [0. 0. 0. 0.]
   [0. 0. 0. 0.]]
  [[0. 0. 0. 0.]
   [0. 0. 0. 0.]
   [0. 0. 0. 0.]
   [0. 0. 0. 0.]]]]