mindspore.mint.nn.functional.batch_norm

View Source On Gitee
mindspore.mint.nn.functional.batch_norm(input, running_mean, running_var, weight=None, bias=None, training=False, momentum=0.1, eps=1e-05)[source]

Batch Normalization for input data and updated parameters.

Batch Normalization is widely used in convolutional neural networks. This operation applies Batch Normalization over inputs to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the features using a mini-batch of data and the learned parameters can be described in the following formula,

\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]

where \(\gamma\) is weight, \(\beta\) is bias, \(\epsilon\) is eps, \(mean\) is the mean of \(x\), \(variance\) is the variance of \(x\).

Parameters

  • input (Tensor) – Tensor of shape \((N, C, *)\), with bfloat16, float16 or float32 data type. For Atlas training products, the shape must be 2-4 dimensions currently.

  • running_mean (Tensor) – The shape \((C,)\), with bfloat, float16 or float32 data type.

  • running_var (Tensor) – The shape \((C,)\), with bfloat, float16 or float32 data type.

  • weight (Tensor, optional) – The shape \((C,)\), with bfloat, float16 or float32 data type, Default: None. Initialized to 1 when weight is None.

  • bias (Tensor, optional) – The shape \((C,)\), with bfloat, float16 or float32 data type. Default: None. Initialized to 0 when weight is None.

  • training (bool, optional) – If training is True, mean and variance are computed during training. If training is False, they're loaded from checkpoint during inference. Default: False .

  • momentum (float, optional) – The hyper parameter to compute moving average for running_mean and running_var (e.g. \(new\_running\_mean = (1 - momentum) * running\_mean + momentum * current\_mean\)). Default: 0.1 .

  • eps (float, optional) – A small value added for numerical stability. Default: 1e-5.

Returns

Tensor, has the same type and shape as input. The shape is \((N, C, *)\).

Raises

  • TypeError – If training is not a bool.

  • TypeError – If dtype of eps or momentum is not float.

  • TypeError – If input, weight, bias, running_mean or running_var is not a Tensor.

Supported Platforms:

Ascend

Examples

>>> import mindspore
>>> from mindspore import Tensor, mint
>>> input_x = Tensor([[1.0, 2.0], [3.0, 4.0]], mindspore.float32)
>>> running_mean = Tensor([0.5, 1.5], mindspore.float32)
>>> running_var = Tensor([0.1, 0.2], mindspore.float32)
>>> weight = Tensor([2.0, 2.0], mindspore.float32)
>>> bias = Tensor([-1.0, -1.0], mindspore.float32)
>>> output = mint.nn.functional.batch_norm(input_x, running_mean, running_var, weight, bias)
>>> print(output)
[[ 2.1621194  1.2360122]
 [14.810596  10.180061 ]]