mindspore.ops.batch_norm

mindspore.ops.batch_norm(input_x, running_mean, running_var, weight, bias, training=False, momentum=0.1, eps=1e-05)[source]

Batch Normalization for input data and updated parameters.

Batch Normalization is widely used in convolutional neural networks. This operation applies Batch Normalization over inputs to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the features using a mini-batch of data and the learned parameters can be described in the following formula,

\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]

where \(\gamma\) is weight, \(\beta\) is bias, \(\epsilon\) is eps, \(mean\) is the mean of input_x, \(variance\) is the variance of input_x.

Warning

  • If this operation is used for inferring and output “reserve_space_1” and “reserve_space_2” are usable,

    then “reserve_space_1” and “reserve_space_2” have the same value as “mean” and “variance” respectively.

  • For Ascend 310, the result accuracy fails to reach 1‰ due to the square root instruction.

Note

  • If training is False, weight, bias, running_mean and running_var are Tensors.

  • If training is True, weight, bias, running_mean and running_var are Parameters.

Parameters

  • input_x (Tensor) – Tensor of shape \((N, C)\), with float16 or float32 data type.

  • running_mean (Union[Tensor, Parameter]) – The shape \((C,)\), has the same data type with weight.

  • running_var (Union[Tensor, Parameter]) – The shape \((C,)\), has the same data type with weight.

  • weight (Union[Tensor, Parameter]) – The shape \((C,)\), with float16 or float32 data type.

  • bias (Union[Tensor, Parameter]) – The shape \((C,)\), has the same data type with weight.

  • training (bool) – If training is True, mean and variance are computed during training. If training is False, they’re loaded from checkpoint during inference. Default: False.

  • momentum (float) – The hyper parameter to compute moving average for running_mean and running_var (e.g. \(new\_running\_mean = (1 - momentum) * running\_mean + momentum * current\_mean\)). Momentum value must be [0, 1]. Default: 0.1.

  • eps (float) – A small value added for numerical stability. Default: 1e-5.

Returns

output_x (Tensor) - The same type and shape as the input_x. The shape is \((N, C)\).

Raises

  • TypeError – If training is not a bool.

  • TypeError – If dtype of eps or momentum is not float.

  • TypeError – If input_x, weight, bias, running_mean or running_var is not a Tensor.

  • TypeError – If dtype of input_x, weight is neither float16 nor float32.

Supported Platforms:

Ascend CPU GPU

Examples

>>> input_x = Tensor(np.ones([2, 2]), mindspore.float32)
>>> running_mean = Tensor(np.ones([2]), mindspore.float32)
>>> running_var = Tensor(np.ones([2]), mindspore.float32)
>>> weight = Tensor(np.ones([2]), mindspore.float32)
>>> bias = Tensor(np.ones([2]), mindspore.float32)
>>> output = ops.batch_norm(input_x, running_mean, running_var, weight, bias)
>>> print(output)
[[1. 1.]
 [1. 1.]]