mindspore.ops.kl_div

mindspore.ops.kl_div(logits, labels, reduction='mean')[source]

Computes the Kullback-Leibler divergence between the logits and the labels.

The updating formulas of KLDivLoss algorithm are as follows,

\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = target_n \cdot (\log target_n - x_n)\]

Then,

\[\begin{split}\ell(x, target) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{batchmean}(L), & \text{if reduction} = \text{'batchmean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

where \(x\) represents logits. \(target\) represents labels. \(\ell(x, target)\) represents output.

Note

  • Currently it does not support float64 input on Ascend.

  • The output aligns with the mathematical definition of Kullback-Leibler divergence only when reduction is set to ‘batchmean’.

Parameters

  • logits (Tensor) – The input Tensor. The data type must be float16, float32 or float64.

  • labels (Tensor) – The label Tensor which has the same shape and data type as logits.

  • reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘batchmean’ or ‘sum’. Default: ‘mean’.

Returns

Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as logits. Otherwise, it is a scalar.

Raises

  • TypeError – If reduction is not a str.

  • TypeError – If neither logits nor labels is a Tensor.

  • TypeError – If dtype of logits or labels is not float32.

Supported Platforms:

Ascend GPU CPU

Examples

>>> logits = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32)
>>> labels = Tensor(np.array([0., 1., 0.]), mindspore.float32)
>>> output = mindspore.ops.kl_div(logits, labels, 'mean')
>>> print(output)
-0.23333333