mindspore.ops.kl_div

mindspore.ops.kl_div(logits, labels, reduction='mean')

Computes the Kullback-Leibler divergence between the logits and the labels.

The updating formulas of KLDivLoss algorithm are as follows,

$$L=\{l_1,\dots ,l_N{\}}^{\mathrm{\top }},l_n=targe{t}_n\cdot (\log targe{t}_n-x_n)$$

Then,

$$\begin{array}{r}\ell (x,target)=\{\begin{array}{ll}L,& \text{if reduction}=\text{'none';}\\ \mathrm{mean}(L),& \text{if reduction}=\text{'mean';}\\ \mathrm{batchmean}(L),& \text{if reduction}=\text{'batchmean';}\\ \mathrm{sum}(L),& \text{if reduction}=\text{'sum'.}\end{array}\end{array}$$

where $x$ represents logits. $target$ represents labels. $\ell (x,target)$ represents output.

Parameters

• logits (Tensor) – The input Tensor. The data type must be float16, float32 or float64.

• labels (Tensor) – The label Tensor which has the same shape and data type as logits.

• reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘batchmean’ or ‘sum’. Default: ‘mean’.

Returns

Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as logits. Otherwise, it is a scalar.

Raises

• TypeError – If reduction is not a str.

• TypeError – If neither logits nor labels is a Tensor.

• TypeError – If dtype of logits or labels is not float32.

Supported Platforms:

Ascend GPU CPU

Note

Currently it does not support float64 input on Ascend. It behaves the same as the mathematical definition only when reduction is set to batchmean.

Examples

>>> logits = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32)
>>> labels = Tensor(np.array([0., 1., 0.]), mindspore.float32)
>>> output = mindspore.ops.kl_div(logits, labels, 'mean')
>>> print(output)
-0.23333333