Document feedback

Question document fragment

When a question document fragment contains a formula, it is displayed as a space.

Submission type
issue

It's a little complicated...

I'd like to ask someone.

PR

Just a small problem.

I can fix it online!

Please select the submission type

Problem type
Specifications and Common Mistakes

- Specifications and Common Mistakes:

- Misspellings or punctuation mistakes,incorrect formulas, abnormal display.

- Incorrect links, empty cells, or wrong formats.

- Chinese characters in English context.

- Minor inconsistencies between the UI and descriptions.

- Low writing fluency that does not affect understanding.

- Incorrect version numbers, including software package names and version numbers on the UI.

Usability

- Usability:

- Incorrect or missing key steps.

- Missing main function descriptions, keyword explanation, necessary prerequisites, or precautions.

- Ambiguous descriptions, unclear reference, or contradictory context.

- Unclear logic, such as missing classifications, items, and steps.

Correctness

- Correctness:

- Technical principles, function descriptions, supported platforms, parameter types, or exceptions inconsistent with that of software implementation.

- Incorrect schematic or architecture diagrams.

- Incorrect commands or command parameters.

- Incorrect code.

- Commands inconsistent with the functions.

- Wrong screenshots.

- Sample code running error, or running results inconsistent with the expectation.

Risk Warnings

- Risk Warnings:

- Lack of risk warnings for operations that may damage the system or important data.

Content Compliance

- Content Compliance:

- Contents that may violate applicable laws and regulations or geo-cultural context-sensitive words and expressions.

- Copyright infringement.

Problem description

Agree to Privacy Statement

mindspore.ops.KLDivLoss

View Source On Gitee
class mindspore.ops.KLDivLoss(reduction='mean')[source]

Computes the Kullback-Leibler divergence between the logits and the labels.

For tensors of the same shape x and target, the updating formulas of KLDivLoss algorithm are as follows,

L(x,target)=target(logtargetx)

Then,

(x,target)={L(x,target),if reduction='none';mean(L(x,target)),if reduction='mean';sum(L(x,target))/x.shape[0],if reduction='batchmean';sum(L(x,target)),if reduction='sum'.

where x represents logits, target represents labels, and (x,target) represents output.

Note

  • On Ascend, float64 dtype is not currently supported.

  • The output aligns with the mathematical definition of Kullback-Leibler divergence only when reduction is set to 'batchmean'.

  • On Ascend, the value of reduction must be one of 'batchmean', 'none' or 'sum'.

  • On GPU, the value of reduction must be one of 'mean', 'none' or 'sum'.

  • On CPU, the value of reduction must be one of 'mean', 'batchmean', 'none' or 'sum'.

Parameters

reduction (str) –

Specifies the reduction to be applied to the output. Default: 'mean' .

  • 'none': no reduction will be applied.

  • 'mean': compute and return the mean of elements in the output.

  • 'sum': the output elements will be summed.

  • 'batchmean': average loss is taken over the batch, similar to the mean mode.

Inputs:
  • logits (Tensor) - The input Tensor. The data type must be float16, float32 or float64.

  • labels (Tensor) - The label Tensor which has the same shape and data type as logits.

Outputs:

Tensor or Scalar, if reduction is 'none', then output is a tensor and has the same shape as logits. Otherwise it is a scalar.

Raises
  • TypeError – If reduction is not a str.

  • TypeError – If neither logits nor labels is a Tensor.

  • TypeError – If dtype of logits or labels is not currently supported.

  • ValueError – If shape of logits is not the same as labels.

  • RuntimeError – If logits or labels is a scalar when reduction is 'batchmean'.

Supported Platforms:

Ascend GPU CPU

Examples

>>> import mindspore
>>> import numpy as np
>>> from mindspore import Tensor, nn, ops
>>> class Net(nn.Cell):
...     def __init__(self):
...         super(Net, self).__init__()
...         self.kldiv_loss = ops.KLDivLoss(reduction='sum')
...     def construct(self, logits, labels):
...         result = self.kldiv_loss(logits, labels)
...         return result
...
>>> net = Net()
>>> logits = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32)
>>> labels = Tensor(np.array([0., 1., 0.]), mindspore.float32)
>>> output = net(logits, labels)
>>> print(output)
-0.7