mindspore.nn.SoftmaxCrossEntropyWithLogits

class mindspore.nn.SoftmaxCrossEntropyWithLogits(sparse=False, reduction='none')[source]

Computes softmax cross entropy between logits and labels.

Measures the distribution error between the probabilities of the input (computed with softmax function) and the labels where the classes are mutually exclusive (only one class is positive) using cross entropy loss.

Typical input into this function is unnormalized scores denoted as x whose shape is (N, C), and the corresponding targets.

Typically, the input to this function is the fractional value of each category and the corresponding target value, and the input format is (N, C).

For each instance \(x_i\), i ranges from 0 to N-1, the loss is given as:

\[\ell(x_i, c) = - \log\left(\frac{\exp(x_i[c])}{\sum_j \exp(x_i[j])}\right) = -x_i[c] + \log\left(\sum_j \exp(x_i[j])\right)\]

where \(x_i\) is a 1D score Tensor, \(c\) is the index of 1 in one-hot.

Note

While the labels classes are mutually exclusive, i.e., only one class is positive in the labels, the predicted probabilities does not need to be exclusive. It is only required that the predicted probability distribution of entry is a valid one.

Parameters

  • sparse (bool) – Specifies whether labels use sparse format or not. Default: False.

  • reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.

Inputs:

  • logits (Tensor) - Tensor of shape (N, C). Data type must be float16 or float32.

  • labels (Tensor) - Tensor of shape (N, ). If sparse is True, The type of labels is int32 or int64. Otherwise, the type of labels is the same as the type of logits.

Outputs:

Tensor, a tensor of the same shape and type as logits with the component-wise logistic losses.

Raises

  • TypeError – If sparse is not a bool.

  • TypeError – If sparse is True and dtype of labels is neither int32 not int64.

  • TypeError – If sparse is False and dtype of labels is neither float16 not float32.

  • ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:

Ascend GPU CPU

Examples

>>> # case 1: sparse=True
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True)
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)
>>> labels_np = np.array([1]).astype(np.int32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[67.]
>>> # case 2: sparse=False
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=False)
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)
>>> labels_np = np.array([[0, 0, 0, 0, 0, 0, 1, 0, 0, 0]]).astype(np.float32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[30.]