mindspore.nn.CrossEntropyLoss

class mindspore.nn.CrossEntropyLoss(weight=None, ignore_index=- 100, reduction='mean', label_smoothing=0.0)[source]

The cross entropy loss between input and target.

The CrossEntropyLoss support two kind of targets:

  • Class indices (int) in the range [0,C) where C is the number of classes, the loss with reduction=none can be described as:

    (x,y)=L={l1,,lN},ln=wynlogexp(xn,yn)c=1Cexp(xn,c)1{ynignore_index}

    where x is the inputs, t is the target, w is the weight, N is the batch size, c belonging to [0, C-1] is class index, where C is the number of classes.

    If reduction is not ‘none’ (default ‘mean’), then

    (x,y)={n=1N1n=1Nwyn1{ynignore_index}ln,if reduction='mean',n=1Nln,if reduction='sum'.
  • Probabilities (float) for each class, useful when labels beyond a single class per minibatch item are required, the loss with reduction=none can be described as:

    (x,y)=L={l1,,lN},ln=c=1Cwclogexp(xn,c)i=1Cexp(xn,i)yn,c

    where x is the inputs, t is the target, w is the weight, N is the batch size, c belonging to [0, C-1] is class index, where C is the number of classes.

    If reduction is not ‘none’ (default ‘mean’), then

    (x,y)={n=1NlnN,if reduction='mean',n=1Nln,if reduction='sum'.
Parameters
  • weight (Tensor) – The rescaling weight to each class. If the value is not None, the shape is (C,). The data type only supports float32 or float16. Default: None.

  • ignore_index (int) – Specifies a target value that is ignored (typically for padding value) and does not contribute to the gradient. Default: -100.

  • reduction (str) – Apply specific reduction method to the output: ‘none’, ‘mean’, or ‘sum’. Default: ‘mean’.

  • label_smoothing (float) – Label smoothing values, a regularization tool used to prevent the model from overfitting when calculating Loss. The value range is [0.0, 1.0]. Default value: 0.0.

Inputs:
  • logits (Tensor) - Tensor of shape (C,) (N,C) or (N,C,d1,d2,...,dK) , where C = number of classes. Data type must be float16 or float32.

  • labels (Tensor) - For class indices, tensor of shape (), (N) or (N,d1,d2,...,dK) , data type must be int32. For probabilities, tensor of shape (C,) (N,C) or (N,C,d1,d2,...,dK) , data type must be float16 or float32.

Returns

Tensor, the computed cross entropy loss value.

Raises
  • TypeError – If weight is not a Tensor.

  • TypeError – If ignore_index is not an int.

  • TypeError – If the data type of weight is not float16 or float32.

  • ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

  • TypeError – If label_smoothing is not a float.

  • TypeError – If logits is not a Tensor.

  • TypeError – If labels is not a Tensor.

Supported Platforms:

Ascend GPU CPU

Examples

>>> # Case 1: Indices labels
>>> inputs = mindspore.Tensor(np.random.randn(3, 5), mindspore.float32)
>>> target = mindspore.Tensor(np.array([1, 0, 4]), mindspore.int32)
>>> loss = nn.CrossEntropyLoss()
>>> output = loss(inputs, target)
>>> # Case 2: Probability labels
>>> inputs = mindspore.Tensor(np.random.randn(3, 5), mindspore.float32)
>>> target = mindspore.Tensor(np.random.randn(3, 5), mindspore.float32)
>>> loss = nn.CrossEntropyLoss()
>>> output = loss(inputs, target)