mindformers.core.loss.CrossEntropyLoss

class mindformers.core.loss.CrossEntropyLoss(**kwargs)

Calculate the cross entropy loss.

Args:

parallel_config (OpParallelConfig): The parallel configure. Default default_dpmp_config,

an instance of OpParallelConfig with default args.

Inputs:

• logits (Tensor) - Tensor of shape (N, C). Data type must be float16 or float32. The output logits of the backbone.

• labels (Tensor) - Tensor of shape (N, ). The ground truth label of the sample.

• input_mask (Tensor) - Tensor of shape (N, ). input_mask indicates whether there are padded inputs and for padded inputs it will not be counted into loss.

Outputs:

Tensor. The corresponding cross entropy loss.

Examples:

>>> import numpy as np
>>> from mindspore import dtype as mstype
>>> from mindspore import Tensor
>>> from mindformers.core import CrossEntropyLoss
>>> loss = CrossEntropyLoss()
>>>
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mstype.float32)
>>> labels_np = np.array([1]).astype(np.int32)
>>> input_mask = Tensor(np.ones(1).astype(np.float32))
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels, input_mask)
>>> print(output.shape)
(1,)