mindspore.nn.TransformerDecoder

class mindspore.nn.TransformerDecoder(decoder_layer, num_layers, norm=None)[source]

Transformer Decoder module with multi-layer stacked of TransformerDecoderLayer, including multihead self attention, cross attention and feedforward layer.

Parameters
Inputs:
  • tgt (Tensor): The sequence to the decoder.

  • memory (Tensor): The sequence from the last layer of the encoder.

  • tgt_mask (Tensor, optional): the mask of the tgt sequence. Default: None.

  • memory_mask (Tensor, optional): the mask of the memory sequence. Default: None.

  • tgt_key_padding_mask (Tensor, optional): the mask of the tgt keys per batch. Default: None.

  • memory_key_padding_mask (Tensor, optional): the mask of the memory keys per batch. Default: None.

Outputs:

Tensor.

Supported Platforms:

Ascend GPU CPU

Examples

>>> decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
>>> transformer_decoder = nn.TransformerDecoder(decoder_layer, num_layers=6)
>>> memory = Tensor(np.random.rand(10, 32, 512), mindspore.float32)
>>> tgt = Tensor(np.random.rand(20, 32, 512), mindspore.float32)
>>> out = transformer_decoder(tgt, memory)
>>> print(out.shape)
(20, 32, 512)