mindspore.nn.TransformerDecoder

class mindspore.nn.TransformerDecoder(decoder_layer, num_layers, norm=None)[source]

Transformer Decoder module with multi-layer stacked of mindspore.nn.TransformerDecoderLayer, including multihead self attention, cross attention and feedforward layer.

Parameters

decoder_layer (Cell) – An instance of the mindspore.nn.TransformerDecoderLayer class.
num_layers (int) – The number of decoder-layers in the decoder.
norm (Cell, optional) – The layer normalization module. Default: None.

Inputs:

tgt (Tensor) - The sequence to the decoder. For unbatched input, the shape is $(T, E)$ ; otherwise if batch_first=False in mindspore.nn.TransformerDecoderLayer, the shape is $(T, N, E)$ and if batch_first=True , the shape is $(N, T, E)$ , where $(T)$ is the target sequence length, $(N)$ is the number of batches, and $(E)$ is the number of features. Supported types: float16, float32, float64.
memory (Tensor) - The sequence from the last layer of the encoder. Supported types: float16, float32, float64.
tgt_mask (Tensor, optional) - the mask of the tgt sequence. The shape is $(T, T)$ or $(N * n h e a d, T, T)$ , where nhead is the arguent in mindspore.nn.TransformerDecoderLayer. Supported types: float16, float32, float64, bool. Default: None.
memory_mask (Tensor, optional) - the mask of the memory sequence. The shape is $(T, S)$ . Supported types: float16, float32, float64, bool. Default: None.
tgt_key_padding_mask (Tensor, optional) - the mask of the tgt keys per batch. Shape is $(T)$ . Supported types: float16, float32, float64, bool. Default: None.
memory_key_padding_mask (Tensor, optional) - the mask of the memory keys per batch. The shape is $(S)$ for unbatched input, otherwise $(N, S)$ . Supported types: float16, float32, float64, bool. Default: None.

Outputs:

Tensor. The shape and dtype of Tensor is the same with tgt .

Supported Platforms:

Ascend GPU CPU

Examples

>>> import mindspore as ms
>>> import numpy as np
>>> decoder_layer = ms.nn.TransformerDecoderLayer(d_model=512, nhead=8)
>>> transformer_decoder = ms.nn.TransformerDecoder(decoder_layer, num_layers=6)
>>> memory = ms.Tensor(np.random.rand(10, 32, 512), ms.float32)
>>> tgt = ms.Tensor(np.random.rand(20, 32, 512), ms.float32)
>>> out = transformer_decoder(tgt, memory)
>>> print(out.shape)
(20, 32, 512)