mindspore.nn.TransformerDecoder
- class mindspore.nn.TransformerDecoder(decoder_layer, num_layers, norm=None)[source]
Transformer Decoder module with multi-layer stacked of
mindspore.nn.TransformerDecoderLayer
, including multihead self attention, cross attention and feedforward layer.- Parameters
decoder_layer (Cell) – An instance of the
mindspore.nn.TransformerDecoderLayer
class.num_layers (int) – The number of decoder-layers in the decoder.
norm (Cell, optional) – The layer normalization module. Default:
None
.
- Inputs:
tgt (Tensor) - The sequence to the decoder. For unbatched input, the shape is \((T, E)\) ; otherwise if batch_first=False in
mindspore.nn.TransformerDecoderLayer
, the shape is \((T, N, E)\) and if batch_first=True , the shape is \((N, T, E)\), where \((T)\) is the target sequence length, \((N)\) is the number of batches, and \((E)\) is the number of features. Supported types: float16, float32, float64.memory (Tensor) - The sequence from the last layer of the encoder. Supported types: float16, float32, float64.
tgt_mask (Tensor, optional) - the mask of the tgt sequence. The shape is \((T, T)\) or \((N*nhead, T, T)\) , where nhead is the arguent in
mindspore.nn.TransformerDecoderLayer
. Supported types: float16, float32, float64, bool. Default:None
.memory_mask (Tensor, optional) - the mask of the memory sequence. The shape is \((T, S)\) . Supported types: float16, float32, float64, bool. Default:
None
.tgt_key_padding_mask (Tensor, optional) - the mask of the tgt keys per batch. Shape is \((T)\). Supported types: float16, float32, float64, bool. Default:
None
.memory_key_padding_mask (Tensor, optional) - the mask of the memory keys per batch. The shape is \((S)\) for unbatched input, otherwise \((N, S)\) . Supported types: float16, float32, float64, bool. Default:
None
.
- Outputs:
Tensor. The shape and dtype of Tensor is the same with tgt .
- Supported Platforms:
Ascend
GPU
CPU
Examples
>>> import mindspore as ms >>> import numpy as np >>> decoder_layer = ms.nn.TransformerDecoderLayer(d_model=512, nhead=8) >>> transformer_decoder = ms.nn.TransformerDecoder(decoder_layer, num_layers=6) >>> memory = ms.Tensor(np.random.rand(10, 32, 512), ms.float32) >>> tgt = ms.Tensor(np.random.rand(20, 32, 512), ms.float32) >>> out = transformer_decoder(tgt, memory) >>> print(out.shape) (20, 32, 512)