mindspore.mint.nn.Embedding

class mindspore.mint.nn.Embedding(num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False, _weight=None, _freeze=False, dtype=None)

The value in input is used as the index, and the corresponding embedding vector is queried from weight .

Warning

• This is an experimental API that is subject to change or deletion.

• On Ascend, the behavior is unpredictable when the value of input is invalid.

Parameters

• num_embeddings (int) – Size of the dictionary of embeddings.

• embedding_dim (int) – The size of each embedding vector.

• padding_idx (int, optional) – If the value is not None, the corresponding row of embedding vector will not be updated in training. The value of embedding vector at padding_idx will default to zeros when the Embedding layer is newly constructed. The value should be in range [-num_embeddings, num_embeddings) if it's not None. Default None.

• max_norm (float, optional) – If the value is not None, firstly get the p-norm result of the embedding vector specified by input where p is specified by norm_type; if the result is larger then max_norm, update the embedding vector` with \(\frac{max\_norm}{result+1e^{-7}}\). Default None.

• norm_type (float, optional) – Indicated the value of p in p-norm. Default 2.0.

• scale_grad_by_freq (bool, optional) – If True the gradients will be scaled by the inverse of frequency of the index in input. Default False.

• sparse (bool, optional) – If True, gradient w.r.t. weight matrix will be a sparse tensor which has not been supported. Default: False.

• _weight (Tensor, optional) – Used to initialize the weight of Embedding. If None, the weight will be initialized from normal distribution \({N}(\text{sigma=1.0}, \text{mean=0.0})\). Default None.

• _freeze (bool, optional) – If weight , the learnable weights of this module, should be freezed. Default: False.

• dtype (mindspore.dtype, optional) – Dtype of Embedding's weight . It is meaningless when _weight is not None. Default: None.

Variables:

weight (Parameter): The learnable weights of this module of shape (num_embeddings, embedding_dim), which

initialized from \({N}(\text{sigma=1.0}, \text{mean=0.0})\) or _weight .

Inputs:

• input (Tensor) - The indices used to lookup in the embedding vector. The data type must be int32 or int64, and the value should be in range [0, num_embeddings).

Outputs:

Tensor, has the same data type as weight, the shape is \((*input.shape, embedding\_dim)\).

Raises

• TypeError – If num_embeddings is not an int.

• TypeError – If embedding_dim is not an int.

• ValueError – If padding_idx is out of valid range.

• TypeError – If max_norm is not a float.

• TypeError – If norm_type is not a float.

• TypeError – If scale_grad_by_freq is not a bool.

• ValueError – If weight.shape is invalid.

• TypeError – If dtype is not one of mindspore.dtype.

Supported Platforms:

Ascend

Examples

>>> import mindspore
>>> import numpy as np
>>> from mindspore import Tensor, mint
>>> input = Tensor([[1, 0, 1, 1], [0, 0, 1, 0]])
>>> embedding = mint.nn.Embedding(num_embeddings=10, embedding_dim=3)
>>> output = embedding(input)
>>> print(output)
[[[-0.0024154  -0.01203444  0.00811537]
  [ 0.00233847 -0.00596091  0.00536799]
  [-0.0024154  -0.01203444  0.00811537]
  [-0.0024154  -0.01203444  0.00811537]]
 [[ 0.00233847 -0.00596091  0.00536799]
  [ 0.00233847 -0.00596091  0.00536799]
  [-0.0024154  -0.01203444  0.00811537]
  [ 0.00233847 -0.00596091  0.00536799]]]