Class Lookup

Inheritance Relationships

Base Type

Class Documentation

class Lookup : public mindspore::dataset::TensorTransform

Look up a word into an id according to the input vocabulary table.

Public Functions

inline explicit Lookup(const std::shared_ptr<Vocab> &vocab, const std::optional<std::string> &unknown_token = {}, mindspore::DataType data_type = mindspore::DataType::kNumberTypeInt32)

Constructor.

Parameters

  • vocab – [in] a Vocab object.

  • unknown_token – [in] Word is used for lookup. In case of the word is out of vocabulary (OOV), the result of lookup will be replaced to unknown_token. If the unknown_token is not specified or it is OOV, runtime error will be thrown (default={}, means no unknown_token is specified).

  • data_type – [in] mindspore::DataType of the tensor after lookup; must be numeric, including bool. (default=mindspore::DataType::kNumberTypeInt32).

explicit Lookup(const std::shared_ptr<Vocab> &vocab, const std::optional<std::vector<char>> &unknown_token, mindspore::DataType data_type = mindspore::DataType::kNumberTypeInt32)

Constructor.

Parameters

  • vocab – [in] a Vocab object.

  • unknown_token – [in] Word is used for lookup. In case of the word is out of vocabulary (OOV), the result of lookup will be replaced to unknown_token. If the unknown_token is not specified or it is OOV, runtime error will be thrown (default={}, means no unknown_token is specified).

  • data_type – [in] mindspore::DataType of the tensor after lookup; must be numeric, including bool. (default=mindspore::DataType::kNumberTypeInt32).

~Lookup() = default

Destructor.