Class Lookup
Defined in File text.h
Inheritance Relationships
Base Type
public mindspore::dataset::TensorTransform
(Class TensorTransform)
Class Documentation
-
class Lookup : public mindspore::dataset::TensorTransform
Look up a word into an id according to the input vocabulary table.
Public Functions
Constructor.
- Parameters
vocab – [in] a Vocab object.
unknown_token – [in] Word is used for lookup. In case of the word is out of vocabulary (OOV), the result of lookup will be replaced to unknown_token. If the unknown_token is not specified or it is OOV, runtime error will be thrown (default={}, means no unknown_token is specified).
data_type – [in] mindspore::DataType of the tensor after lookup; must be numeric, including bool. (default=mindspore::DataType::kNumberTypeInt32).
样例/* Define operations */ std::vector<std::string> list = {"a", "b", "c", "d"}; std::shared_ptr<Vocab> vocab = std::make_shared<Vocab>(); Status s = Vocab::BuildFromVector(list, {}, true, &vocab); auto lookup_op = text::Lookup(vocab, "[unk]"); /* dataset is an instance of Dataset object */ dataset = dataset->Map({lookup_op}, // operations {"text"}); // input columns
Constructor.
- Parameters
vocab – [in] a Vocab object.
unknown_token – [in] Word is used for lookup. In case of the word is out of vocabulary (OOV), the result of lookup will be replaced to unknown_token. If the unknown_token is not specified or it is OOV, runtime error will be thrown (default={}, means no unknown_token is specified).
data_type – [in] mindspore::DataType of the tensor after lookup; must be numeric, including bool. (default=mindspore::DataType::kNumberTypeInt32).
-
~Lookup() = default
Destructor.