Class AddToken
Defined in File text.h
Inheritance Relationships
Base Type
public mindspore::dataset::TensorTransform
(Class TensorTransform)
Class Documentation
-
class AddToken : public mindspore::dataset::TensorTransform
Add token to beginning or end of sequence.
Public Functions
-
explicit AddToken(const std::string &token, bool begin = true)
Constructor.
- 参数
token – [in] The token to be added.
begin – [in] Whether to insert token at start or end of sequence. Default: true.
样例/* Define operations */ auto add_token_op = text::AddToken(token='TOKEN', begin=True); /* dataset is an instance of Dataset object */ dataset = dataset->Map({add_token_op}, // operations {"text"}); // input columns
-
~AddToken() override = default
Destructor.
-
explicit AddToken(const std::string &token, bool begin = true)