mindspore.dataset.text.AddToken
- class mindspore.dataset.text.AddToken(token, begin=True)[source]
Add token to beginning or end of sequence.
- Parameters
- Raises
- Supported Platforms:
CPU
Examples
>>> import mindspore.dataset as ds >>> import mindspore.dataset.text as text >>> >>> dataset = ds.NumpySlicesDataset(data={"text": [['a', 'b', 'c', 'd', 'e']]}) >>> # Data before >>> # | text | >>> # +---------------------------+ >>> # | ['a', 'b', 'c', 'd', 'e'] | >>> # +---------------------------+ >>> add_token_op = text.AddToken(token='TOKEN', begin=True) >>> dataset = dataset.map(operations=add_token_op) >>> # Data after >>> # | text | >>> # +---------------------------+ >>> # | ['TOKEN', 'a', 'b', 'c', 'd', 'e'] | >>> # +---------------------------+
- Tutorial Examples: