mindspore.dataset.text.SPieceTokenizerLoadType
- class mindspore.dataset.text.SPieceTokenizerLoadType[source]
An enumeration for loading type of
mindspore.dataset.text.SentencePieceTokenizer
.Possible enumeration values are:
SPieceTokenizerLoadType.FILE
,SPieceTokenizerLoadType.MODEL
.SPieceTokenizerLoadType.FILE: Load SentencePiece tokenizer from a Vocab file.
SPieceTokenizerLoadType.MODEL: Load SentencePiece tokenizer from a SentencePieceVocab object.