mindspore.dataset.text.SPieceTokenizerLoadType
- class mindspore.dataset.text.SPieceTokenizerLoadType[source]
An enumeration for loading type of
mindspore.dataset.text.SentencePieceTokenizer
.Possible enumeration values are: SPieceTokenizerLoadType.FILE, SPieceTokenizerLoadType.MODEL.
SPieceTokenizerLoadType.FILE: Load SentencePiece tokenizer from a Vocab file.
SPieceTokenizerLoadType.MODEL: Load SentencePiece tokenizer from a SentencePieceVocab object.