mindspore.dataset.text.SPieceTokenizerLoadType

class mindspore.dataset.text.SPieceTokenizerLoadType

An enumeration for loading type of mindspore.dataset.text.SentencePieceTokenizer .

Possible enumeration values are: SPieceTokenizerLoadType.FILE, SPieceTokenizerLoadType.MODEL.

• SPieceTokenizerLoadType.FILE: Load SentencePiece tokenizer from a Vocab file.

• SPieceTokenizerLoadType.MODEL: Load SentencePiece tokenizer from a SentencePieceVocab object.