mindspore.dataset.text.SPieceTokenizerLoadType
- class mindspore.dataset.text.SPieceTokenizerLoadType(value)[source]
An enumeration for SPieceTokenizerLoadType.
Possible enumeration values are: SPieceTokenizerLoadType.FILE, SPieceTokenizerLoadTypeMODEL.
SPieceTokenizerLoadType.FILE: Load sentencepiece tokenizer from local sentencepiece vocab file.
SPieceTokenizerLoadType.MODEL: Load sentencepiece tokenizer from sentencepiece vocab instance.