mindspore.dataset.text.SPieceTokenizerLoadType

class mindspore.dataset.text.SPieceTokenizerLoadType(value)[source]

An enumeration for SPieceTokenizerLoadType.

Possible enumeration values are: SPieceTokenizerLoadType.FILE, SPieceTokenizerLoadTypeMODEL.

  • SPieceTokenizerLoadType.FILE: Load sentencepiece tokenizer from local sentencepiece vocab file.

  • SPieceTokenizerLoadType.MODEL: Load sentencepiece tokenizer from sentencepiece vocab instance.