mindspore.dataset.text.SentencePieceModel
- class mindspore.dataset.text.SentencePieceModel[source]
Subword algorithms for SentencePiece.
Available values are as follows:
SentencePieceModel.UNIGRAM
: Unigram Language Model subword algorithm.SentencePieceModel.BPE
: Byte-Pair-Encoding subword algorithm.SentencePieceModel.CHAR
: Character-based subword algorithm.SentencePieceModel.WORD
: Word-based subword algorithm.