mindspore.dataset.text ====================== .. automodule:: mindspore.dataset.text mindspore.dataset.text.transforms --------------------------------- .. msnoteautosummary:: :toctree: dataset_text :nosignatures: :template: classtemplate.rst mindspore.dataset.text.transforms.BasicTokenizer mindspore.dataset.text.transforms.BertTokenizer mindspore.dataset.text.transforms.CaseFold mindspore.dataset.text.transforms.JiebaTokenizer mindspore.dataset.text.transforms.Lookup mindspore.dataset.text.transforms.Ngram mindspore.dataset.text.transforms.NormalizeUTF8 mindspore.dataset.text.transforms.PythonTokenizer mindspore.dataset.text.transforms.RegexReplace mindspore.dataset.text.transforms.RegexTokenizer mindspore.dataset.text.transforms.SentencePieceTokenizer mindspore.dataset.text.transforms.SlidingWindow mindspore.dataset.text.transforms.ToNumber mindspore.dataset.text.transforms.TruncateSequencePair mindspore.dataset.text.transforms.UnicodeCharTokenizer mindspore.dataset.text.transforms.UnicodeScriptTokenizer mindspore.dataset.text.transforms.WhitespaceTokenizer mindspore.dataset.text.transforms.WordpieceTokenizer mindspore.dataset.text.utils ---------------------------- .. msnoteautosummary:: :toctree: dataset_text :nosignatures: :template: classtemplate.rst mindspore.dataset.text.utils.SentencePieceVocab mindspore.dataset.text.utils.to_str mindspore.dataset.text.utils.to_bytes mindspore.dataset.text.utils.Vocab