mindspore.dataset.text.CharNGram
- class mindspore.dataset.text.CharNGram[source]
CharNGram object that is used to map tokens into pre-trained vectors.
- classmethod from_file(file_path, max_vectors=None)[source]
Build a CharNGram vector from a file.
- Parameters
file_path (str) – Path of the file that contains the CharNGram vectors.
max_vectors (int, optional) – This can be used to limit the number of pre-trained vectors loaded. Most pre-trained vector sets are sorted in the descending order of word frequency. Thus, in situations where the entire set doesn’t fit in memory, or is not needed for another reason, passing max_vectors can limit the size of the loaded set. Default:
None, no limit.
- Returns
CharNGram, CharNGram vector build from a file.
- Raises
RuntimeError – If file_path contains invalid data.
ValueError – If max_vectors is invalid.
TypeError – If max_vectors is not type of integer.
Examples
>>> import mindspore.dataset.text as text >>> char_n_gram = text.CharNGram.from_file("/path/to/char_n_gram/file", max_vectors=None)