mindspore.dataset.text.FilterWikipediaXML
- class mindspore.dataset.text.FilterWikipediaXML[source]
Filter Wikipedia XML dumps to “clean” text consisting only of lowercase letters (a-z, converted from A-Z), and spaces (never consecutive).
Note
FilterWikipediaXML is not supported on Windows platform yet.
- Supported Platforms:
CPU
Examples
>>> import mindspore.dataset as ds >>> import mindspore.dataset.text as text >>> >>> replace_op = text.FilterWikipediaXML() >>> text_file_list = ["/path/to/text_file_dataset_file"] >>> text_file_dataset = ds.TextFileDataset(dataset_files=text_file_list) >>> text_file_dataset = text_file_dataset.map(operations=replace_op)
- Tutorial Examples: