mindspore.dataset.text.FilterWikipediaXML

class mindspore.dataset.text.FilterWikipediaXML

Filter Wikipedia XML dumps to “clean” text consisting only of lowercase letters (a-z, converted from A-Z), and spaces (never consecutive).

Note

FilterWikipediaXML is not supported on Windows platform yet.

Supported Platforms:

CPU

Examples

>>> import mindspore.dataset as ds
>>> import mindspore.dataset.text as text
>>>
>>> replace_op = text.FilterWikipediaXML()
>>> text_file_list = ["/path/to/text_file_dataset_file"]
>>> text_file_dataset = ds.TextFileDataset(dataset_files=text_file_list)
>>> text_file_dataset = text_file_dataset.map(operations=replace_op)

Tutorial Examples:

• Illustration of text transforms