mindspore.dataset.text.FilterWikipediaXML
- class mindspore.dataset.text.FilterWikipediaXML[源代码]
将Wikipedia XML格式转储过滤为仅由小写字母(a-z,从A-Z转换而来)和空格(从不连续)组成的“干净”文本。
说明
Windows平台尚不支持 FilterWikipediaXML 。
- 支持平台:
CPU
样例:
>>> import mindspore.dataset as ds >>> import mindspore.dataset.text as text >>> >>> # Use the transform in dataset pipeline mode >>> numpy_slices_dataset = ds.NumpySlicesDataset(data=["Welcome to China", "!!!", "ABC"], ... column_names=["text"], shuffle=False) >>> replace_op = text.FilterWikipediaXML() >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=replace_op) >>> for item in numpy_slices_dataset.create_dict_iterator(num_epochs=1, output_numpy=True): ... print(item["text"]) ... break welcome to china >>> >>> # Use the transform in eager mode >>> data = "Welcome to China" >>> output = replace_op(data) >>> print(output) welcome to china
- 教程样例: