mindspore.dataset.config.set_error_samples_mode
- mindspore.dataset.config.set_error_samples_mode(error_samples_mode)
Set the method in which erroneous samples should be processed in a dataset pipeline.
Note
This error samples feature is only applicable to the Map operation in a dataset pipeline.
For
ErrorSamplesMode.REPLACE
mode, a cache of other samples will be used.If
ErrorSamplesMode.SKIP
mode is used in a distributed setting, beware to manually ensure the number of valid samples are the same for each shard (otherwise one may encounter hangs). One technique is to manually concat a dataset of all valid samples plus a take operation for the number of skipped erroneous samples.
- Parameters
error_samples_mode (ErrorSamplesMode) –
The method in which erroneous samples should be processed in a dataset pipeline. It can be any of [ErrorSamplesMode.RETURN, ErrorSamplesMode.REPLACE, ErrorSamplesMode.SKIP].
ErrorSamplesMode.RETURN
: means erroneous sample results in error raised and returned.ErrorSamplesMode.REPLACE
: means erroneous sample is replaced with a correct sample.ErrorSamplesMode.SKIP
: means erroneous sample is skipped.
- Raises
TypeError – If error_samples_mode is not of type ErrorSamplesMode.
Examples
>>> import mindspore.dataset as ds >>> ds.config.set_error_samples_mode(ds.config.ErrorSamplesMode.SKIP)