mindspore.dataset.config.set_error_samples_mode

mindspore.dataset.config.set_error_samples_mode(error_samples_mode)

Set the method in which erroneous samples should be processed in a dataset pipeline.

Note

  • This error samples feature is only applicable to the Map operation in a dataset pipeline.

  • For ErrorSamplesMode.REPLACE mode, a cache of other samples will be used.

  • If ErrorSamplesMode.SKIP mode is used in a distributed setting, beware to manually ensure the number of valid samples are the same for each shard (otherwise one may encounter hangs). One technique is to manually concat a dataset of all valid samples plus a take operation for the number of skipped erroneous samples.

Parameters

error_samples_mode (ErrorSamplesMode) –

The method in which erroneous samples should be processed in a dataset pipeline. It can be any of [ErrorSamplesMode.RETURN, ErrorSamplesMode.REPLACE, ErrorSamplesMode.SKIP].

  • ErrorSamplesMode.RETURN: means erroneous sample results in error raised and returned.

  • ErrorSamplesMode.REPLACE: means erroneous sample is replaced with a correct sample.

  • ErrorSamplesMode.SKIP: means erroneous sample is skipped.

Raises

TypeError – If error_samples_mode is not of type ErrorSamplesMode.

Examples

>>> import mindspore.dataset as ds
>>> ds.config.set_error_samples_mode(ds.config.ErrorSamplesMode.SKIP)