mindspore.dataset.config.set_auto_num_workers

mindspore.dataset.config.set_auto_num_workers(enable)

Set num_parallel_workers for each op automatically(This feature is turned off by default).

If turned on, the num_parallel_workers in each op will be adjusted automatically, possibly overwriting the num_parallel_workers passed in by user or the default value (if user doesn't pass anything) set by mindspore.dataset.config.set_num_parallel_workers().

For now, this function is only optimized for YoloV3 dataset with per_batch_map (running map in batch). This feature aims to provide a baseline for optimized num_workers assignment for each operation. Operation whose num_parallel_workers is adjusted to a new value will be logged.

Parameters

enable (bool) – Whether to enable auto num_workers feature or not.

Raises

TypeError – If enable is not of boolean type.

Examples

>>> # Enable auto_num_worker feature, this might override the num_parallel_workers passed in by user
>>> import mindspore.dataset as ds
>>> ds.config.set_auto_num_workers(True)