mindspore.dataset.config.set_auto_num_workers
- mindspore.dataset.config.set_auto_num_workers(enable)
Set num_parallel_workers for each op automatically(This feature is turned off by default).
If turned on, the num_parallel_workers in each op will be adjusted automatically, possibly overwriting the num_parallel_workers passed in by user or the default value (if user doesn’t pass anything) set by
mindspore.dataset.config.set_num_parallel_workers()
.For now, this function is only optimized for YoloV3 dataset with per_batch_map (running map in batch). This feature aims to provide a baseline for optimized num_workers assignment for each operation. Operation whose num_parallel_workers is adjusted to a new value will be logged.
- Parameters
enable (bool) – Whether to enable auto num_workers feature or not.
- Raises
TypeError – If enable is not of boolean type.
Examples
>>> # Enable auto_num_worker feature, this might override the num_parallel_workers passed in by user >>> import mindspore.dataset as ds >>> ds.config.set_auto_num_workers(True)