mindspore.dataset.config.set_prefetch_size

mindspore.dataset.config.set_prefetch_size(size)

Set the queue capacity of the thread in pipeline.

Parameters

size (int) – The length of the cache queue.

Raises

• TypeError – If size is not of type int.

• ValueError – If size <= 0 or size > INT32_MAX(2147483647).

Note

Since total memory used for prefetch can grow very large with high number of workers, when the number of workers is greater than 4, the per worker prefetch size will be reduced. The actual prefetch size at runtime per-worker will be prefetchsize * (4 / num_parallel_workers).

Examples

>>> # Set a new global configuration value for the prefetch size.
>>> ds.config.set_prefetch_size(1000)