mindspore.dataset.config.set_prefetch_size

mindspore.dataset.config.set_prefetch_size(size)

Set the queue capacity of the thread in pipeline.

Parameters

size (int) – The length of the cache queue. The size must be greater than 0, otherwise the queue capacity of the thread is invalid.

Raises

• TypeError – If size is not of type int.

• ValueError – If size is not a positive number.

Note

Since total memory used for prefetch can grow very large with high number of workers, when the number of workers is greater than 4, the per worker prefetch size will be reduced. The actual prefetch size at runtime per-worker will be prefetchsize * (4 / num_parallel_workers).

Examples

>>> # Set a new global configuration value for the prefetch size.
>>> import mindspore.dataset as ds
>>> ds.config.set_prefetch_size(1000)