mindspore.mint.distributed.broadcast

View Source On Gitee
mindspore.mint.distributed.broadcast(tensor, src, group=None, async_op=False)[source]

Broadcasts the tensor to the whole group.

Note

  • The tensors must have the same shape and format in all processes of the collection.

  • Only support PyNative mode, Graph mode is not currently supported.

Parameters

  • tensor (Tensor) – Data to be sent if src is the rank of current process, and tensor to be used to save received data otherwise.

  • src (int) – Specifies the rank(global rank) of the process that broadcast the tensor. And only process src will broadcast the tensor.

  • group (str, optional) – The communication group to work on. If None, which means "hccl_world_group" in Ascend. Default: None.

  • async_op (bool, optional) – Whether this operator should be an async operator. Default: False .

Returns

CommHandle, CommHandle is an async work handle, if async_op is set to True. CommHandle will be None, when async_op is False.

Raises

  • TypeError – If the type of the tensor parameter is not Tensor, src is not an integer, group is not a string or async_op is not bool.

  • RuntimeError – If device target is invalid, or backend is invalid, or distributed initialization fails.

Supported Platforms:

Ascend

Examples

Note

Before running the following examples, you need to configure the communication environment variables.

For Ascend devices, it is recommended to use the msrun startup method without any third-party or configuration file dependencies. Please see the msrun start up for more details.

This example should be run with 2 devices.

>>> import mindspore as ms
>>> from mindspore.mint.distributed import init_process_group, broadcast
>>> import numpy as np
>>> # Launch 2 processes.
>>>
>>> init_process_group()
>>> data = ms.Tensor(np.arange(8).reshape([2, 4]).astype(np.float32))
>>> handle = broadcast(tensor=data, src=0)
>>> print(data)
[[0. 1. 2. 3.]
 [4. 5. 6. 7.]]
Tutorial Examples: