mindspore.communication.comm_func.batch_isend_irecv

View Source On Gitee
mindspore.communication.comm_func.batch_isend_irecv(p2p_op_list)[source]

Batch send and recv tensors asynchronously.

Note

  • The 'isend' and 'irecv' of P2POp in p2p_op_list between ranks need to match each other.

  • P2POp in p2p_op_list can only use the same communication group.

  • tag of P2POp in p2p_op_list is not support yet.

  • tensor of P2POp in p2p_op_list will not be modified by result inplace.

  • Only support PyNative mode, Graph mode is not currently supported.

Parameters

p2p_op_list (P2POp) – list contains P2POp. P2POp is type of mindspore.communication.comm_func.P2POp

Returns

tuple(Tensor). Output tensors is corresponding to p2p_op_list. At P2POp with 'isend' position, output tensor is a fake tensor with scalar, which has no meaning. At P2POp with 'irecv' position, output tensor is a tensor received from remote device.

Raises

TypeError – If p2p_op_list are not all type of P2POp.

Supported Platforms:

Ascend

Examples

Note

Before running the following examples, you need to configure the communication environment variables.

For Ascend/GPU/CPU devices, it is recommended to use the msrun startup method without any third-party or configuration file dependencies. Please see the msrun start up for more details.

This example should be run with 2 devices.

>>> import numpy as np
>>> import mindspore
>>> from mindspore.communication import init, get_rank, get_group_size
>>> from mindspore.communication.comm_func import batch_isend_irecv, P2POp
>>> from mindspore import Tensor
>>>
>>> init()
>>> this_rank = get_rank()
>>> world_size = get_group_size()
>>> next_rank = (this_rank + 1) % world_size
>>> prev_rank = (this_rank + world_size - 1) % world_size
>>>
>>> send_tensor = Tensor(this_rank + 1, dtype=mindspore.float32)
>>> recv_tensor = Tensor(0., dtype=mindspore.float32)
>>>
>>> send_op = P2POp('isend', send_tensor, next_rank)
>>> recv_op = P2POp('irecv', recv_tensor, prev_rank)
>>>
>>> p2p_op_list = [send_op, recv_op]
>>> output = batch_isend_irecv(p2p_op_list)
>>> print(output)
rank 0:
(Tensor(shape=[], dtype=Float32, value= 0), Tensor(shape=[], dtype=Float32, value= 2))
rank 1:
(Tensor(shape=[], dtype=Float32, value= 0), Tensor(shape=[], dtype=Float32, value= 1))