MindSpore Distributed API List

API name	constraints
mindspore.ops.Abs	None
mindspore.ops.ACos	None
mindspore.ops.Acosh	None
mindspore.ops.Add	None
mindspore.ops.ApproximateEqual	None
mindspore.ops.ArgMaxWithValue	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ArgMinWithValue	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.Asin	None
mindspore.ops.Asinh	None
mindspore.ops.Assign	None
mindspore.ops.AssignAdd	None
mindspore.ops.AssignSub	None
mindspore.ops.Atan	None
mindspore.ops.Atan2	None
mindspore.ops.Atanh	None
mindspore.ops.AvgPool	1. The data format only supports ‘NCHW’; 2. The shapes of output H/W dimension must be divisible by the split strategies of input H/W dimension; 3. If H/W is split: 1) If the kernel_size <= stride, the input slice size must be divisible by stride; 2) It does not support kernel_size > stride; 4. In auto_parallel mode, the strategy’s searching algorithm can not use “recursive_programming”.
mindspore.ops.BatchMatMul	transpore_a=True is not supported.
mindspore.ops.BatchNorm	It does not support GPU.
mindspore.ops.BesselI0e	None
mindspore.ops.BesselI1e	None
mindspore.ops.BiasAdd	None
mindspore.ops.BoundingBoxEncode	1. The first dimension of input (anchor_box) and input (groundtruth_box) can be split; 2. The sharding strategies of input (anchor_box) and input (groundtruth_box) must be the same.
mindspore.ops.BroadcastTo	None
mindspore.ops.Cast	The shard strategy is ignored in the Auto Parallel and Semi Auto Parallel mode.
mindspore.ops.Ceil	None
mindspore.ops.Concat	The input_x can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.Conv2D	1. The data format only supports ‘NCHW’; 2. If data exchange between adjacent nodes is involved, only Ascend is supported; 3. When the value of group is not 1, can not split C-in or C-out; 4. The last two dimensions of weight can not be split; 5. The output shape of H/W dimension must be divisible by the strategy of input H/W dimensions; 6. In valid mode: If H/W dimension is split: 1) When the kernel_size <= stride (kernel_size is dilation (kernel_size - 1) + 1, the same below), the input‘s slice shape of H/W dimension must be divisible by stride; 2) It does not support that kernel_size > stride; 7. In the same/pad mode: If H/W dimension is split: 1) (Total input length including pad - kernel_size) must be divisible by stride; 2) (Output length stride - input length) must be divisible by strategy: 3) The length of data sent and received between adjacent cards must be greater than or equal to 0 and less than or equal to the slice size;
mindspore.ops.Cos	None
mindspore.ops.Cosh	None
mindspore.ops.CropAndResize	1. Sharding of the H/W dimension of input (x) and the second dimension of input (boxes) is not supported. 2. The shard strategy for the first dimension of inputs (boxes) and (box_index) must be the same.
mindspore.ops.Div	None
mindspore.ops.DivNoNan	None
mindspore.ops.Dropout	None
mindspore.ops.Elu	None
mindspore.ops.EmbeddingLookup	The same as Gather.
mindspore.ops.Equal	None
mindspore.ops.Erf	None
mindspore.ops.Erfc	None
mindspore.ops.Exp	None
mindspore.ops.ExpandDims	None
mindspore.ops.Expm1	None
mindspore.ops.Floor	None
mindspore.ops.FloorDiv	None
mindspore.ops.FloorMod	None
mindspore.ops.Gather	Only support 1-dim and 2-dim parameters and the last dimension of the input_params should be 32-byte aligned; Scalar input_indices is not supported; Repeated calculation is not supported when the parameters are split in the dimension of the axis; Split input_indices and input_params at the same time is not supported；In uniform split, when axis = 0 and the parameter is split in the dimension of axis, the output strategy can be configured. The legal output segmentation strategy is (index_strategy, param_strategy[1:]) or ((index_strategy[0]*param_strategy[0], index_strategy[1:]), param_strategy[1:])
mindspore.ops.GatherD	The dimension corresponding to dim cannot be segmented; In auto_parallel mode, the strategy’s searching algorithm can not use “recursive_programming”.
mindspore.ops.GatherNd	The first input can’t be split, and the last dimension of the second input can’t be split; In auto_parallel mode, the strategy’s searching algorithm can not use “recursive_programming”.
mindspore.ops.GeLU	None
mindspore.ops.Greater	None
mindspore.ops.GreaterEqual	None
mindspore.ops.Inv	None
mindspore.ops.IOU	The first dimension of the inputs(anchor_boxes) and inputs(gt_boxes) can be spilt.
mindspore.ops.L2Normalize	The input_x can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.Less	None
mindspore.ops.LessEqual	None
mindspore.ops.LogicalAnd	None
mindspore.ops.LogicalNot	None
mindspore.ops.LogicalOr	None
mindspore.ops.Log	None
mindspore.ops.Log1p	None
mindspore.ops.LogSoftmax	The logits can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.MatMul	1. transpose_a=True is not supported; 2. When transpose_b=True is set, the input’s split strategy must be in the form of ((A, B), (C, B)); 3. When transpose_b=False is set, the input’s split strategy must be in the form of ((A, B), (B, C)); 4. It is supported to set the output’s split strategy, the legal output’s split strategy is ((A, C),) or ((A * B, C),)
mindspore.ops.Maximum	None
mindspore.ops.MaxPool	1. The data format only supports ‘NCHW’; 2. The shapes of output H/W dimension must be divisible by the split strategies of input H/W dimension; 3. If H/W is split: 1) If the kernel_size <= stride, the input slice size must be divisible by stride; 2) It does not support kernel_size > stride; 4. In auto_parallel mode, the strategy’s searching algorithm can not use “recursive_programming”.
mindspore.ops.Minimum	None
mindspore.ops.Mod	None
mindspore.ops.Mul	None
mindspore.ops.Neg	None
mindspore.ops.NotEqual	None
mindspore.ops.OneHot	Only support 1-dim indices. Must configure strategy for the output and the first and second inputs.
mindspore.ops.OnesLike	None
mindspore.ops.Pow	None
mindspore.ops.PReLU	When the shape of weight is not [1], the shard strategy in channel dimension of input_x should be consistent with weight.
mindspore.ops.RandomChoiceWithMask	1. Only the all-1 strategy is supported. 2. Only support on the GPU platform.
mindspore.ops.RealDiv	None
mindspore.ops.Reciprocal	None
mindspore.ops.ReduceMax	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ReduceMin	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ReduceSum	None
mindspore.ops.ReduceMean	None
mindspore.ops.ReLU	None
mindspore.ops.ReLU6	None
mindspore.ops.Reshape	Configuring shard strategy is not supported. In auto parallel mode, if multiple operators are followed by the reshape operator, different shard strategys are not allowed to be configured for these operators.
mindspore.ops.ResizeBilinear	Under GPU platform, can not split H or W dimension; Under Ascend platform, can not split H dimension, and the output shape of W dimension can be divided by the strategy.
mindspore.ops.ResizeNearestNeighbor	When align_corners=True is set, only the first dimension and the second dimension are supported to split.
mindspore.ops.ROIAlign	Sharding the H/W dimension of the input(features) and the second dimension of input(rois) is not supported.
mindspore.ops.Round	None
mindspore.ops.Rsqrt	None
mindspore.ops.ScatterUpdate	The first dimension of first input can not be split, the second input can not be split, and the first n dimensions (n is the dimension size of the second input) of the third input can not be split; In auto_parallel mode, the strategy’s searching algorithm can not use “recursive_programming”.
mindspore.ops.Select	In auto_parallel mode, the strategy’s searching algorithm can not use “recursive_programming”.
mindspore.ops.Sigmoid	None
mindspore.ops.SigmoidCrossEntropyWithLogits	None
mindspore.ops.Sign	None
mindspore.ops.Sin	None
mindspore.ops.Sinh	None
mindspore.ops.Softmax	The logits can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.SoftmaxCrossEntropyWithLogits	The last dimension of logits and labels can’t be splited; Only supports using output[0].
mindspore.ops.Softplus	None
mindspore.ops.Softsign	None
mindspore.ops.SparseGatherV2	The same as Gather.
mindspore.ops.Split	The input_x can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.Sqrt	None
mindspore.ops.Square	None
mindspore.ops.Squeeze	None
mindspore.ops.Stack	None
mindspore.ops.StridedSlice	Only support mask with all 0 values; The dimension needs to be split should be all extracted; Split is supported when the strides of dimension is 1.
mindspore.ops.Slice	The dimension needs to be split should be all extracted.
mindspore.ops.Sub	None
mindspore.ops.Tan	None
mindspore.ops.Tanh	None
mindspore.ops.Tile	Only support configuring shard strategy for multiples.
mindspore.ops.TopK	The input_x can’t be split into the last dimension, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.Transpose	None
mindspore.ops.Unique	Only support the repeat calculate shard strategy (1,).
mindspore.ops.UnsortedSegmentSum	The shard of input_x and segment_ids must be the same as the dimension of segment_ids.
mindspore.ops.UnsortedSegmentMin	The shard of input_x and segment_ids must be the same as the dimension of segment_ids. Note that if the segment id i is missing, then the output[i] will be filled with the maximum of the input type. The user needs to mask the maximum value to avoid value overflow. The communication operation such as AllReudce will raise an Run Task Error due to overflow.
mindspore.ops.UnsortedSegmentMax	The shard of input_x and segment_ids must be the same as the dimension of segment_ids. Note that if the segment id i is missing, then the output[i] will be filled with the minimum of the input type. The user needs to mask the minimum value to avoid value overflow. The communication operation such as AllReudce will raise an Run Task Error due to overflow.
mindspore.ops.ZerosLike	None

Repeated calculation means that the device is not fully used. For example, the cluster has 8 devices to run distributed training, the splitting strategy only cuts the input into 4 copies. In this case, double counting will occur.