MindSpore Distributed Operator List

Linux Ascend GPU CPU Model Development Beginner Intermediate Expert

Distributed Operator

op name	constraints
mindspore.ops.ACos	None
mindspore.ops.Cos	None
mindspore.ops.LogicalNot	None
mindspore.ops.Log	None
mindspore.ops.Exp	None
mindspore.ops.LogSoftmax	The logits can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.Softmax	The logits can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.Tanh	None
mindspore.ops.Gelu	None
mindspore.ops.ReLU	None
mindspore.ops.Sqrt	None
mindspore.ops.Cast	None
mindspore.ops.Neg	None
mindspore.ops.ExpandDims	None
mindspore.ops.Squeeze	None
mindspore.ops.Square	None
mindspore.ops.Sigmoid	None
mindspore.ops.Dropout	Repeated calculation is not supported.
mindspore.ops.Div	None
mindspore.ops.TensorAdd	None
mindspore.ops.RealDiv	None
mindspore.ops.Mul	None
mindspore.ops.Sub	None
mindspore.ops.Pow	None
mindspore.ops.FloorDiv	None
mindspore.ops.Greater	None
mindspore.ops.AssignSub	None
mindspore.ops.SigmoidCrossEntropyWithLogits	None
mindspore.ops.Equal	None
mindspore.ops.NotEqual	None
mindspore.ops.Maximum	None
mindspore.ops.Minimum	None
mindspore.ops.BiasAdd	None
mindspore.ops.Concat	The input_x can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.DropoutGenMask	Need to be used in conjunction with DropoutDoMask.
mindspore.ops.DropoutDoMask	Need to be used in conjunction with DropoutGenMask，configuring shard strategy is not supported.
mindspore.ops.GatherV2	Only support 1-dim and 2-dim parameters and the last dimension of the input_params should be 32-byte aligned; Scalar input_indices is not supported; Repeated calculation is not supported when the parameters are split in the dimension of the axis; Split input_indices and input_params at the same time is not supported.
mindspore.ops.SparseGatherV2	The same as GatherV2.
mindspore.ops.EmbeddingLookup	The same as GatherV2.
mindspore.ops.L2Normalize	The input_x can’t be split into the dimension of axis, otherwise it’s inconsistent with the single machine in the mathematical logic.
mindspore.ops.SoftmaxCrossEntropyWithLogits	The last dimension of logits and labels can’t be splited; Only supports using output[0].
mindspore.ops.MatMul	transpose_a=True is not supported.
mindspore.ops.BatchMatMul	transpore_a=True is not supported.
mindspore.ops.PReLU	When the shape of weight is not [1], the shard strategy in channel dimension of input_x should be consistent with weight.
mindspore.ops.OneHot	Only support 1-dim indices. Must configure strategy for the output and the first and second inputs.
mindspore.ops.ReduceSum	None
mindspore.ops.ReduceMax	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ReduceMin	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ArgMinWithValue	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ArgMaxWithValue	When the input_x is splited on the axis dimension, the distributed result may be inconsistent with that on the single machine.
mindspore.ops.ReduceMean	None
mindspore.ops.Reshape	Configuring shard strategy is not supported.
mindspore.ops.StridedSlice	Only support mask with all 0 values; The dimension needs to be split should be all extracted; Split is not supported when the strides of dimension is 1.
mindspore.ops.Tile	Only support configuring shard strategy for multiples.
mindspore.ops.Transpose	None

Repeated calculation means that the device is not fully used. For example, the cluster has 8 devices to run distributed training, the splitting strategy only cuts the input into 4 copies. In this case, double counting will occur.