Using the Process Control Statement

Ascend GPU CPU Model Development

View Source On Gitee

Overview

The MindSpore process control statement is similar to the native Python syntax, especially in PYNATIVE_MODE mode. However, there are some special constraints in GRAPH_MODE mode. The following process control statements are executed in GRAPH_MODE mode.

When a process control statement is used, MindSpore determines whether to generate a control flow operator on a network based on whether the condition is a variable. The control flow operator is generated on the network only when the condition is a variable. If a condition expression result needs to be determined during graph build, the condition is a constant. Otherwise, the condition is a variable. It should be specially noted that, when a control flow operator exists in a network, the network is divided into multiple execution subgraphs, and process jumping and data transmission between the subgraphs cause performance loss to some extent.

In the scenario where the condition is a variable:

  • The condition expression contains tensors or a list, tuple, or dict of the tensor type, and the condition expression result is affected by the tensor value.

Common variable conditions are as follows:

  • (x < y).all(), where x or y is the operator output. In this case, whether the condition is true depends on the operator output x and y, and the operator output can be determined only when each step is executed.

  • x in list, where x is the operator output.

In the scenario where the condition is a constant:

  • The condition expression does not contain tensors or a list, tuple, or dict of the tensor type.

  • The condition expression contains tensors or a list, tuple, or dict of the tensor type, but the condition expression result is not affected by the tensor value.

Common constant conditions are as follows:

  • self.flag, which is a scalar of the Boolean type. The value of self.flag is determined when the cell object is created. Therefore, self.flag is a constant condition.

  • x + 1 < 10, where x is a scalar. Although the value of x + 1 is uncertain when a cell object is created, MindSpore computes the results of all scalar expressions during graph build. Therefore, the expression value is determined during build and this is a constant condition.

  • len(my_list) < 10, where my_list is a list object of the tensor type. Although the condition expression contains tensors, the expression result is not affected by the tensor value and is related only to the number of tensors in my_list. Therefore, this is a constant condition.

  • for i in range (0,10), where i is a scalar, and the potential condition expression i < 10 is a constant condition.

Using the if Statement

When using the if statement, ensure that the same variable name in different branches is assigned the same data type if the condition is a variable. In addition, the number of subgraphs of the execution graph generated by the network is in direct proportion to the number of if. Too many if statements generate high performance overheads of the control flow operators and those of the subgraph data transmission.

Using an if Statement with a Variable Condition

In example 1, out is set to [0] in the true branch and to [0, 1] in the false branch. x < y is a variable. Therefore, the data type of out cannot be determined in the out = out + 1 statement, causing a graph build exception.

Example 1:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class SingleIfNet(nn.Cell):
    def construct(self, x, y, z):
        if x < y:
            out = x
        else:
            out = z
        out = out + 1
        return out

forward_net = SingleIfNet()
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
z = Tensor(np.array([0, 1]), dtype=ms.int32)
output = forward_net(x, y, z)

The error information in example 1 is as follows:

ValueError: mindspore/ccsrc/pipeline/jit/static_analysis/static_analysis.cc:734 ProcessEvalResults] The return values of different branches do not match. Shape Join Failed: shape1 = (2), shape2 = ()..

Using an if Statement with a Constant Condition

In example 2, out is assigned to scalar 0 in the true branch and is assigned to [0, 1] in the false branch. x and y are scalars, and x < y + 1 is a constant. It can be determined that the true branch is used in the build phase; therefore, only the content of the true branch exists on the network and there is no control flow operator. The input out data type of out = out + 1 is fixed. Therefore, the test case can be executed properly.

Example 2:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class SingleIfNet(nn.Cell):
    def construct(self, z):
        x = 0
        y = 1
        if x < y + 1:
            out = x
        else:
            out = z
        out = out + 1
        return out

forward_net = SingleIfNet()
z = Tensor(np.array([0, 1]), dtype=ms.int32)
output = forward_net(z)

Using the for Statement

The for statement expands the loop body. In example 3, for is cycled for three times, which is the same as the structure of the execution graph generated in example 4. Therefore, the number of subgraphs and operators of the network using the for statement depends on the number of for iterations. If there are too many operators or subgraphs, hardware resources are limited. If there are too many subgraphs due to the for statement, you can refer to the while writing mode and try to convert the for statement to the while statement whose condition is variable, seeing Using a while Statement instead of a for Statement.

Example 3:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class IfInForNet(nn.Cell):
    def construct(self, x, y):
        out = 0
        for i in range(0,3):
            if x + i < y :
                out = out + x
            else:
                out = out + y
            out = out + 1
        return out

forward_net = IfInForNet()
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
output = forward_net(x, y)

Example 4:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class IfInForNet(nn.Cell):
    def construct(self, x, y):
        out = 0
        #######cycle 0
        if x + 0 < y :
            out = out + x
        else:
            out = out + y
        out = out + 1

         #######cycle 1
        if x + 1 < y :
            out = out + x
        else:
            out = out + y
        out = out + 1

         #######cycle 2
        if x + 2 < y :
            out = out + x
        else:
            out = out + y
        out = out + 1
        return out

forward_net = IfInForNet()
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
output = forward_net(x, y)

Using the while Statement

The while statement is more flexible than the for statement. When the condition of while is a constant, while processes and expands the loop body in a similar way as for. When the condition of while is a variable, while does not expand the loop body. In this case, a control flow operator is generated when the graph is executed.

Using a while Statement with a Constant Condition

As shown in example 5, the condition i < 3 is a constant, and the content of the while loop body is copied for three times. Therefore, the generated execution diagram is the same as that in example 4. When the while statement condition is a constant, the number of operators and subgraphs is proportional to the number of while loops. If there are too many operators or subgraphs, hardware resources are limited.

Example 5:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class IfInWhileNet(nn.Cell):
    def construct(self, x, y):
        i = 0
        out = x
        while i < 3:
            if x + i < y :
                out = out + x
            else:
                out = out + y
            out = out + 1
            i = i + 1
        return out

forward_net = IfInWhileNet()
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
output = forward_net(x, y)

Using a while Statement with a Variable Condition

As shown in example 6, the while condition is changed to a variable, and while is not expanded. The final network output result is the same as that in example 5, but the structure of the execution graph is different. In example 6, there are fewer operators and more subgraphs in an execution graph that is not expanded. A shorter build time and a smaller device memory are used, but extra performance overheads caused by execution of a control flow operator and data transfer between subgraphs are generated.

Example 6:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class IfInWhileNet(nn.Cell):
    def construct(self, x, y, i):
        out = x
        while i < 3:
            if x + i < y :
                out = out + x
            else:
                out = out + y
            out = out + 1
            i = i + 1
        return out

forward_net = IfInWhileNet()
i = Tensor(np.array(0), dtype=ms.int32)
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
output = forward_net(x, y, i)

When the condition of while is a variable, the while loop body cannot be expanded. The expressions in the while loop body are calculated during the running of each step. Therefore, computation types other than tensor, such as scalar, list, and tuple operations cannot exist in the loop body. These types of computation need to be completed during graph build, which conflicts with the computation mechanism of while during execution. As shown in example 7, the condition i < 3 is a variable condition, but the j = j + 1 scalar computation operation exists in the loop body. As a result, an error occurs during graph build.

Example 7:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms

class IfInWhileNet(nn.Cell):
    def __init__(self):
        super().__init__()
        self.nums = [1, 2, 3]

    def construct(self, x, y, i):
        j = 0
        out = x
        while i < 3:
            if x + i < y :
                out = out + x
            else:
                out = out + y
            out = out + self.nums[j]
            i = i + 1
            j = j + 1
        return out

forward_net = IfInWhileNet()
i = Tensor(np.array(0), dtype=ms.int32)
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
output = forward_net(x, y, i)

The error information in example 7 is as follows:

IndexError: mindspore/core/abstract/prim_structures.cc:178 InferTupleOrListGetItem] list_getitem evaluator index should be in range[-3, 3), but got 3.

When the while condition is a variable, the input shape of the operator cannot be changed in the loop body. MindSpore requires that the input shape of the same operator on the network be determined during graph build. However, changing the input shape of the operator in the while loop body takes effect during graph execution. As shown in example 8, the condition i < 3 is a variable condition, and while is not expanded. The ExpandDims operator in the loop body changes the input shape of the expression out = out + 1 in the next loop. As a result, an error occurs during graph build.

Example 8:

import numpy as np
from mindspore import context
from mindspore import Tensor, nn
from mindspore import dtype as ms
from mindspore import ops

class IfInWhileNet(nn.Cell):
    def __init__(self):
        super().__init__()
        self.expand_dims = ops.ExpandDims()

    def construct(self, x, y, i):
        out = x
        while i < 3:
            if x + i < y :
                out = out + x
            else:
                out = out + y
            out = out + 1
            out = self.expand_dims(out, -1)
            i = i + 1
        return out

forward_net = IfInWhileNet()
i = Tensor(np.array(0), dtype=ms.int32)
x = Tensor(np.array(0), dtype=ms.int32)
y = Tensor(np.array(1), dtype=ms.int32)
output = forward_net(x, y, i)

The error information in example 8 is as follows:

ValueError: mindspore/ccsrc/pipeline/jit/static_analysis/static_analysis.cc:734 ProcessEvalResults] The return values of different branches do not match. Shape Join Failed: shape1 = (1, 1), shape2 = (1)..

Using a while Statement instead of a for Statement

The for statement will expand the loop body. While it improves execution performance, it also brings compilation problems, such as increasing compilation time, exceeding function call depth limit, sharing the different weights and so on. In order to solve those problems, the for statement needs to be equivalently converted to a while statement.

A simple example

As shown in example 9, the calculation of multiplying two numbers through addition is realized.

Example 9:

from mindspore import Tensor
from mindspore import ms_function

one = Tensor(1)
zero = Tensor(0)

@ms_function
def mul_by_for(x, y):
    r = zero
    for _ in range(y):
        r = r + x
    return r


a = Tensor(2)
b = 1000
out = mul_by_for(a, b)
print(out)

But, it executes failed.

RuntimeError: mindspore/ccsrc/pipeline/jit/static_analysis/evaluator.cc:201 Eval] Exceed function call depth limit 1000, (function call depth: 1001, simulate call depth: 998).
It's always happened with complex construction of code or infinite recursion or loop.
Please check the code if it's has the infinite recursion or call 'context.set_context(max_call_depth=value)' to adjust this value.
If max_call_depth is set larger, the system max stack depth should be set larger too to avoid stack overflow.

This is because the loop body in the for statement will be expanded. In this example, the loop body will execute 1000 times. The expanded subgraphs are too many, and exceed function call depth limit. In order not to expand the loop body, it can be equivalently replaced with while implementation, as shown in Example 10.

Example 10:

from mindspore import Tensor
from mindspore import ms_function

one = Tensor(1)
zero = Tensor(0)

@ms_function
def mul_by_while(x, y):
    y = Tensor(y)
    r = zero
    while y > 0:
        y = y - one
        r = r + x
    return r

a = Tensor(2)
b = 1000
out = mul_by_while(a, b)
print(out)

And the result is:

2000

Weight in the loop body

As shown in Example 11. The calculation of 1+2+3 is realized, and the value of the counter for each iteration in the loop is saved to the weight.

Example 11:

import mindspore
from mindspore import nn, Tensor
from mindspore import Parameter

class AddIndexNet(nn.Cell):
    def __init__(self, index):
        super(AddIndexNet, self).__init__()
        self.weight = Parameter(Tensor(0, mindspore.float32), name="weight")
        self.idx = Tensor(index)

    def construct(self, x):
        self.weight = self.weight + self.idx
        x = x + self.weight
        return x

class Net(nn.Cell):
    def __init__(self):
        super(Net, self).__init__()
        self.idx = Tensor(0)
        self.block_nums = 3
        self.nets = []
        for i in range(self.block_nums):
            self.nets.append(AddIndexNet(i + 1))

    def construct(self, x):
        for i in range(self.block_nums):
            x = self.nets[i](x)
        return x

x = Tensor(0, mindspore.float32)
net = Net()
out = net(x)
print(out)

The execution result is as follows:

10.0

The result does not match the expectation. The expected result should be 1.0+2.0+3.0=6.0. This is because after expanding the loop body, the operators in different iterations share the same weight (that is self.weight), which causes the same weight to be updated in each iteration.

To solve this problem, we equivalently replace the for statement with the while statement, as shown in Example 12.

Example 12:

import numpy as np
import mindspore
from mindspore import nn, Tensor, ops
from mindspore import Parameter

class AddIndexNet(nn.Cell):
    def __init__(self, block_nums):
        super(AddIndexNet, self).__init__()
        self.weights = Parameter(Tensor(np.zeros((block_nums, 1)), mindspore.float32), name="weights")
        self.gather = ops.Gather()

    def construct(self, x, index):
        weight = self.gather(self.weights, index, 0)
        weight += (index + 1)
        x = x + weight
        return x


class Net(nn.Cell):
    def __init__(self):
        super(Net, self).__init__()
        self.idx = Parameter(Tensor(0), name="index")
        self.iter_num = 3
        self.add_net = AddIndexNet(self.iter_num)

    def construct(self, x):
        while self.idx < self.iter_num:
            x = self.add_net(x, self.idx)
            self.idx += 1
        return x


x = Tensor([0], mindspore.float32)
net = Net()
out = net(x)
print(out)

The execution result is as follows:

[6.]

In this example, we expand the dimension of the weight to [iter_num, 1]. Even if the same weight is shared in different iterations, the data outside the 0th dimension is relatively independent. We use the Gather operator to retrieve the corresponding data to compute. The expected result is 1.0+2.0+3.0=6.0, and the execution result in this example meets the expectation.

Constraints

In addition to the constraints in the conditional variable scenario, the current process statement has constraints in other specific scenarios.

Dynamic Shape

If dynamic shape exists in network model, process control statements are forbidden to use, otherwise an unexpected expcetion may be raised.