{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 构建网络\n",
"\n",
"[](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r1.8/tutorials/zh_cn/advanced/network/mindspore_forward.ipynb) [](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r1.8/tutorials/zh_cn/advanced/network/mindspore_forward.py) [](https://gitee.com/mindspore/docs/blob/r1.8/tutorials/source_zh_cn/advanced/network/forward.ipynb)\n",
"\n",
"MindSpore的`Cell`类是构建所有网络的基类，也是网络的基本单元。自定义网络时，需要继承`Cell`类，本章主要介绍网络基本单元`Cell`和自定义前向网络。\n",
"\n",
"本章主要介绍前向网络模型的构建和网络模型的基本单元，因为不涉及到训练，因此没有反向传播和反向图。\n",
"\n",
"\n",
"\n",
"## 网络基本单元 Cell\n",
"\n",
"当用户需要自定义网络时，需要继承`Cell`类，并重写`__init__`方法和`construct`方法。损失函数、优化器和模型层等本质上也属于网络结构，也需要继承`Cell`类才能实现功能，同样用户也可以根据业务需求自定义这部分内容。\n",
"\n",
"下面介绍`Cell`的关键成员函数。\n",
"\n",
"### construct方法\n",
"\n",
"`Cell`类重写了`__call__`方法，在`Cell`类的实例被调用时，会执行`construct`方法。网络结构在`construct`方法里面定义。\n",
"\n",
"如下样例中，构建了一个简单的卷积网络，卷积网络在`__init__`中定义，在`construct`方法传入输入数据`x`执行卷积计算，并返回计算结果。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from mindspore import nn\n",
"\n",
"\n",
"class Net(nn.Cell):\n",
" def __init__(self):\n",
" super(Net, self).__init__()\n",
" self.conv = nn.Conv2d(10, 20, 3, has_bias=True, weight_init='normal')\n",
"\n",
" def construct(self, x):\n",
" out = self.conv(x)\n",
" return out"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 获取网络参数\n",
"\n",
"`nn.Cell`中返回参数的方法有`parameters_dict`、`get_parameters`和`trainable_params`。\n",
"\n",
"- `parameters_dict`：获取网络结构中所有参数，返回一个以key为参数名，value为参数值的OrderedDict。\n",
"- `get_parameters`：获取网络结构中的所有参数，返回Cell中Parameter的迭代器。\n",
"- `trainable_params`：获取Parameter中`requires_grad`为True的属性，返回可训参数的列表。\n",
"\n",
"如下示例分别使用上述方法获取网络参数并打印。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"parameters_dict of result:\n",
" OrderedDict([('conv.weight', Parameter (name=conv.weight, shape=(20, 10, 3, 3), dtype=Float32, requires_grad=True)), ('conv.bias', Parameter (name=conv.bias, shape=(20,), dtype=Float32, requires_grad=True))])\n",
"\n",
"get_parameters of result:\n",
"Parameter (name=conv.weight, shape=(20, 10, 3, 3), dtype=Float32, requires_grad=True)\n",
"Parameter (name=conv.bias, shape=(20,), dtype=Float32, requires_grad=True)\n",
"\n",
"trainable_params of result:\n",
" [Parameter (name=conv.weight, shape=(20, 10, 3, 3), dtype=Float32, requires_grad=True), Parameter (name=conv.bias, shape=(20,), dtype=Float32, requires_grad=True)]\n"
]
}
],
"source": [
"net = Net()\n",
"\n",
"# 获取网络结构中的所有参数\n",
"result = net.parameters_dict()\n",
"print(\"parameters_dict of result:\\n\", result)\n",
"\n",
"# 获取网络结构中的所有参数\n",
"print(\"\\nget_parameters of result:\")\n",
"for m in net.get_parameters():\n",
" print(m)\n",
"\n",
"# 获取可训练参数列表\n",
"result = net.trainable_params()\n",
"print(\"\\ntrainable_params of result:\\n\", result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 相关属性\n",
"\n",
"1. cells_and_names\n",
"\n",
"`cells_and_names`方法是一个迭代器，返回网络中每个`Cell`的名字和它的内容本身。代码样例如下："
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('', Net<\n",
" (conv): Conv2d<input_channels=10, output_channels=20, kernel_size=(3, 3), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=True, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" >)\n",
"('conv', Conv2d<input_channels=10, output_channels=20, kernel_size=(3, 3), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=True, weight_init=normal, bias_init=zeros, format=NCHW>)\n"
]
}
],
"source": [
"net = Net()\n",
"for m in net.cells_and_names():\n",
" print(m)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. set_grad\n",
"\n",
"`set_grad`用于指定网络是否需要计算梯度。在不传入参数调用时，默认设置`requires_grad`为True，在执行前向网络时将会构建用于计算梯度的反向网络。`TrainOneStepCell`和`GradOperation`接口，无需使用`set_grad`，其内部已实现。若用户需要自定义此类训练功能的接口，需要在其内部或者外部设置`set_grad`。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"class CustomTrainOneStepCell(nn.Cell):\n",
" def __init__(self, network, optimizer, sens=1.0):\n",
" \"\"\"入参有三个：训练网络，优化器和反向传播缩放比例\"\"\"\n",
" super(CustomTrainOneStepCell, self).__init__(auto_prefix=False)\n",
" self.network = network # 前向网络\n",
" self.network.set_grad() # 构建计算梯度的反向网络\n",
" self.optimizer = optimizer # 优化器"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`CustomTrainOneStepCell`代码详细内容可参见[自定义训练与评估网络](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/train/train_eval.html#自定义训练网络)\n",
"\n",
"3. set_train\n",
"\n",
"`set_train`接口指定模型是否为训练模式，在不传入参数调用时，默认设置的`mode`属性为True。\n",
"\n",
"在实现训练和推理结构不同的网络时可以通过`training`属性区分训练和推理场景，当`mode`设置为True时，为训练场景；当`mode`设置为False时，为推理场景。\n",
"\n",
"MindSpore中的`nn.Dropout`算子，根据`Cell`的`mode`属性区分了两种执行逻辑，`mode`为False时直接返回输入，`mode`为True时执行算子。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"training result:\n",
" [[[1.4285715 1.4285715 1.4285715]\n",
" [1.4285715 0. 0. ]]\n",
"\n",
" [[1.4285715 1.4285715 1.4285715]\n",
" [1.4285715 1.4285715 1.4285715]]]\n",
"\n",
"infer result:\n",
" [[[1. 1. 1.]\n",
" [1. 1. 1.]]\n",
"\n",
" [[1. 1. 1.]\n",
" [1. 1. 1.]]]\n"
]
}
],
"source": [
"import numpy as np\n",
"import mindspore as ms\n",
"\n",
"x = ms.Tensor(np.ones([2, 2, 3]), ms.float32)\n",
"net = nn.Dropout(keep_prob=0.7)\n",
"\n",
"# 执行训练\n",
"net.set_train()\n",
"output = net(x)\n",
"print(\"training result:\\n\", output)\n",
"\n",
"# 执行推理\n",
"net.set_train(mode=False)\n",
"output = net(x)\n",
"print(\"\\ninfer result:\\n\", output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4. to_float\n",
"\n",
"`to_float`接口递归地在配置了当前`Cell`和所有子`Cell`的强制转换类型，以使当前网络结构以使用特定的Float类型运行，通常在混合精度场景使用。\n",
"\n",
"如下示例分别对`nn.dense`层使用float32类型和float16类型进行运算，并打印输出结果的数据类型。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Float32\n",
"Float16\n"
]
}
],
"source": [
"import numpy as np\n",
"from mindspore import nn\n",
"import mindspore as ms\n",
"\n",
"# float32进行计算\n",
"x = ms.Tensor(np.ones([2, 2, 3]), ms.float32)\n",
"net = nn.Dense(3, 2)\n",
"output = net(x)\n",
"print(output.dtype)\n",
"\n",
"# float16进行计算\n",
"net1 = nn.Dense(3, 2)\n",
"net1.to_float(ms.float16)\n",
"output = net1(x)\n",
"print(output.dtype)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 构建网络\n",
"\n",
"构建网络时，可以继承`nn.Cell`类，在`__init__`构造函数中申明各个层的定义，在`construct`中实现层之间的连接关系，完成神经网络的前向构造。\n",
"\n",
"`mindspore.ops`模块提供了基础算子的实现，如神经网络算子、数组算子和数学算子等。\n",
"\n",
"`mindspore.nn`模块实现了对基础算子的进一步封装，用户可以根据需要，灵活使用不同的算子。\n",
"\n",
"同时，为了更好地构建和管理复杂的网络，`mindspore.nn`提供了两种容器对网络中的子模块或模型层进行管理，分别为`nn.CellList`和`nn.SequentialCell`两种方式。\n",
"\n",
"### Ops算子构建网络\n",
"\n",
"[mindspore.ops](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/mindspore.ops.html)模块提供了基础算子的实现，如神经网络算子、数组算子和数学算子等。\n",
"\n",
"用户可使用`mindspore.ops`中的算子来构建一个简单的算法 $f(x)=x^2+w$，示例如下："
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 3. 6. 11.]\n"
]
}
],
"source": [
"import numpy as np\n",
"import mindspore as ms\n",
"from mindspore import nn, ops\n",
"\n",
"class Net(nn.Cell):\n",
" def __init__(self):\n",
" super(Net, self).__init__()\n",
" self.mul = ops.Mul()\n",
" self.add = ops.Add()\n",
" self.weight = ms.Parameter(ms.Tensor(np.array([2, 2, 2]), ms.float32))\n",
"\n",
" def construct(self, x):\n",
" return self.add(self.mul(x, x), self.weight)\n",
"\n",
"net = Net()\n",
"input = ms.Tensor(np.array([1, 2, 3]), ms.float32)\n",
"output = net(input)\n",
"\n",
"print(output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### nn层构建网络\n",
"\n",
"尽管`mindspore.ops`模块提供的多样算子可以基本满足网络构建的诉求，但为了在复杂的深度网络中提供更方便易用的接口，[mindspore.nn](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/mindspore.nn.html#)对`mindspore.ops`算子进行了进一步的封装。\n",
"\n",
"`mindspore.nn`模块主要包括神经网络（neural network）中常用的卷积层（如`nn.Conv2d`）、池化层（`nn.MaxPool2d`）、非线性激活函数（如`nn.ReLU`）、损失函数（如`nn.LossBase`）、优化器（如`nn.Momentum`）等，为用户的使用提供了便利。\n",
"\n",
"下面示例代码中，使用`mindspore.nn`模块构建一个Conv + Batch Normalization + ReLu模型网络。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ConvBNReLU<\n",
" (conv): Conv2d<input_channels=3, output_channels=64, kernel_size=(3, 3), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (bn): BatchNorm2d<num_features=64, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=bn.gamma, shape=(64,), dtype=Float32, requires_grad=True), beta=Parameter (name=bn.beta, shape=(64,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)>\n",
" (relu): ReLU<>\n",
" >\n"
]
}
],
"source": [
"import numpy as np\n",
"from mindspore import nn\n",
"\n",
"class ConvBNReLU(nn.Cell):\n",
" def __init__(self):\n",
" super(ConvBNReLU, self).__init__()\n",
" self.conv = nn.Conv2d(3, 64, 3)\n",
" self.bn = nn.BatchNorm2d(64)\n",
" self.relu = nn.ReLU()\n",
"\n",
" def construct(self, x):\n",
" x = self.conv(x)\n",
" x = self.bn(x)\n",
" out = self.relu(x)\n",
" return out\n",
"\n",
"net = ConvBNReLU()\n",
"print(net)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 容器构建网络\n",
"\n",
"为了便于管理和组成更复杂的网络，`mindspore.nn`提供了容器对网络中的子模型块或模型层进行管理，有`nn.CellList`和`nn.SequentialCell`两种方式。\n",
"\n",
"1. CellList构建网络\n",
"\n",
"使用`nn.CellList`构造的Cell既可以是模型层，也可以是构建的网络子块。`nn.CellList`支持`append`、`extend`方法和`insert`方法三种方法。\n",
"\n",
"在运行网络时，可以在construct方法里，使用for循环，运行输出结果。\n",
"\n",
"- `append(cell)`：在列表末尾添加一个cell。\n",
"- `extend（cells)`：将cells添加至列表末尾。\n",
"- `insert(index, cell)`：在列表给定的索引之前插入给定的cell。\n",
"\n",
"如下使用`nn.CellList`构建并执行一个网络，依次包含一个之前定义的模型子块ConvBNReLU、一个Conv2d层、一个BatchNorm2d层和一个ReLU层："
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MyNet<\n",
" (build_block): CellList<\n",
" (0): ConvBNReLU<\n",
" (conv): Conv2d<input_channels=3, output_channels=64, kernel_size=(3, 3), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (bn): BatchNorm2d<num_features=64, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block.0.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block.0.bn.beta, shape=(64,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block.0.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block.0.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)>\n",
" (relu): ReLU<>\n",
" >\n",
" (1): Conv2d<input_channels=64, output_channels=4, kernel_size=(4, 4), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (2): BatchNorm2d<num_features=4, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block.2.gamma, shape=(4,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block.2.beta, shape=(4,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block.2.moving_mean, shape=(4,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block.2.moving_variance, shape=(4,), dtype=Float32, requires_grad=False)>\n",
" (3): ReLU<>\n",
" >\n",
" >\n"
]
}
],
"source": [
"import numpy as np\n",
"import mindspore as ms\n",
"from mindspore import nn\n",
"\n",
"class MyNet(nn.Cell):\n",
"\n",
" def __init__(self):\n",
" super(MyNet, self).__init__()\n",
" layers = [ConvBNReLU()]\n",
" # 使用CellList对网络进行管理\n",
" self.build_block = nn.CellList(layers)\n",
"\n",
" # 使用append方法添加Conv2d层和ReLU层\n",
" self.build_block.append(nn.Conv2d(64, 4, 4))\n",
" self.build_block.append(nn.ReLU())\n",
"\n",
" # 使用insert方法在Conv2d层和ReLU层中间插入BatchNorm2d\n",
" self.build_block.insert(-1, nn.BatchNorm2d(4))\n",
"\n",
" def construct(self, x):\n",
" # for循环执行网络\n",
" for layer in self.build_block:\n",
" x = layer(x)\n",
" return x\n",
"\n",
"net = MyNet()\n",
"print(net)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"把数据输入到网络模型中："
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(1, 4, 64, 32)\n"
]
}
],
"source": [
"input = ms.Tensor(np.ones([1, 3, 64, 32]), ms.float32)\n",
"output = net(input)\n",
"print(output.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. SequentialCell构建网络\n",
"\n",
"使用`nn.SequentialCell`构造Cell顺序容器，支持子模块以List或OrderedDict格式作为输入。\n",
"\n",
"不同于`nn.CellList`的是，`nn.SequentialCell`类内部实现了`construct`方法，可以直接输出结果。\n",
"\n",
"如下示例使用`nn.SequentialCell`构建一个网络，输入为List，网络结构依次包含一个之前定义的模型子块ConvBNReLU、一个Conv2d层、一个BatchNorm2d层和一个ReLU层："
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MyNet<\n",
" (build_block): SequentialCell<\n",
" (0): ConvBNReLU<\n",
" (conv): Conv2d<input_channels=3, output_channels=64, kernel_size=(3, 3), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (bn): BatchNorm2d<num_features=64, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block.0.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block.0.bn.beta, shape=(64,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block.0.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block.0.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)>\n",
" (relu): ReLU<>\n",
" >\n",
" (1): Conv2d<input_channels=64, output_channels=4, kernel_size=(4, 4), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (2): BatchNorm2d<num_features=4, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block.2.gamma, shape=(4,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block.2.beta, shape=(4,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block.2.moving_mean, shape=(4,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block.2.moving_variance, shape=(4,), dtype=Float32, requires_grad=False)>\n",
" (3): ReLU<>\n",
" >\n",
" >\n"
]
}
],
"source": [
"import numpy as np\n",
"import mindspore as ms\n",
"from mindspore import nn\n",
"\n",
"class MyNet(nn.Cell):\n",
"\n",
" def __init__(self):\n",
" super(MyNet, self).__init__()\n",
"\n",
" layers = [ConvBNReLU()]\n",
" layers.extend([nn.Conv2d(64, 4, 4),\n",
" nn.BatchNorm2d(4),\n",
" nn.ReLU()])\n",
" self.build_block = nn.SequentialCell(layers) # 使用SequentialCell对网络进行管理\n",
"\n",
" def construct(self, x):\n",
" return self.build_block(x)\n",
"\n",
"net = MyNet()\n",
"print(net)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"把数据输入到网络模型中："
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(1, 4, 64, 32)\n"
]
}
],
"source": [
"input = ms.Tensor(np.ones([1, 3, 64, 32]), ms.float32)\n",
"output = net(input)\n",
"print(output.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"如下示例使用`nn.SequentialCell`构建一个网络，输入为OrderedDict："
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MyNet<\n",
" (build_block): SequentialCell<\n",
" (ConvBNReLU): ConvBNReLU<\n",
" (conv): Conv2d<input_channels=3, output_channels=64, kernel_size=(3, 3), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (bn): BatchNorm2d<num_features=64, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block.ConvBNReLU.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block.ConvBNReLU.bn.beta, shape=(64,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block.ConvBNReLU.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block.ConvBNReLU.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)>\n",
" (relu): ReLU<>\n",
" >\n",
" (conv): Conv2d<input_channels=64, output_channels=4, kernel_size=(4, 4), stride=(1, 1), pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=False, weight_init=normal, bias_init=zeros, format=NCHW>\n",
" (norm): BatchNorm2d<num_features=4, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block.norm.gamma, shape=(4,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block.norm.beta, shape=(4,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block.norm.moving_mean, shape=(4,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block.norm.moving_variance, shape=(4,), dtype=Float32, requires_grad=False)>\n",
" (relu): ReLU<>\n",
" >\n",
" >\n",
"(1, 4, 64, 32)\n"
]
}
],
"source": [
"import numpy as np\n",
"import mindspore as ms\n",
"from mindspore import nn\n",
"from collections import OrderedDict\n",
"\n",
"class MyNet(nn.Cell):\n",
"\n",
" def __init__(self):\n",
" super(MyNet, self).__init__()\n",
" layers = OrderedDict()\n",
"\n",
" # 将cells加入字典\n",
" layers[\"ConvBNReLU\"] = ConvBNReLU()\n",
" layers[\"conv\"] = nn.Conv2d(64, 4, 4)\n",
" layers[\"norm\"] = nn.BatchNorm2d(4)\n",
" layers[\"relu\"] = nn.ReLU()\n",
"\n",
" # 使用SequentialCell对网络进行管理\n",
" self.build_block = nn.SequentialCell(layers)\n",
"\n",
" def construct(self, x):\n",
" return self.build_block(x)\n",
"\n",
"net = MyNet()\n",
"print(net)\n",
"\n",
"input = ms.Tensor(np.ones([1, 3, 64, 32]), ms.float32)\n",
"output = net(input)\n",
"print(output.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## nn与ops关系\n",
"\n",
"`mindspore.nn`模块是Python实现的模型组件，对低阶API的封装，主要包括神经网络模型相关的各种模型层、损失函数、优化器等。\n",
"\n",
"同时`mindspore.nn`也提供了部分与`mindspore.ops`算子同名的接口，主要作用是对`mindspore.ops`算子进行进一步封装，为用户提供更友好的API。用户也可使用`mindspore.ops`算子根据实际场景实现自定义的网络。\n",
"\n",
"如下示例使用`mindspore.ops.Conv2D`算子实现卷积计算功能，即`nn.Conv2d`算子功能。"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"import mindspore.nn as nn\n",
"import mindspore.ops as ops\n",
"import mindspore as ms\n",
"from mindspore.common.initializer import initializer\n",
"\n",
"\n",
"class Net(nn.Cell):\n",
" def __init__(self, in_channels=10, out_channels=20, kernel_size=3):\n",
" super(Net, self).__init__()\n",
" self.conv2d = ops.Conv2D(out_channels, kernel_size)\n",
" self.bias_add = ops.BiasAdd()\n",
" self.weight = ms.Parameter(\n",
" initializer('normal', [out_channels, in_channels, kernel_size, kernel_size]),\n",
" name='conv.weight')\n",
" self.bias = ms.Parameter(initializer('normal', [out_channels]), name='conv.bias')\n",
"\n",
" def construct(self, x):\n",
" \"\"\"输入数据x\"\"\"\n",
" output = self.conv2d(x, self.weight)\n",
" output = self.bias_add(output, self.bias)\n",
" return output"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "MindSpore",
"language": "python",
"name": "mindspore"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}