{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 常用网络组件\n", "\n", "[![](https://gitee.com/mindspore/docs/raw/r1.2/docs/programming_guide/source_zh_cn/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.2/docs/programming_guide/source_zh_cn/network_component.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.2/docs/programming_guide/source_zh_cn/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r1.2/programming_guide/mindspore_network_component.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.2/docs/programming_guide/source_zh_cn/_static/logo_modelarts.png)](https://console.huaweicloud.com/modelarts/?region=cn-north-4#/notebook/loading?share-url-b64=aHR0cHM6Ly9vYnMuZHVhbHN0YWNrLmNuLW5vcnRoLTQubXlodWF3ZWljbG91ZC5jb20vbWluZHNwb3JlLXdlYnNpdGUvbm90ZWJvb2svbW9kZWxhcnRzL3Byb2dyYW1taW5nX2d1aWRlL21pbmRzcG9yZV9uZXR3b3JrX2NvbXBvbmVudC5pcHluYg==&image_id=65f636a0-56cf-49df-b941-7d2a07ba8c8c)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 概述\n", "\n", "MindSpore封装了一些常用的网络组件，用于网络的训练、推理、求梯度和数据处理等操作。\n", "\n", "这些网络组件可以直接被用户使用，同样也会在`model.train`和`model.eval`等更高级的封装接口内部进行使用。\n", "\n", "本节内容将会介绍三个网络组件，分别是`GradOperation`、`WithLossCell`和`TrainOneStepCell`，将会从功能、用户使用和内部使用三个方面来进行介绍。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GradOperation\n", "\n", "GradOperation组件用于生成输入函数的梯度，利用`get_all`、`get_by_list`和`sens_param`参数控制梯度的计算方式，细节内容详见[API文档](https://www.mindspore.cn/doc/api_python/zh-CN/r1.2/mindspore/ops/mindspore.ops.GradOperation.html)。\n", "\n", "GradOperation的使用实例如下：" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "ExecuteTime": { "end_time": "2021-02-09T02:41:47.549559Z", "start_time": "2021-02-09T02:41:46.596650Z" } }, "outputs": [ { "data": { "text/plain": [ "Tensor(shape=[2, 3], dtype=Float32, value=\n", "[[1.41000009e+000, 1.60000002e+000, 6.59999943e+000],\n", " [1.41000009e+000, 1.60000002e+000, 6.59999943e+000]])" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "import mindspore.nn as nn\n", "from mindspore import Tensor, Parameter\n", "from mindspore import dtype as mstype\n", "import mindspore.ops as ops\n", "\n", "\n", "class Net(nn.Cell):\n", " def __init__(self):\n", " super(Net, self).__init__()\n", " self.matmul = ops.MatMul()\n", " self.z = Parameter(Tensor(np.array([1.0], np.float32)), name='z')\n", " def construct(self, x, y):\n", " x = x * self.z\n", " out = self.matmul(x, y)\n", " return out\n", "\n", "class GradNetWrtX(nn.Cell):\n", " def __init__(self, net):\n", " super(GradNetWrtX, self).__init__()\n", " self.net = net\n", " self.grad_op = ops.GradOperation()\n", " def construct(self, x, y):\n", " gradient_function = self.grad_op(self.net)\n", " return gradient_function(x, y)\n", "\n", "x = Tensor([[0.5, 0.6, 0.4], [1.2, 1.3, 1.1]], dtype=mstype.float32)\n", "y = Tensor([[0.01, 0.3, 1.1], [0.1, 0.2, 1.3], [2.1, 1.2, 3.3]], dtype=mstype.float32)\n", "GradNetWrtX(Net())(x, y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "上面的例子是计算`Net`相对与`x`的梯度值，首先需要定义网络`Net`作为`GradOperation`的输入，实例创建了包含梯度运算的`GradNetWrtX`。调用`GradNetWrtX`是将网络传入`GradOperation`生成梯度函数，将输入数据传入梯度函数中返回最终结果。\n", "\n", "MindSpore涉及梯度计算的其他组件，例如`WithGradCell`和`TrainOneStepCell`等，都用到了`GradOperation`， 感兴趣的读者可以查看这些接口的内部实现。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## WithLossCell\n", "\n", "`WithLossCell`本质上是一个包含损失函数的`Cell`，构造`WithLossCell`需要事先定义好网络和损失函数。\n", "\n", "下面通过一个实例来介绍其具体的使用， 首先需要构造一个网络，内容如下：" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "ExecuteTime": { "end_time": "2021-02-09T02:41:47.564738Z", "start_time": "2021-02-09T02:41:47.551810Z" } }, "outputs": [], "source": [ "import numpy as np\n", "\n", "import mindspore.context as context\n", "import mindspore.nn as nn\n", "from mindspore import Tensor\n", "from mindspore.nn import TrainOneStepCell, WithLossCell\n", "from mindspore.nn.optim import Momentum\n", "import mindspore.ops as ops\n", "\n", "context.set_context(mode=context.GRAPH_MODE, device_target=\"GPU\")\n", "\n", "\n", "class LeNet(nn.Cell):\n", " def __init__(self):\n", " super(LeNet, self).__init__()\n", " self.relu = ops.ReLU()\n", " self.batch_size = 32\n", "\n", " self.conv1 = nn.Conv2d(1, 6, kernel_size=5, stride=1, padding=0, has_bias=False, pad_mode='valid')\n", " self.conv2 = nn.Conv2d(6, 16, kernel_size=5, stride=1, padding=0, has_bias=False, pad_mode='valid')\n", " self.pool = nn.MaxPool2d(kernel_size=2, stride=2)\n", " self.reshape = ops.Reshape()\n", " self.fc1 = nn.Dense(400, 120)\n", " self.fc2 = nn.Dense(120, 84)\n", " self.fc3 = nn.Dense(84, 10)\n", "\n", " def construct(self, input_x):\n", " output = self.conv1(input_x)\n", " output = self.relu(output)\n", " output = self.pool(output)\n", " output = self.conv2(output)\n", " output = self.relu(output)\n", " output = self.pool(output)\n", " output = self.reshape(output, (self.batch_size, -1))\n", " output = self.fc1(output)\n", " output = self.relu(output)\n", " output = self.fc2(output)\n", " output = self.relu(output)\n", " output = self.fc3(output)\n", " return output" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "下面是`WithLossCell`的使用实例，分别定义好网络和损失函数，然后创建一个`WithLossCell`，传入输入数据和标签数据，`WithLossCell`内部根据网络和损失函数返回计算结果。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "ExecuteTime": { "end_time": "2021-02-09T02:41:47.802622Z", "start_time": "2021-02-09T02:41:47.567396Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "+++++++++Loss+++++++++++++\n", "2.302585\n" ] } ], "source": [ "data = Tensor(np.ones([32, 1, 32, 32]).astype(np.float32) * 0.01)\n", "label = Tensor(np.ones([32]).astype(np.int32))\n", "net = LeNet()\n", "criterion = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n", "net_with_criterion = WithLossCell(net, criterion)\n", "loss = net_with_criterion(data, label)\n", "print(\"+++++++++Loss+++++++++++++\")\n", "print(loss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## TrainOneStepCell\n", "\n", "`TrainOneStepCell`功能是执行网络的单步训练，返回每次训练结果后的loss结果。\n", "\n", "下面构造一个使用`TrainOneStepCell`接口进行网络训练的实例，其中`LeNet`和包名的导入代码和上个用例共用。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "ExecuteTime": { "end_time": "2021-02-09T02:41:48.319959Z", "start_time": "2021-02-09T02:41:47.804721Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "+++++++++result:0++++++++++++\n", "2.302585\n", "+++++++++result:1++++++++++++\n", "2.2935712\n", "+++++++++result:2++++++++++++\n", "2.2764661\n", "+++++++++result:3++++++++++++\n", "2.2521412\n", "+++++++++result:4++++++++++++\n", "2.2214084\n" ] } ], "source": [ "data = Tensor(np.ones([32, 1, 32, 32]).astype(np.float32) * 0.01)\n", "label = Tensor(np.ones([32]).astype(np.int32))\n", "net = LeNet()\n", "learning_rate = 0.01\n", "momentum = 0.9\n", "\n", "optimizer = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), learning_rate, momentum)\n", "criterion = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n", "net_with_criterion = WithLossCell(net, criterion)\n", "train_network = TrainOneStepCell(net_with_criterion, optimizer) # optimizer\n", "for i in range(5):\n", " train_network.set_train()\n", " res = train_network(data, label)\n", " print(f\"+++++++++result:{i}++++++++++++\")\n", " print(res)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "用例中构造了优化器和一个`WithLossCell`的实例，然后传入`TrainOneStepCell`中初始化一个训练网络，用例循环五次，相当于网络训练了五次，并输出每次的loss结果，由结果可以看出每次训练后loss值在逐渐减小。\n", "\n", "后续内容会介绍MindSpore使用更加高级封装的接口，即`Model`类中的`train`方法训练模型，在其内部实现中会用到 `TrainOneStepCell`和`WithLossCell` 等许多网络组件，感兴趣的读者可以查看其内部实现。" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }