{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[![在线运行](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9vYnMuZHVhbHN0YWNrLmNuLW5vcnRoLTQubXlodWF3ZWljbG91ZC5jb20vbWluZHNwb3JlLXdlYnNpdGUvbm90ZWJvb2svcjIuMC4wLWFscGhhL3R1dG9yaWFscy96aF9jbi9iZWdpbm5lci9taW5kc3BvcmVfbW9kZWwuaXB5bmI=&imageid=77ef960a-bd26-4de4-9695-5b85a786fb90) [![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r2.0.0-alpha/tutorials/zh_cn/beginner/mindspore_model.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r2.0.0-alpha/tutorials/zh_cn/beginner/mindspore_model.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r2.0.0-alpha/tutorials/source_zh_cn/beginner/model.ipynb)\n", "\n", "[基本介绍](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/introduction.html) || [快速入门](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/quick_start.html) || [张量 Tensor](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/tensor.html) || [数据集 Dataset](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/dataset.html) || [数据变换 Transforms](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/transforms.html) || **网络构建** || [函数式自动微分](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/autograd.html) || [模型训练](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/train.html) || [保存与加载](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/beginner/save_load.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 网络构建" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "神经网络模型是由神经网络层和Tensor操作构成的,`mindspore.nn`提供了常见神经网络层的实现,在MindSpore中,`Cell`类是构建所有网络的基类,也是网络的基本单元。一个神经网络模型表示为一个`Cell`,它由不同的子`Cell`构成。使用这样的嵌套结构,可以简单地使用面向对象编程的思维,对神经网络结构进行构建和管理。\n", "\n", "下面我们将构建一个用于Mnist数据集分类的神经网络模型。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import mindspore\n", "from mindspore import nn, ops" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## 定义模型类\n", "\n", "当我们定义神经网络时,可以继承`nn.Cell`类,在`__init__`方法中进行子Cell的实例化和状态管理,在`construct`方法中实现Tensor操作。\n", "\n", "> `construct`意为神经网络(计算图)构建,相关内容详见[计算图](https://www.mindspore.cn/tutorials/zh-CN/r2.0.0-alpha/advanced/compute_graph.html)。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "class Network(nn.Cell):\n", " def __init__(self):\n", " super().__init__()\n", " self.flatten = nn.Flatten()\n", " self.dense_relu_sequential = nn.SequentialCell(\n", " nn.Dense(28*28, 512),\n", " nn.ReLU(),\n", " nn.Dense(512, 512),\n", " nn.ReLU(),\n", " nn.Dense(512, 10)\n", " )\n", "\n", " def construct(self, x):\n", " x = self.flatten(x)\n", " logits = self.dense_relu_sequential(x)\n", " return logits" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "构建完成后,实例化`Network`对象,并查看其结构。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "Network<\n", " (flatten): Flatten<>\n", " (dense_relu_sequential): SequentialCell<\n", " (0): Dense\n", " (1): ReLU<>\n", " (2): Dense\n", " (3): ReLU<>\n", " (4): Dense\n", " >\n", " >" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model = Network()\n", "print(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "我们构造一个输入数据,直接调用模型,可以获得一个10维的Tensor输出,其包含每个类别的原始预测值。\n", "\n", "> `model.construct()`方法不可直接调用。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Tensor(shape=[1, 10], dtype=Float32, value=\n", "[[-5.08734025e-04, 3.39190010e-04, 4.62840870e-03 ... -1.20305456e-03, -5.05689112e-03, 3.99264274e-03]])" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X = ops.ones((1, 28, 28), mindspore.float32)\n", "logits = model(X)\n", "print(logits)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "在此基础上,我们通过一个`nn.Softmax`层实例来获得预测概率。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Predicted class: [4]\n" ] } ], "source": [ "pred_probab = nn.Softmax(axis=1)(logits)\n", "y_pred = pred_probab.argmax(1)\n", "print(f\"Predicted class: {y_pred}\")" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## 模型层\n", "\n", "本节中我们分解上节构造的神经网络模型中的每一层。首先我们构造一个shape为(3, 28, 28)的随机数据(3个28x28的图像),依次通过每一个神经网络层来观察其效果。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(3, 28, 28)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "input_image = ops.ones((3, 28, 28), mindspore.float32)\n", "print(input_image.shape)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### nn.Flatten\n", "\n", "实例化`nn.Flatten`层,将28x28的2D张量转换为784大小的连续数组。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(3, 784)" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "flatten = nn.Flatten()\n", "flat_image = flatten(input_image)\n", "print(flat_image.shape)" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### nn.Dense\n", "\n", "`nn.Dense`为全连接层,其使用权重和偏差对输入进行线性变换。" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(3, 20)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "layer1 = nn.Dense(in_channels=28*28, out_channels=20)\n", "hidden1 = layer1(flat_image)\n", "print(hidden1.shape)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### nn.ReLU\n", "\n", "`nn.ReLU`层给网络中加入非线性的激活函数,帮助神经网络学习各种复杂的特征。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before ReLU: [[-0.04736331 0.2939465 -0.02713677 -0.30988005 -0.11504349 -0.11661264\n", " 0.18007928 0.43213072 0.12091967 -0.17465964 0.53133243 0.12605792\n", " 0.01825903 0.01287796 0.17238477 -0.1621131 -0.0080034 -0.24523425\n", " -0.10083733 0.05171938]\n", " [-0.04736331 0.2939465 -0.02713677 -0.30988005 -0.11504349 -0.11661264\n", " 0.18007928 0.43213072 0.12091967 -0.17465964 0.53133243 0.12605792\n", " 0.01825903 0.01287796 0.17238477 -0.1621131 -0.0080034 -0.24523425\n", " -0.10083733 0.05171938]\n", " [-0.04736331 0.2939465 -0.02713677 -0.30988005 -0.11504349 -0.11661264\n", " 0.18007928 0.43213072 0.12091967 -0.17465964 0.53133243 0.12605792\n", " 0.01825903 0.01287796 0.17238477 -0.1621131 -0.0080034 -0.24523425\n", " -0.10083733 0.05171938]]\n", "\n", "\n", "After ReLU: [[0. 0.2939465 0. 0. 0. 0.\n", " 0.18007928 0.43213072 0.12091967 0. 0.53133243 0.12605792\n", " 0.01825903 0.01287796 0.17238477 0. 0. 0.\n", " 0. 0.05171938]\n", " [0. 0.2939465 0. 0. 0. 0.\n", " 0.18007928 0.43213072 0.12091967 0. 0.53133243 0.12605792\n", " 0.01825903 0.01287796 0.17238477 0. 0. 0.\n", " 0. 0.05171938]\n", " [0. 0.2939465 0. 0. 0. 0.\n", " 0.18007928 0.43213072 0.12091967 0. 0.53133243 0.12605792\n", " 0.01825903 0.01287796 0.17238477 0. 0. 0.\n", " 0. 0.05171938]]\n" ] } ], "source": [ "print(f\"Before ReLU: {hidden1}\\n\\n\")\n", "hidden1 = nn.ReLU()(hidden1)\n", "print(f\"After ReLU: {hidden1}\")" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### nn.SequentialCell\n", "\n", "`nn.SequentialCell`是一个有序的Cell容器。输入Tensor将按照定义的顺序通过所有Cell。我们可以使用`SequentialCell`来快速组合构造一个神经网络模型。" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(3, 10)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seq_modules = nn.SequentialCell(\n", " flatten,\n", " layer1,\n", " nn.ReLU(),\n", " nn.Dense(20, 10)\n", ")\n", "\n", "logits = seq_modules(input_image)\n", "print(logits.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### nn.Softmax\n", "\n", "最后使用`nn.Softmax`将神经网络最后一个全连接层返回的logits的值缩放为\\[0, 1\\],表示每个类别的预测概率。`axis`指定的维度数值和为1。" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "softmax = nn.Softmax(axis=1)\n", "pred_probab = softmax(logits)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## 模型参数\n", "\n", "网络内部神经网络层具有权重参数和偏置参数(如`nn.Dense`),这些参数会在训练过程中不断进行优化,可通过 `model.parameters_and_names()` 来获取参数名及对应的参数详情。" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model structure: Network<\n", " (flatten): Flatten<>\n", " (dense_relu_sequential): SequentialCell<\n", " (0): Dense\n", " (1): ReLU<>\n", " (2): Dense\n", " (3): ReLU<>\n", " (4): Dense\n", " >\n", " >\n", "\n", "\n", "Layer: dense_relu_sequential.0.weight\n", "Size: (512, 784)\n", "Values : [[-0.01491369 0.00353318 -0.00694948 ... 0.01226766 -0.00014423\n", " 0.00544263]\n", " [ 0.00212971 0.0019974 -0.00624789 ... -0.01214037 0.00118004\n", " -0.01594325]] \n", "\n", "Layer: dense_relu_sequential.0.bias\n", "Size: (512,)\n", "Values : [0. 0.] \n", "\n", "Layer: dense_relu_sequential.2.weight\n", "Size: (512, 512)\n", "Values : [[ 0.00565423 0.00354313 0.00637383 ... -0.00352688 0.00262949\n", " 0.01157355]\n", " [-0.01284141 0.00657666 -0.01217057 ... 0.00318963 0.00319115\n", " -0.00186801]] \n", "\n", "Layer: dense_relu_sequential.2.bias\n", "Size: (512,)\n", "Values : [0. 0.] \n", "\n", "Layer: dense_relu_sequential.4.weight\n", "Size: (10, 512)\n", "Values : [[ 0.0087168 -0.00381866 -0.00865665 ... -0.00273731 -0.00391623\n", " 0.00612853]\n", " [-0.00593031 0.0008721 -0.0060081 ... -0.00271535 -0.00850481\n", " -0.00820513]] \n", "\n", "Layer: dense_relu_sequential.4.bias\n", "Size: (10,)\n", "Values : [0. 0.] \n", "\n" ] } ], "source": [ "print(f\"Model structure: {model}\\n\\n\")\n", "\n", "for name, param in model.parameters_and_names():\n", " print(f\"Layer: {name}\\nSize: {param.shape}\\nValues : {param[:2]} \\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "更多内置神经网络层详见[mindspore.nn API](https://www.mindspore.cn/docs/zh-CN/r2.0.0-alpha/api_python/mindspore.nn.html)。" ] } ], "metadata": { "kernelspec": { "display_name": "MindSpore", "language": "python", "name": "mindspore" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" } }, "nbformat": 4, "nbformat_minor": 4 }