{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r2.3.1/tutorials/zh_cn/beginner/mindspore_model.ipynb) [](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r2.3.1/tutorials/zh_cn/beginner/mindspore_model.py) [](https://gitee.com/mindspore/docs/blob/r2.3.1/tutorials/source_zh_cn/beginner/model.ipynb)\n",
"\n",
"[基本介绍](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/introduction.html) || [快速入门](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/quick_start.html) || [张量 Tensor](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/tensor.html) || [数据集 Dataset](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/dataset.html) || [数据变换 Transforms](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/transforms.html) || **网络构建** || [函数式自动微分](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/autograd.html) || [模型训练](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/train.html) || [保存与加载](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/save_load.html) || [使用静态图加速](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/accelerate_with_static_graph.html)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 网络构建"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"神经网络模型是由神经网络层和Tensor操作构成的，[mindspore.nn](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/mindspore.nn.html)提供了常见神经网络层的实现，在MindSpore中，[Cell](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/nn/mindspore.nn.Cell.html)类是构建所有网络的基类，也是网络的基本单元。一个神经网络模型表示为一个`Cell`，它由不同的子`Cell`构成。使用这样的嵌套结构，可以简单地使用面向对象编程的思维，对神经网络结构进行构建和管理。\n",
"\n",
"下面我们将构建一个用于Mnist数据集分类的神经网络模型。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import mindspore\n",
"from mindspore import nn, ops"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## 定义模型类\n",
"\n",
"当我们定义神经网络时，可以继承`nn.Cell`类，在`__init__`方法中进行子Cell的实例化和状态管理，在`construct`方法中实现Tensor操作。\n",
"\n",
"> `construct`意为神经网络（计算图）构建，相关内容详见[使用静态图加速](https://www.mindspore.cn/tutorials/zh-CN/r2.3.1/beginner/accelerate_with_static_graph.html)。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"pycharm": {
"is_executing": true,
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"class Network(nn.Cell):\n",
" def __init__(self):\n",
" super().__init__()\n",
" self.flatten = nn.Flatten()\n",
" self.dense_relu_sequential = nn.SequentialCell(\n",
" nn.Dense(28*28, 512, weight_init=\"normal\", bias_init=\"zeros\"),\n",
" nn.ReLU(),\n",
" nn.Dense(512, 512, weight_init=\"normal\", bias_init=\"zeros\"),\n",
" nn.ReLU(),\n",
" nn.Dense(512, 10, weight_init=\"normal\", bias_init=\"zeros\")\n",
" )\n",
"\n",
" def construct(self, x):\n",
" x = self.flatten(x)\n",
" logits = self.dense_relu_sequential(x)\n",
" return logits"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"构建完成后，实例化`Network`对象，并查看其结构。"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/plain": [
"Network<\n",
" (flatten): Flatten<>\n",
" (dense_relu_sequential): SequentialCell<\n",
" (0): Dense<input_channels=784, output_channels=512, has_bias=True>\n",
" (1): ReLU<>\n",
" (2): Dense<input_channels=512, output_channels=512, has_bias=True>\n",
" (3): ReLU<>\n",
" (4): Dense<input_channels=512, output_channels=10, has_bias=True>\n",
" >\n",
" >"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model = Network()\n",
"print(model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"我们构造一个输入数据，直接调用模型，可以获得一个二维的Tensor输出，其包含每个类别的原始预测值。\n",
"\n",
"> `model.construct()`方法不可直接调用。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Tensor(shape=[1, 10], dtype=Float32, value=\n",
"[[-5.08734025e-04, 3.39190010e-04, 4.62840870e-03 ... -1.20305456e-03, -5.05689112e-03, 3.99264274e-03]])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X = ops.ones((1, 28, 28), mindspore.float32)\n",
"logits = model(X)\n",
"# print logits\n",
"logits"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"在此基础上，我们通过一个`nn.Softmax`层实例来获得预测概率。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Predicted class: [4]\n"
]
}
],
"source": [
"pred_probab = nn.Softmax(axis=1)(logits)\n",
"y_pred = pred_probab.argmax(1)\n",
"print(f\"Predicted class: {y_pred}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## 模型层\n",
"\n",
"本节中我们分解上节构造的神经网络模型中的每一层。首先我们构造一个shape为(3, 28, 28)的随机数据（3个28x28的图像），依次通过每一个神经网络层来观察其效果。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(3, 28, 28)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"input_image = ops.ones((3, 28, 28), mindspore.float32)\n",
"print(input_image.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"### nn.Flatten\n",
"\n",
"实例化[nn.Flatten](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/nn/mindspore.nn.Flatten.html)层，将28x28的2D张量转换为784大小的连续数组。"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/plain": [
"(3, 784)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"flatten = nn.Flatten()\n",
"flat_image = flatten(input_image)\n",
"print(flat_image.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"### nn.Dense\n",
"\n",
"[nn.Dense](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/nn/mindspore.nn.Dense.html)为全连接层，其使用权重和偏差对输入进行线性变换。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(3, 20)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"layer1 = nn.Dense(in_channels=28*28, out_channels=20)\n",
"hidden1 = layer1(flat_image)\n",
"print(hidden1.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"### nn.ReLU\n",
"\n",
"[nn.ReLU](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/nn/mindspore.nn.ReLU.html)层给网络中加入非线性的激活函数，帮助神经网络学习各种复杂的特征。"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Before ReLU: [[-0.04736331 0.2939465 -0.02713677 -0.30988005 -0.11504349 -0.11661264\n",
" 0.18007928 0.43213072 0.12091967 -0.17465964 0.53133243 0.12605792\n",
" 0.01825903 0.01287796 0.17238477 -0.1621131 -0.0080034 -0.24523425\n",
" -0.10083733 0.05171938]\n",
" [-0.04736331 0.2939465 -0.02713677 -0.30988005 -0.11504349 -0.11661264\n",
" 0.18007928 0.43213072 0.12091967 -0.17465964 0.53133243 0.12605792\n",
" 0.01825903 0.01287796 0.17238477 -0.1621131 -0.0080034 -0.24523425\n",
" -0.10083733 0.05171938]\n",
" [-0.04736331 0.2939465 -0.02713677 -0.30988005 -0.11504349 -0.11661264\n",
" 0.18007928 0.43213072 0.12091967 -0.17465964 0.53133243 0.12605792\n",
" 0.01825903 0.01287796 0.17238477 -0.1621131 -0.0080034 -0.24523425\n",
" -0.10083733 0.05171938]]\n",
"\n",
"\n",
"After ReLU: [[0. 0.2939465 0. 0. 0. 0.\n",
" 0.18007928 0.43213072 0.12091967 0. 0.53133243 0.12605792\n",
" 0.01825903 0.01287796 0.17238477 0. 0. 0.\n",
" 0. 0.05171938]\n",
" [0. 0.2939465 0. 0. 0. 0.\n",
" 0.18007928 0.43213072 0.12091967 0. 0.53133243 0.12605792\n",
" 0.01825903 0.01287796 0.17238477 0. 0. 0.\n",
" 0. 0.05171938]\n",
" [0. 0.2939465 0. 0. 0. 0.\n",
" 0.18007928 0.43213072 0.12091967 0. 0.53133243 0.12605792\n",
" 0.01825903 0.01287796 0.17238477 0. 0. 0.\n",
" 0. 0.05171938]]\n"
]
}
],
"source": [
"print(f\"Before ReLU: {hidden1}\\n\\n\")\n",
"hidden1 = nn.ReLU()(hidden1)\n",
"print(f\"After ReLU: {hidden1}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"### nn.SequentialCell\n",
"\n",
"[nn.SequentialCell](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/nn/mindspore.nn.SequentialCell.html)是一个有序的Cell容器。输入Tensor将按照定义的顺序通过所有Cell。我们可以使用`nn.SequentialCell`来快速组合构造一个神经网络模型。"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/plain": [
"(3, 10)"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"seq_modules = nn.SequentialCell(\n",
" flatten,\n",
" layer1,\n",
" nn.ReLU(),\n",
" nn.Dense(20, 10)\n",
")\n",
"\n",
"logits = seq_modules(input_image)\n",
"print(logits.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### nn.Softmax\n",
"\n",
"最后使用[nn.Softmax](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/nn/mindspore.nn.Softmax.html)将神经网络最后一个全连接层返回的logits的值缩放为\\[0, 1\\]，表示每个类别的预测概率。`axis`指定的维度数值和为1。"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"softmax = nn.Softmax(axis=1)\n",
"pred_probab = softmax(logits)"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## 模型参数\n",
"\n",
"网络内部神经网络层具有权重参数和偏置参数（如`nn.Dense`），这些参数会在训练过程中不断进行优化，可通过 `model.parameters_and_names()` 来获取参数名及对应的参数详情。"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model structure: Network<\n",
" (flatten): Flatten<>\n",
" (dense_relu_sequential): SequentialCell<\n",
" (0): Dense<input_channels=784, output_channels=512, has_bias=True>\n",
" (1): ReLU<>\n",
" (2): Dense<input_channels=512, output_channels=512, has_bias=True>\n",
" (3): ReLU<>\n",
" (4): Dense<input_channels=512, output_channels=10, has_bias=True>\n",
" >\n",
" >\n",
"\n",
"\n",
"Layer: dense_relu_sequential.0.weight\n",
"Size: (512, 784)\n",
"Values : [[-0.01491369 0.00353318 -0.00694948 ... 0.01226766 -0.00014423\n",
" 0.00544263]\n",
" [ 0.00212971 0.0019974 -0.00624789 ... -0.01214037 0.00118004\n",
" -0.01594325]] \n",
"\n",
"Layer: dense_relu_sequential.0.bias\n",
"Size: (512,)\n",
"Values : [0. 0.] \n",
"\n",
"Layer: dense_relu_sequential.2.weight\n",
"Size: (512, 512)\n",
"Values : [[ 0.00565423 0.00354313 0.00637383 ... -0.00352688 0.00262949\n",
" 0.01157355]\n",
" [-0.01284141 0.00657666 -0.01217057 ... 0.00318963 0.00319115\n",
" -0.00186801]] \n",
"\n",
"Layer: dense_relu_sequential.2.bias\n",
"Size: (512,)\n",
"Values : [0. 0.] \n",
"\n",
"Layer: dense_relu_sequential.4.weight\n",
"Size: (10, 512)\n",
"Values : [[ 0.0087168 -0.00381866 -0.00865665 ... -0.00273731 -0.00391623\n",
" 0.00612853]\n",
" [-0.00593031 0.0008721 -0.0060081 ... -0.00271535 -0.00850481\n",
" -0.00820513]] \n",
"\n",
"Layer: dense_relu_sequential.4.bias\n",
"Size: (10,)\n",
"Values : [0. 0.] \n",
"\n"
]
}
],
"source": [
"print(f\"Model structure: {model}\\n\\n\")\n",
"\n",
"for name, param in model.parameters_and_names():\n",
" print(f\"Layer: {name}\\nSize: {param.shape}\\nValues : {param[:2]} \\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"更多内置神经网络层详见[mindspore.nn API](https://www.mindspore.cn/docs/zh-CN/r2.3.1/api_python/mindspore.nn.html)。"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "MindSpore",
"language": "python",
"name": "mindspore"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}