{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[![在线运行](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9vYnMuZHVhbHN0YWNrLmNuLW5vcnRoLTQubXlodWF3ZWljbG91ZC5jb20vbWluZHNwb3JlLXdlYnNpdGUvbm90ZWJvb2svcjIuMC90dXRvcmlhbHMvemhfY24vYmVnaW5uZXIvbWluZHNwb3JlX3F1aWNrX3N0YXJ0LmlweW5i=&imageid=e225a9aa-230a-4ea5-a538-b5faed64a6a6) [![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0/resource/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r2.0/tutorials/zh_cn/beginner/mindspore_quick_start.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0/resource/_static/logo_download_code.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r2.0/tutorials/zh_cn/beginner/mindspore_quick_start.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r2.0/tutorials/source_zh_cn/beginner/quick_start.ipynb)\n", "\n", "[基本介绍](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/introduction.html) || **快速入门** || [张量 Tensor](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/tensor.html) || [数据集 Dataset](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/dataset.html) || [数据变换 Transforms](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/transforms.html) || [网络构建](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/model.html) || [函数式自动微分](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/autograd.html) || [模型训练](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/train.html) || [保存与加载](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/save_load.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 快速入门\n", "\n", "本节通过MindSpore的API来快速实现一个简单的深度学习模型。若想要深入了解MindSpore的使用方法,请参阅各节最后提供的参考链接。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import mindspore\n", "from mindspore import nn\n", "from mindspore.dataset import vision, transforms\n", "from mindspore.dataset import MnistDataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 处理数据集\n", "\n", "MindSpore提供基于Pipeline的[数据引擎](https://www.mindspore.cn/docs/zh-CN/r2.0/design/data_engine.html),通过[数据集(Dataset)](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/dataset.html)和[数据变换(Transforms)](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/transforms.html)实现高效的数据预处理。在本教程中,我们使用Mnist数据集,自动下载完成后,使用`mindspore.dataset`提供的数据变换进行预处理。\n", "\n", "> 本章节中的示例代码依赖`download`,可使用命令`pip install download`安装。如本文档以Notebook运行时,完成安装后需要重启kernel才能执行后续代码。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading data from https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/MNIST_Data.zip (10.3 MB)\n", "\n", "file_sizes: 100%|██████████████████████████| 10.8M/10.8M [00:01<00:00, 6.73MB/s]\n", "Extracting zip file...\n", "Successfully downloaded / unzipped to ./\n" ] } ], "source": [ "# Download data from open datasets\n", "from download import download\n", "\n", "url = \"https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/\" \\\n", " \"notebook/datasets/MNIST_Data.zip\"\n", "path = download(url, \"./\", kind=\"zip\", replace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "数据下载完成后,获得数据集对象。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "train_dataset = MnistDataset('MNIST_Data/train')\n", "test_dataset = MnistDataset('MNIST_Data/test')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "打印数据集中包含的数据列名,用于dataset的预处理。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['image', 'label']\n" ] } ], "source": [ "print(train_dataset.get_col_names())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MindSpore的dataset使用数据处理流水线(Data Processing Pipeline),需指定map、batch、shuffle等操作。这里我们使用map对图像数据及标签进行变换处理,然后将处理好的数据集打包为大小为64的batch。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def datapipe(dataset, batch_size):\n", " image_transforms = [\n", " vision.Rescale(1.0 / 255.0, 0),\n", " vision.Normalize(mean=(0.1307,), std=(0.3081,)),\n", " vision.HWC2CHW()\n", " ]\n", " label_transform = transforms.TypeCast(mindspore.int32)\n", "\n", " dataset = dataset.map(image_transforms, 'image')\n", " dataset = dataset.map(label_transform, 'label')\n", " dataset = dataset.batch(batch_size)\n", " return dataset" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# Map vision transforms and batch dataset\n", "train_dataset = datapipe(train_dataset, 64)\n", "test_dataset = datapipe(test_dataset, 64)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "使用`create_tuple_iterator`或`create_dict_iterator`对数据集进行迭代。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape of image [N, C, H, W]: (64, 1, 28, 28) Float32\n", "Shape of label: (64,) Int32\n" ] } ], "source": [ "for image, label in test_dataset.create_tuple_iterator():\n", " print(f\"Shape of image [N, C, H, W]: {image.shape} {image.dtype}\")\n", " print(f\"Shape of label: {label.shape} {label.dtype}\")\n", " break" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape of image [N, C, H, W]: (64, 1, 28, 28) Float32\n", "Shape of label: (64,) Int32\n" ] } ], "source": [ "for data in test_dataset.create_dict_iterator():\n", " print(f\"Shape of image [N, C, H, W]: {data['image'].shape} {data['image'].dtype}\")\n", " print(f\"Shape of label: {data['label'].shape} {data['label'].dtype}\")\n", " break" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "更多细节详见[数据集 Dataset](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/dataset.html)与[数据变换 Transforms](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/transforms.html)。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 网络构建\n", "\n", "`mindspore.nn`类是构建所有网络的基类,也是网络的基本单元。当用户需要自定义网络时,可以继承`nn.Cell`类,并重写`__init__`方法和`construct`方法。`__init__`包含所有网络层的定义,`construct`中包含数据([Tensor](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/tensor.html))的变换过程(即[计算图](https://www.mindspore.cn/tutorials/zh-CN/r2.0/advanced/compute_graph.html)的构造过程)。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Network<\n", " (flatten): Flatten<>\n", " (dense_relu_sequential): SequentialCell<\n", " (0): Dense\n", " (1): ReLU<>\n", " (2): Dense\n", " (3): ReLU<>\n", " (4): Dense\n", " >\n", " >\n" ] } ], "source": [ "# Define model\n", "class Network(nn.Cell):\n", " def __init__(self):\n", " super().__init__()\n", " self.flatten = nn.Flatten()\n", " self.dense_relu_sequential = nn.SequentialCell(\n", " nn.Dense(28*28, 512),\n", " nn.ReLU(),\n", " nn.Dense(512, 512),\n", " nn.ReLU(),\n", " nn.Dense(512, 10)\n", " )\n", "\n", " def construct(self, x):\n", " x = self.flatten(x)\n", " logits = self.dense_relu_sequential(x)\n", " return logits\n", "\n", "model = Network()\n", "print(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "更多细节详见[网络构建](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/model.html)。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 模型训练" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# Instantiate loss function and optimizer\n", "loss_fn = nn.CrossEntropyLoss()\n", "optimizer = nn.SGD(model.trainable_params(), 1e-2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "在模型训练中,一个完整的训练过程(step)需要实现以下三步:\n", "\n", "1. **正向计算**:模型预测结果(logits),并与正确标签(label)求预测损失(loss)。\n", "2. **反向传播**:利用自动微分机制,自动求模型参数(parameters)对于loss的梯度(gradients)。\n", "3. **参数优化**:将梯度更新到参数上。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MindSpore使用函数式自动微分机制,因此针对上述步骤需要实现:\n", "\n", "1. 正向计算函数定义。\n", "2. 通过函数变换获得梯度计算函数。\n", "3. 训练函数定义,执行正向计算、反向传播和参数优化。" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# Define forward function\n", "def forward_fn(data, label):\n", " logits = model(data)\n", " loss = loss_fn(logits, label)\n", " return loss, logits\n", "\n", "# Get gradient function\n", "grad_fn = mindspore.value_and_grad(forward_fn, None, optimizer.parameters, has_aux=True)\n", "\n", "# Define function of one-step training\n", "def train_step(data, label):\n", " (loss, _), grads = grad_fn(data, label)\n", " optimizer(grads)\n", " return loss\n", "\n", "def train(model, dataset):\n", " size = dataset.get_dataset_size()\n", " model.set_train()\n", " for batch, (data, label) in enumerate(dataset.create_tuple_iterator()):\n", " loss = train_step(data, label)\n", "\n", " if batch % 100 == 0:\n", " loss, current = loss.asnumpy(), batch\n", " print(f\"loss: {loss:>7f} [{current:>3d}/{size:>3d}]\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "除训练外,我们定义测试函数,用来评估模型的性能。" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "def test(model, dataset, loss_fn):\n", " num_batches = dataset.get_dataset_size()\n", " model.set_train(False)\n", " total, test_loss, correct = 0, 0, 0\n", " for data, label in dataset.create_tuple_iterator():\n", " pred = model(data)\n", " total += len(data)\n", " test_loss += loss_fn(pred, label).asnumpy()\n", " correct += (pred.argmax(1) == label).asnumpy().sum()\n", " test_loss /= num_batches\n", " correct /= total\n", " print(f\"Test: \\n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "训练过程需多次迭代数据集,一次完整的迭代称为一轮(epoch)。在每一轮,遍历训练集进行训练,结束后使用测试集进行预测。打印每一轮的loss值和预测准确率(Accuracy),可以看到loss在不断下降,Accuracy在不断提高。" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1\n", "-------------------------------\n", "loss: 2.302088 [ 0/938]\n", "loss: 2.290692 [100/938]\n", "loss: 2.266338 [200/938]\n", "loss: 2.205240 [300/938]\n", "loss: 1.907198 [400/938]\n", "loss: 1.455603 [500/938]\n", "loss: 0.861103 [600/938]\n", "loss: 0.767219 [700/938]\n", "loss: 0.422253 [800/938]\n", "loss: 0.513922 [900/938]\n", "Test: \n", " Accuracy: 83.8%, Avg loss: 0.529534 \n", "\n", "Epoch 2\n", "-------------------------------\n", "loss: 0.580867 [ 0/938]\n", "loss: 0.479347 [100/938]\n", "loss: 0.677991 [200/938]\n", "loss: 0.550141 [300/938]\n", "loss: 0.226565 [400/938]\n", "loss: 0.314738 [500/938]\n", "loss: 0.298739 [600/938]\n", "loss: 0.459540 [700/938]\n", "loss: 0.332978 [800/938]\n", "loss: 0.406709 [900/938]\n", "Test: \n", " Accuracy: 90.2%, Avg loss: 0.334828 \n", "\n", "Epoch 3\n", "-------------------------------\n", "loss: 0.461890 [ 0/938]\n", "loss: 0.242303 [100/938]\n", "loss: 0.281414 [200/938]\n", "loss: 0.207835 [300/938]\n", "loss: 0.206000 [400/938]\n", "loss: 0.409646 [500/938]\n", "loss: 0.193608 [600/938]\n", "loss: 0.217575 [700/938]\n", "loss: 0.212817 [800/938]\n", "loss: 0.202862 [900/938]\n", "Test: \n", " Accuracy: 91.9%, Avg loss: 0.280962 \n", "\n", "Done!\n" ] } ], "source": [ "epochs = 3\n", "for t in range(epochs):\n", " print(f\"Epoch {t+1}\\n-------------------------------\")\n", " train(model, train_dataset)\n", " test(model, test_dataset, loss_fn)\n", "print(\"Done!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "更多细节详见[模型训练](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/train.html)。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 保存模型\n", "\n", "模型训练完成后,需要将其参数进行保存。" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Saved Model to model.ckpt\n" ] } ], "source": [ "# Save checkpoint\n", "mindspore.save_checkpoint(model, \"model.ckpt\")\n", "print(\"Saved Model to model.ckpt\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 加载模型" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "加载保存的权重分为两步:\n", "\n", "1. 重新实例化模型对象,构造模型。\n", "2. 加载模型参数,并将其加载至模型上。" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[]\n" ] } ], "source": [ "# Instantiate a random initialized model\n", "model = Network()\n", "# Load checkpoint and load parameter to model\n", "param_dict = mindspore.load_checkpoint(\"model.ckpt\")\n", "param_not_load, _ = mindspore.load_param_into_net(model, param_dict)\n", "print(param_not_load)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> `param_not_load`是未被加载的参数列表,为空时代表所有参数均加载成功。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "加载后的模型可以直接用于预测推理。" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Predicted: \"Tensor(shape=[10], dtype=Int32, value= [3, 9, 6, 1, 6, 7, 4, 5, 2, 2])\", Actual: \"Tensor(shape=[10], dtype=Int32, value= [3, 9, 6, 1, 6, 7, 4, 5, 2, 2])\"\n" ] } ], "source": [ "model.set_train(False)\n", "for data, label in test_dataset:\n", " pred = model(data)\n", " predicted = pred.argmax(1)\n", " print(f'Predicted: \"{predicted[:10]}\", Actual: \"{label[:10]}\"')\n", " break" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "更多细节详见[保存与加载](https://www.mindspore.cn/tutorials/zh-CN/r2.0/beginner/save_load.html)。" ] } ], "metadata": { "kernelspec": { "display_name": "MindSpore", "language": "python", "name": "mindspore" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5 (default, Oct 25 2019, 15:51:11) \n[GCC 7.3.0]" }, "vscode": { "interpreter": { "hash": "8c9da313289c39257cb28b126d2dadd33153d4da4d524f730c81a4aaccbd2ca7" } } }, "nbformat": 4, "nbformat_minor": 4 }