{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true, "pycharm": { "name": "#%% md\n" } }, "source": [ "# Ascend910与GPU推理\n", "\n", "[![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.3/tutorials/source_zh_cn/intermediate/inference/ascend910_and_gpu_inference.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.3/tutorials/zh_cn/mindspore_ascend910_and_gpu_inference.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9yMS4zL3R1dG9yaWFscy96aF9jbi9taW5kc3BvcmVfYXNjZW5kOTEwX2FuZF9ncHVfaW5mZXJlbmNlLmlweW5i&imageid=65f636a0-56cf-49df-b941-7d2a07ba8c8c)\n", "\n", "\n", "本文将介绍如何在Ascend910和GPU硬件环境中，利用MindIR和Checkpoint执行推理。MindIR是MindSpore的统一模型文件，同时存储了网络结构和权重参数值，定义了可扩展的图结构以及算子的IR表示，消除了不同后端的模型差异，一般用于跨硬件平台执行推理任务。Checkpoint是训练参数，采用了Protocol Buffers格式，一般用于训练任务中断后恢复训练，或训练后的微调（Fine Tune）任务。\n", "\n", "下面将针对这两种情况，介绍如何使用MindSpore进行单卡推理。\n", "\n", "## 使用checkpoint格式文件单卡推理\n", "\n", "### 使用本地模型推理\n", "\n", "用户可以通过`load_checkpoint`和`load_param_into_net`接口从本地加载模型与参数，传入验证数据集后使用`model.eval`即可进行模型验证，使用`model.predict`可进行模型推理。在这里我们下载MindSpore Hub中已经预训练好的LeNet和MINIST数据集进行推理演示：" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "!mkdir -p ./datasets/MNIST_Data/test\n", "!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-labels-idx1-ubyte\n", "!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-images-idx3-ubyte\n", "!tree ./datasets/MNIST_Data\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "!mkdir -p ./checkpoint\n", "!wget -NP ./checkpoint https://download.mindspore.cn/model_zoo/r1.1/lenet_ascend_v111_offical_cv_mnist_bs32_acc98/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\n", "!tree ./checkpoint" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "配置运行所需信息，进行推理的数据处理：" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "import os\n", "import argparse\n", "\n", "from mindspore import context\n", "import mindspore.dataset as ds\n", "import mindspore.dataset.transforms.c_transforms as C\n", "import mindspore.dataset.vision.c_transforms as CV\n", "from mindspore.dataset.vision import Inter\n", "from mindspore import dtype as mstype\n", "\n", "context.set_context(mode=context.GRAPH_MODE, device_target=\"GPU\")\n", "\n", "def create_dataset(data_path, batch_size=32, repeat_size=1,\n", " num_parallel_workers=1):\n", " # 定义数据集\n", " mnist_ds = ds.MnistDataset(data_path)\n", " resize_height, resize_width = 32, 32\n", " rescale = 1.0 / 255.0\n", " shift = 0.0\n", " rescale_nml = 1 / 0.3081\n", " shift_nml = -1 * 0.1307 / 0.3081\n", "\n", " # 定义所需要操作的map映射\n", " resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)\n", " rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)\n", " rescale_op = CV.Rescale(rescale, shift)\n", " hwc2chw_op = CV.HWC2CHW()\n", " type_cast_op = C.TypeCast(mstype.int32)\n", "\n", " # 使用map映射函数，将数据操作应用到数据集\n", " mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns=\"label\", num_parallel_workers=num_parallel_workers)\n", " mnist_ds = mnist_ds.map(operations=resize_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n", " mnist_ds = mnist_ds.map(operations=rescale_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n", " mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n", " mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n", "\n", " # 进行shuffle、batch操作\n", " buffer_size = 10000\n", " mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)\n", " mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n", "\n", " return mnist_ds\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "创建LeNet模型：" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "import mindspore.nn as nn\n", "from mindspore.common.initializer import Normal\n", "\n", "class LeNet5(nn.Cell):\n", " \"\"\"\n", " Lenet网络结构\n", " \"\"\"\n", " def __init__(self, num_class=10, num_channel=1):\n", " super(LeNet5, self).__init__()\n", " # 定义所需要的运算\n", " self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')\n", " self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')\n", " self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))\n", " self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))\n", " self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))\n", " self.relu = nn.ReLU()\n", " self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n", " self.flatten = nn.Flatten()\n", "\n", " def construct(self, x):\n", " # 使用定义好的运算构建前向网络\n", " x = self.conv1(x)\n", " x = self.relu(x)\n", " x = self.max_pool2d(x)\n", " x = self.conv2(x)\n", " x = self.relu(x)\n", " x = self.max_pool2d(x)\n", " x = self.flatten(x)\n", " x = self.fc1(x)\n", " x = self.relu(x)\n", " x = self.fc2(x)\n", " x = self.relu(x)\n", " x = self.fc3(x)\n", " return x\n", "\n", "# 实例化网络\n", "net = LeNet5()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "在推理进行前，需要使用`load_checkpoint`和`load_param_into_net`接口从本地加载模型与参数。这样一来就可以使用本地模型完成后面的推理过程。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "from mindspore import load_checkpoint, load_param_into_net\n", "ckpt_file_name = \"./checkpoint/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\"\n", "param_dict = load_checkpoint(ckpt_file_name)\n", "load_param_into_net(net, param_dict)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "设置损失函数与优化器，并调用`model`接口创建对象：" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from mindspore.nn import Accuracy\n", "from mindspore import Model, Tensor\n", "\n", "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction=\"mean\")\n", "net_opt = nn.Momentum(net.trainable_params(), learning_rate=0.01, momentum=0.9)\n", "model = Model(net, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "下面调用`model.eval`接口执行验证过程：" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "============== {'Accuracy': 0.9846754807692307} ==============\n" ] } ], "source": [ "mnist_path = \"./datasets/MNIST_Data/test\"\n", "ds_eval = create_dataset(mnist_path)\n", "acc = model.eval(ds_eval, dataset_sink_mode=False)\n", "print(\"============== {} ==============\".format(acc))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "推理完整样例代码参见。\n", "\n", "调用`model.predict`接口执行验证过程，这里选取数据集中的一张图片进行预测：" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Predicted: \"6\", Actual: \"6\"\n" ] } ], "source": [ "ds_eval = ds_eval.create_dict_iterator()\n", "data = next(ds_eval)\n", "\n", "# images为测试图片，labels为测试图片的实际分类\n", "images = data[\"image\"].asnumpy()\n", "labels = data[\"label\"].asnumpy()\n", "\n", "# 使用函数model.predict预测image对应分类\n", "output = model.predict(Tensor(data['image']))\n", "predicted = np.argmax(output.asnumpy(), axis=1)\n", "\n", "# 输出预测分类与实际分类\n", "print(f'Predicted: \"{predicted[0]}\", Actual: \"{labels[0]}\"')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 加载MindSpore Hub模型执行推理\n", "\n", "除了使用`load_checkpoint`和`load_param_into_net`从本地加载模型之外，也可以通过[安装MindSpore Hub](https://gitee.com/mindspore/hub/blob/r1.3/README_CN.md)，通过`mindspore_hub.load`从云端加载模型参数执行推理。\n", "\n", "之前使用的加载本地模型的方法为：\n", "\n", "```python\n", "from mindspore import load_checkpoint, load_param_into_net\n", "ckpt_file_name = \"./checkpoint/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\"\n", "param_dict = load_checkpoint(ckpt_file_name)\n", "load_param_into_net(net, param_dict)\n", "```\n", "\n", "可替换为`mindspore_hub.load`方法：\n", "\n", "```python\n", "import mindspore_hub\n", "model_uid = \"mindspore/ascend/1.2/lenet_v1.2_mnist\"\n", "network = mindspore_hub.load(model_uid)\n", "```\n", "\n", "## 在Ascend中使用C++接口推理MindIR格式文件\n", "\n", "本小节将介绍如何使用C++接口推理MINDIR格式的模型。完整代码可参考[ascend910_resnet50_preprocess_sample](https://gitee.com/mindspore/docs/tree/r1.3/docs/sample_code/ascend910_resnet50_preprocess_sample)。\n", "\n", "> 本小节内容及代码仅适用于Ascend环境。\n", "\n", "### 推理代码介绍\n", "\n", "完成的推理代码为`main.cc`文件，现在对其中的功能实现进行说明。\n", "\n", "引用`mindspore`和`mindspore::dataset`的名字空间。\n", "\n", "```cpp\n", "namespace ms = mindspore;\n", "namespace ds = mindspore::dataset;\n", "```\n", "\n", "环境初始化，指定硬件为Ascend 910，DeviceID为0：\n", "\n", "```c++\n", "auto context = std::make_shared();\n", "auto ascend910_info = std::make_shared();\n", "ascend910_info->SetDeviceID(0);\n", "context->MutableDeviceInfo().push_back(ascend910_info);\n", "```\n", "\n", "加载模型文件:\n", "\n", "```cpp\n", "// 加载 MindIR 模型\n", "ms::Graph graph;\n", "ms::Status ret = ms::Serialization::Load(resnet_file, ms::ModelType::kMindIR, &graph);\n", "// 进行图编译\n", "ms::Model resnet50;\n", "ret = resnet50.Build(ms::GraphCell(graph), context);\n", "```\n", "\n", "获取模型所需输入信息：\n", "\n", "```cpp\n", "std::vector model_inputs = resnet50.GetInputs();\n", "```\n", "\n", "加载图片文件:\n", "\n", "```cpp\n", "ms::MSTensor ReadFile(const std::string &file);\n", "auto image = ReadFile(image_file);\n", "```\n", "\n", "图片预处理:\n", "\n", "```cpp\n", "// 对图片进行解码，变为RGB格式，并重设尺寸\n", "std::shared_ptr decode(new ds::vision::Decode());\n", "std::shared_ptr resize(new ds::vision::Resize({256}));\n", "// 输入归一化\n", "std::shared_ptr normalize(new ds::vision::Normalize(\n", " {0.485 * 255, 0.456 * 255, 0.406 * 255}, {0.229 * 255, 0.224 * 255, 0.225 * 255}));\n", "// 剪裁图片\n", "std::shared_ptr center_crop(new ds::vision::CenterCrop({224, 224}));\n", "// shape (H, W, C) 变为 shape (C, H, W)\n", "std::shared_ptr hwc2chw(new ds::vision::HWC2CHW());\n", "\n", "// 定义preprocessor\n", "ds::Execute preprocessor({decode, resize, normalize, center_crop, hwc2chw});\n", "\n", "// 调用函数，获取处理后的图像\n", "ret = preprocessor(image, &image);\n", "```\n", "\n", "执行推理:\n", "\n", "```cpp\n", "// 创建输入输出向量\n", "std::vector outputs;\n", "std::vector inputs;\n", "\n", "inputs.emplace_back(model_inputs[0].Name(), model_inputs[0].DataType(), model_inputs[0].Shape(),\n", " image.Data().get(), image.DataSize());\n", "// 执行推理\n", "ret = resnet50.Predict(inputs, &outputs);\n", "```\n", "\n", "获取推理结果:\n", "\n", "```cpp\n", "// 获取推理结果的最大可能性\n", "std::cout << \"Image: \" << image_file << \" infer result: \" << GetMax(outputs[0]) << std::endl;\n", "```\n", "\n", "### 构建脚本介绍\n", "\n", "构建脚本用于构建用户程序，完整代码位于`CMakeLists.txt` ，下面进行解释说明。\n", "\n", "为编译器添加头文件搜索路径：\n", "\n", "```cmake\n", "option(MINDSPORE_PATH \"mindspore install path\" \"\")\n", "include_directories(${MINDSPORE_PATH})\n", "include_directories(${MINDSPORE_PATH}/include)\n", "```\n", "\n", "在MindSpore中查找所需动态库：\n", "\n", "```cmake\n", "find_library(MS_LIB libmindspore.so ${MINDSPORE_PATH}/lib)\n", "file(GLOB_RECURSE MD_LIB ${MINDSPORE_PATH}/_c_dataengine*)\n", "```\n", "\n", "使用指定的源文件生成目标可执行文件，并为目标文件链接MindSpore库：\n", "\n", "```cmake\n", "add_executable(resnet50_sample main.cc)\n", "target_link_libraries(resnet50_sample ${MS_LIB} ${MD_LIB})\n", "```\n", "\n", "\n", "### 编译并执行推理代码\n", "\n", "可选择将实验的脚本下载至Ascend910环境中编译并执行。\n", "\n", "进入工程目录`ascend910_resnet50_preprocess_sample`，执行`cmake`命令，其中`pip3`需要按照实际情况修改：：\n", "\n", "```bash\n", "cmake . -DMINDSPORE_PATH=`pip3 show mindspore-ascend | grep Location | awk '{print $2\"/mindspore\"}' | xargs realpath`\n", "```\n", "\n", "再执行`make`命令编译即可。\n", "\n", "```bash\n", "make\n", "```\n", "\n", "编译成功后，会获得`resnet50_sample`可执行文件。在工程目录`ascend910_resnet50_preprocess_sample`下创建`model`目录放置MindIR文件[resnet50_imagenet.mindir](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/sample_resources/ascend310_resnet50_preprocess_sample/resnet50_imagenet.mindir)。此外，创建`test_data`目录用于存放待分类的图片，图片可来自ImageNet等各类开源数据集，输入执行命令即可获取推理结果：\n", "\n", "```bash\n", "./resnet50_sample\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" } }, "nbformat": 4, "nbformat_minor": 1 }