{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"# Ascend910与GPU推理\n",
"\n",
"[](https://gitee.com/mindspore/docs/blob/r1.2/tutorials/source_zh_cn/intermediate/inference/ascend910_and_gpu_inference.ipynb) [](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.2/quick_start/mindspore_ascend910_and_gpu_inference.ipynb) [](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9tYXN0ZXIvdHV0b3JpYWxzL3poX2NuL21pbmRzcG9yZV9hc2NlbmQ5MTBfYW5kX2dwdV9pbmZlcmVuY2UuaXB5bmI=&imageid=65f636a0-56cf-49df-b941-7d2a07ba8c8c)\n",
"\n",
"本文将介绍如何在Ascend910和GPU硬件环境中，利用MindIR和Checkpoint执行推理。MindIR是MindSpore的统一模型文件，同时存储了网络结构和权重参数值，定义了可扩展的图结构以及算子的IR表示，消除了不同后端的模型差异，一般用于跨硬件平台执行推理任务。Checkpoint是训练参数，采用了Protocol Buffers格式，一般用于训练任务中断后恢复训练，或训练后的微调（Fine Tune）任务。\n",
"\n",
"下面将针对这两种情况，介绍如何使用MindSpore进行单卡推理。\n",
"\n",
"## 使用checkpoint格式文件单卡推理\n",
"\n",
"### 使用本地模型推理\n",
"\n",
"用户可以通过`load_checkpoint`和`load_param_into_net`接口从本地加载模型与参数，传入验证数据集后使用`model.eval`即可进行模型验证，使用`model.predict`可进行模型推理。在这里我们下载MindSpore Hub中已经预训练好的LeNet和MINIST数据集进行推理演示："
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"!mkdir -p ./datasets/MNIST_Data/test\n",
"!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-labels-idx1-ubyte\n",
"!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-images-idx3-ubyte\n",
"!tree ./datasets/MNIST_Data\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"!mkdir -p ./checkpoint\n",
"!wget -NP ./checkpoint https://download.mindspore.cn/model_zoo/r1.1/lenet_ascend_v111_offical_cv_mnist_bs32_acc98/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\n",
"!tree ./checkpoint"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"配置运行所需信息，进行推理的数据处理："
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import os\n",
"import argparse\n",
"\n",
"from mindspore import context\n",
"import mindspore.dataset as ds\n",
"import mindspore.dataset.transforms.c_transforms as C\n",
"import mindspore.dataset.vision.c_transforms as CV\n",
"from mindspore.dataset.vision import Inter\n",
"from mindspore import dtype as mstype\n",
"\n",
"context.set_context(mode=context.GRAPH_MODE, device_target=\"GPU\")\n",
"\n",
"def create_dataset(data_path, batch_size=32, repeat_size=1,\n",
" num_parallel_workers=1):\n",
" # 定义数据集\n",
" mnist_ds = ds.MnistDataset(data_path)\n",
" resize_height, resize_width = 32, 32\n",
" rescale = 1.0 / 255.0\n",
" shift = 0.0\n",
" rescale_nml = 1 / 0.3081\n",
" shift_nml = -1 * 0.1307 / 0.3081\n",
"\n",
" # 定义所需要操作的map映射\n",
" resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)\n",
" rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)\n",
" rescale_op = CV.Rescale(rescale, shift)\n",
" hwc2chw_op = CV.HWC2CHW()\n",
" type_cast_op = C.TypeCast(mstype.int32)\n",
"\n",
" # 使用map映射函数，将数据操作应用到数据集\n",
" mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns=\"label\", num_parallel_workers=num_parallel_workers)\n",
" mnist_ds = mnist_ds.map(operations=resize_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
" mnist_ds = mnist_ds.map(operations=rescale_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
" mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
" mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
"\n",
" # 进行shuffle、batch操作\n",
" buffer_size = 10000\n",
" mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)\n",
" mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n",
"\n",
" return mnist_ds\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"创建LeNet模型："
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import mindspore.nn as nn\n",
"from mindspore.common.initializer import Normal\n",
"\n",
"class LeNet5(nn.Cell):\n",
" \"\"\"\n",
" Lenet网络结构\n",
" \"\"\"\n",
" def __init__(self, num_class=10, num_channel=1):\n",
" super(LeNet5, self).__init__()\n",
" # 定义所需要的运算\n",
" self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')\n",
" self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')\n",
" self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))\n",
" self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))\n",
" self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))\n",
" self.relu = nn.ReLU()\n",
" self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n",
" self.flatten = nn.Flatten()\n",
"\n",
" def construct(self, x):\n",
" # 使用定义好的运算构建前向网络\n",
" x = self.conv1(x)\n",
" x = self.relu(x)\n",
" x = self.max_pool2d(x)\n",
" x = self.conv2(x)\n",
" x = self.relu(x)\n",
" x = self.max_pool2d(x)\n",
" x = self.flatten(x)\n",
" x = self.fc1(x)\n",
" x = self.relu(x)\n",
" x = self.fc2(x)\n",
" x = self.relu(x)\n",
" x = self.fc3(x)\n",
" return x\n",
"\n",
"# 实例化网络\n",
"net = LeNet5()"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"在推理进行前，需要使用`load_checkpoint`和`load_param_into_net`接口从本地加载模型与参数。这样一来就可以使用本地模型完成后面的推理过程。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"from mindspore import load_checkpoint, load_param_into_net\n",
"ckpt_file_name = \"./checkpoint/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\"\n",
"param_dict = load_checkpoint(ckpt_file_name)\n",
"load_param_into_net(net, param_dict)"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"设置损失函数与优化器，并调用`model`接口创建对象："
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from mindspore.nn import Accuracy\n",
"from mindspore import Model, Tensor\n",
"\n",
"net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction=\"mean\")\n",
"net_opt = nn.Momentum(net.trainable_params(), learning_rate=0.01, momentum=0.9)\n",
"model = Model(net, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"下面调用`model.eval`接口执行验证过程："
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"============== {'Accuracy': 0.9846754807692307} ==============\n"
]
}
],
"source": [
"mnist_path = \"./datasets/MNIST_Data/test\"\n",
"ds_eval = create_dataset(mnist_path)\n",
"acc = model.eval(ds_eval, dataset_sink_mode=False)\n",
"print(\"============== {} ==============\".format(acc))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"推理完整样例代码参见<https://gitee.com/mindspore/models/blob/r1.2/official/cv/lenet/eval.py>。\n",
"\n",
"调用`model.predict`接口执行验证过程，这里选取数据集中的一张图片进行预测："
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Predicted: \"6\", Actual: \"6\"\n"
]
}
],
"source": [
"ds_eval = ds_eval.create_dict_iterator()\n",
"data = next(ds_eval)\n",
"\n",
"# images为测试图片，labels为测试图片的实际分类\n",
"images = data[\"image\"].asnumpy()\n",
"labels = data[\"label\"].asnumpy()\n",
"\n",
"# 使用函数model.predict预测image对应分类\n",
"output = model.predict(Tensor(data['image']))\n",
"predicted = np.argmax(output.asnumpy(), axis=1)\n",
"\n",
"# 输出预测分类与实际分类\n",
"print(f'Predicted: \"{predicted[0]}\", Actual: \"{labels[0]}\"')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 加载MindSpore Hub模型执行推理\n",
"\n",
"除了使用`load_checkpoint`和`load_param_into_net`从本地加载模型之外，也可以通过[安装MindSpore Hub](https://gitee.com/mindspore/hub/blob/r1.2/README_CN.md)，通过`mindspore_hub.load`从云端加载模型参数执行推理。\n",
"\n",
"之前使用的加载本地模型的方法为：\n",
"\n",
"```python\n",
"from mindspore import load_checkpoint, load_param_into_net\n",
"ckpt_file_name = \"./checkpoint/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\"\n",
"param_dict = load_checkpoint(ckpt_file_name)\n",
"load_param_into_net(net, param_dict)\n",
"```\n",
"\n",
"可替换为`mindspore_hub.load`方法：\n",
"\n",
"```python\n",
"import mindspore_hub\n",
"model_uid = \"mindspore/ascend/1.2/lenet_v1.2_mnist\"\n",
"network = mindspore_hub.load(model_uid)\n",
"```\n",
"\n",
"## 在Ascend中使用C++接口推理MindIR格式文件\n",
"\n",
"本小节将介绍如何使用C++接口推理MINDIR格式的模型。完整代码可参考[ascend910_resnet50_preprocess_sample](https://gitee.com/mindspore/docs/tree/r1.2/tutorials/tutorial_code/ascend910_resnet50_preprocess_sample)。\n",
"\n",
"> 本小节内容及代码仅适用于Ascend环境。\n",
"\n",
"### 推理代码介绍\n",
"\n",
"完成的推理代码为`main.cc`文件，现在对其中的功能实现进行说明。\n",
"\n",
"引用`mindspore`和`mindspore::dataset`的名字空间。\n",
"\n",
"```cpp\n",
"namespace ms = mindspore;\n",
"namespace ds = mindspore::dataset;\n",
"```\n",
"\n",
"环境初始化，指定硬件为Ascend 910，DeviceID为0：\n",
"\n",
"```c++\n",
"auto context = std::make_shared<ms::Context>();\n",
"auto ascend910_info = std::make_shared<ms::Ascend910DeviceInfo>();\n",
"ascend910_info->SetDeviceID(0);\n",
"context->MutableDeviceInfo().push_back(ascend910_info);\n",
"```\n",
"\n",
"加载模型文件:\n",
"\n",
"```cpp\n",
"// 加载 MindIR 模型\n",
"ms::Graph graph;\n",
"ms::Status ret = ms::Serialization::Load(resnet_file, ms::ModelType::kMindIR, &graph);\n",
"// 进行图编译\n",
"ms::Model resnet50;\n",
"ret = resnet50.Build(ms::GraphCell(graph), context);\n",
"```\n",
"\n",
"获取模型所需输入信息：\n",
"\n",
"```cpp\n",
"std::vector<ms::MSTensor> model_inputs = resnet50.GetInputs();\n",
"```\n",
"\n",
"加载图片文件:\n",
"\n",
"```cpp\n",
"ms::MSTensor ReadFile(const std::string &file);\n",
"auto image = ReadFile(image_file);\n",
"```\n",
"\n",
"图片预处理:\n",
"\n",
"```cpp\n",
"// 对图片进行解码，变为RGB格式，并重设尺寸\n",
"std::shared_ptr<ds::TensorTransform> decode(new ds::vision::Decode());\n",
"std::shared_ptr<ds::TensorTransform> resize(new ds::vision::Resize({256}));\n",
"// 输入归一化\n",
"std::shared_ptr<ds::TensorTransform> normalize(new ds::vision::Normalize(\n",
" {0.485 * 255, 0.456 * 255, 0.406 * 255}, {0.229 * 255, 0.224 * 255, 0.225 * 255}));\n",
"// 剪裁图片\n",
"std::shared_ptr<ds::TensorTransform> center_crop(new ds::vision::CenterCrop({224, 224}));\n",
"// shape (H, W, C) 变为 shape (C, H, W)\n",
"std::shared_ptr<ds::TensorTransform> hwc2chw(new ds::vision::HWC2CHW());\n",
"\n",
"// 定义preprocessor\n",
"ds::Execute preprocessor({decode, resize, normalize, center_crop, hwc2chw});\n",
"\n",
"// 调用函数，获取处理后的图像\n",
"ret = preprocessor(image, &image);\n",
"```\n",
"\n",
"执行推理:\n",
"\n",
"```cpp\n",
"// 创建输入输出向量\n",
"std::vector<ms::MSTensor> outputs;\n",
"std::vector<ms::MSTensor> inputs;\n",
"\n",
"inputs.emplace_back(model_inputs[0].Name(), model_inputs[0].DataType(), model_inputs[0].Shape(),\n",
" image.Data().get(), image.DataSize());\n",
"// 执行推理\n",
"ret = resnet50.Predict(inputs, &outputs);\n",
"```\n",
"\n",
"获取推理结果:\n",
"\n",
"```cpp\n",
"// 获取推理结果的最大可能性\n",
"std::cout << \"Image: \" << image_file << \" infer result: \" << GetMax(outputs[0]) << std::endl;\n",
"```\n",
"\n",
"### 构建脚本介绍\n",
"\n",
"构建脚本用于构建用户程序，完整代码位于`CMakeLists.txt` ，下面进行解释说明。\n",
"\n",
"为编译器添加头文件搜索路径：\n",
"\n",
"```cmake\n",
"option(MINDSPORE_PATH \"mindspore install path\" \"\")\n",
"include_directories(${MINDSPORE_PATH})\n",
"include_directories(${MINDSPORE_PATH}/include)\n",
"```\n",
"\n",
"在MindSpore中查找所需动态库：\n",
"\n",
"```cmake\n",
"find_library(MS_LIB libmindspore.so ${MINDSPORE_PATH}/lib)\n",
"file(GLOB_RECURSE MD_LIB ${MINDSPORE_PATH}/_c_dataengine*)\n",
"```\n",
"\n",
"使用指定的源文件生成目标可执行文件，并为目标文件链接MindSpore库：\n",
"\n",
"```cmake\n",
"add_executable(resnet50_sample main.cc)\n",
"target_link_libraries(resnet50_sample ${MS_LIB} ${MD_LIB})\n",
"```\n",
"\n",
"\n",
"### 编译并执行推理代码\n",
"\n",
"可选择将实验的脚本下载至Ascend910环境中编译并执行。\n",
"\n",
"进入工程目录`ascend910_resnet50_preprocess_sample`，执行`cmake`命令，其中`pip3`需要按照实际情况修改：：\n",
"\n",
"```bash\n",
"cmake . -DMINDSPORE_PATH=`pip3 show mindspore-ascend | grep Location | awk '{print $2\"/mindspore\"}' | xargs realpath`\n",
"```\n",
"\n",
"再执行`make`命令编译即可。\n",
"\n",
"```bash\n",
"make\n",
"```\n",
"\n",
"编译成功后，会获得`resnet50_sample`可执行文件。在工程目录`ascend910_resnet50_preprocess_sample`下创建`model`目录放置MindIR文件[resnet50_imagenet.mindir](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/sample_resources/ascend310_resnet50_preprocess_sample/resnet50_imagenet.mindir)。此外，创建`test_data`目录用于存放待分类的图片，图片可来自ImageNet等各类开源数据集，输入执行命令即可获取推理结果：\n",
"\n",
"```bash\n",
"./resnet50_sample\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.5"
}
},
"nbformat": 4,
"nbformat_minor": 1
}