{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 静态图网络编译性能优化\n",
    "\n",
    "[![在线运行](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9vYnMuZHVhbHN0YWNrLmNuLW5vcnRoLTQubXlodWF3ZWljbG91ZC5jb20vbWluZHNwb3JlLXdlYnNpdGUvbm90ZWJvb2svcjIuMC4wLWFscGhhL3R1dG9yaWFscy9leHBlcnRzL3poX2NuL25ldHdvcmsvbWluZHNwb3JlX29wX292ZXJsb2FkLmlweW5i=&imageid=77ef960a-bd26-4de4-9695-5b85a786fb90) [![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r2.0.0-alpha/tutorials/experts/zh_cn/network/mindspore_op_overload.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r2.0.0-alpha/tutorials/experts/zh_cn/network/mindspore_op_overload.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.0.0-alpha/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r2.0.0-alpha/tutorials/experts/source_zh_cn/network/op_overload.ipynb)\n",
    "\n",
    "## 概述\n",
    "\n",
    "在深度学习网络进行训练或者推理时，网络端到端的耗时基本由编译耗时与运行耗时两部分组成，尤其在推理场景，编译耗时往往远大于运行耗时，因此优化编译性能对于提升网络在实际应用时的部署效果有着极为重要的意义。MindSpore静态图模式下，部分场景可以通过改变网络写法，使用等价语义替换，或者设置编译选项改变编译机制来优化网络编译性能。\n",
    "\n",
    "## 使用HyperMap优化编译性能\n",
    "\n",
    "`HyperMap`是一个特殊的类，类对象构造时需要传入映射函数f，调用对象时需要传入f的n个参数序列，更多使用方法见：[HyperMap](https://www.mindspore.cn/docs/zh-CN/r2.0.0-alpha/api_python/ops/mindspore.ops.HyperMap.html)。映射函数f必须是`MultitypeFuncGraph`类型, 可参考[MultitypeFuncGraph](https://www.mindspore.cn/docs/zh-CN/r2.0.0-alpha/api_python/ops/mindspore.ops.MultitypeFuncGraph.html)。在使用for循环批量处理列表元素时，可以通过`HyperMap`等价语义替换来优化网络编译性能。\n",
    "\n",
    "一个使用HyperMap替换for循环来优化编译性能的代码样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-09-02T01:56:28.534965Z",
     "start_time": "2022-09-02T01:56:24.071992Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hyper map cost time: 0.1894233226776123\n",
      "for loop cost time: 1.2634551525115967\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from mindspore.ops import MultitypeFuncGraph, HyperMap\n",
    "from mindspore import ops\n",
    "import mindspore as ms\n",
    "\n",
    "add = MultitypeFuncGraph('add')\n",
    "@add.register(\"Number\", \"Number\")\n",
    "def add_scalar(x, y):\n",
    "    return ops.scalar_add(x, y)\n",
    "\n",
    "add_map = HyperMap(add)\n",
    "list1 = [i for i in range(200)]\n",
    "list2 = [i for i in range(200)]\n",
    "@ms.jit\n",
    "def hyper_map_net():\n",
    "    output = add_map(list1, list2)\n",
    "    return output\n",
    "\n",
    "start_time = time.time()\n",
    "output = hyper_map_net()\n",
    "end_time = time.time()\n",
    "print(\"hyper map cost time:\", end_time - start_time)\n",
    "\n",
    "@ms.jit\n",
    "def for_loop_net():\n",
    "    out = []\n",
    "    for i in range(200):\n",
    "        out.append(ops.scalar_add(i, i))\n",
    "    return out\n",
    "\n",
    "start_time = time.time()\n",
    "for_loop_net()\n",
    "end_time = time.time()\n",
    "print(\"for loop cost time:\", end_time - start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用Select算子优化编译性能\n",
    "\n",
    "编写网络时，会经常使用到if语句，如果if语句的条件是变量条件，每个if语句都会产生额外的子图，if语句的使用可参考：[if语句](https://mindspore.cn/tutorials/experts/zh-CN/r2.0.0-alpha/network/control_flow.html#if语句)。在静态图模式下，子图数量越多，编译耗时越久，因此部分场景可以通过`Select`算子等价替换if语句来优化编译性能。\n",
    "\n",
    "需要注意的是，使用`Select`算子替换if语句会影响网络的运行性能。一方面，`Select`算子会同时执行true分支和false分支，而if语句只执行其一个分支，因此使用if运行耗时相比使用`Select`算子耗时减少；另一方面，`Select`算子性能优于if语句产生的控制流算子，使用if运行耗时相比使用`Select`算子运行耗时增加。综合上述两种因素，最终运行性能变化情况需要结合实际情况判断。一般来讲，当分支中算子数量较少，建议使用`Select`算子；当分支中算子数量较多，建议使用if语句。\n",
    "\n",
    "一个使用`Select`算子替代if语句来优化编译性能的代码样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-09-02T01:56:42.681341Z",
     "start_time": "2022-09-02T01:56:41.028750Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "if net cost time: 1.1603329181671143\n",
      "select net cost time: 0.483151912689209\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from mindspore import ops\n",
    "import mindspore as ms\n",
    "\n",
    "@ms.jit\n",
    "def if_net(x, y):\n",
    "    out = 0\n",
    "    for _ in range(100):\n",
    "        if x < y:\n",
    "            x = x - y\n",
    "        else:\n",
    "            x = x + y\n",
    "        out = out + x\n",
    "    return out\n",
    "\n",
    "start_time = time.time()\n",
    "out = if_net(ms.Tensor([0]), ms.Tensor([1]))\n",
    "end_time = time.time()\n",
    "print(\"if net cost time:\", end_time - start_time)\n",
    "\n",
    "@ms.jit\n",
    "def select_net(x, y):\n",
    "    out = x\n",
    "    for _ in range(100):\n",
    "        cond = x < y\n",
    "        x = ops.Select()(cond, x - y, x + y)\n",
    "        out = out + x\n",
    "    return out\n",
    "\n",
    "start_time = time.time()\n",
    "out = select_net(ms.Tensor([0]), ms.Tensor([1]))\n",
    "end_time = time.time()\n",
    "print(\"select net cost time:\", end_time - start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用编译缓存优化编译性能\n",
    "\n",
    "在进行训练或者推理时，如果某个网络结构未作任何变更，那么可以通过使用编译缓存来缩短编译时间。编译缓存的本质是存储了网络模型的编译中间过程文件，当网络模型不变时，生产的编译中间过程文件也是一样的，因此可以复用上一次编程产生的中间过程文件，详细使用方法可参考[设置context](https://www.mindspore.cn/docs/zh-CN/r2.0.0-alpha/api_python/mindspore/mindspore.set_context.html?highlight=enable_compile_cache)中的`enable_compile_cache`相关内容。\n",
    "\n",
    "一个通过使能编译缓存来优化编译性能的代码样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-09-02T01:56:43.505471Z",
     "start_time": "2022-09-02T01:56:42.892637Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disable comile_cache cost time: 0.5485098361968994\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from mindspore import set_context\n",
    "from mindspore import dtype\n",
    "import mindspore as ms\n",
    "\n",
    "@ms.jit\n",
    "def func(input_x, input_y):\n",
    "    output = input_x\n",
    "    for _ in range(200):\n",
    "        output = input_x + input_x * input_y + output\n",
    "    return output\n",
    "\n",
    "set_context(enable_compile_cache=False)\n",
    "x = ms.Tensor([1], dtype.float32)\n",
    "y = ms.Tensor([2], dtype.float32)\n",
    "start_time = time.time()\n",
    "out = func(x, y)\n",
    "end_time = time.time()\n",
    "print(\"Disable comile_cache cost time:\", end_time - start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述测试样例是关闭编译缓存状态，执行上述测试样例2次，第1次耗时和第2次耗时如下：（实际耗时与硬件环境有关，以下数据仅供参考）\n",
    "\n",
    "```text\n",
    "Disable comile_cache cost time: 0.5485098361968994\n",
    "\n",
    "Disable comile_cache cost time: 0.4614279270172119\n",
    "```\n",
    "\n",
    "可以看到，关闭编译缓存时，第1次执行样例与第2次执行样例耗时基本接近。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-09-02T01:56:45.139722Z",
     "start_time": "2022-09-02T01:56:44.502524Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enable comile_cache cost time: 0.6357541084289551\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from mindspore import set_context\n",
    "from mindspore import dtype\n",
    "import mindspore as ms\n",
    "\n",
    "@ms.jit\n",
    "def func(input_x, input_y):\n",
    "    output = input_x\n",
    "    for _ in range(200):\n",
    "        output = input_x + input_x * input_y + output\n",
    "    return output\n",
    "\n",
    "set_context(enable_compile_cache=True, compile_cache_path=\"my_compile_cache\")\n",
    "x = ms.Tensor([1], dtype.float32)\n",
    "y = ms.Tensor([2], dtype.float32)\n",
    "start_time = time.time()\n",
    "out = func(x, y)\n",
    "end_time = time.time()\n",
    "print(\"Enable comile_cache cost time:\", end_time - start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述测试样例是开启编译缓存状态，执行上述测试样例2次，第1次耗时和第2次耗时如下：（实际耗时与硬件环境有关，以下数据仅供参考）\n",
    "\n",
    "```text\n",
    "Enable comile_cache cost time: 0.6357541084289551\n",
    "\n",
    "Enable comile_cache cost time: 0.09379792213439941\n",
    "```\n",
    "\n",
    "可以看到，开启编译缓存时，第2次执行样例耗时只有第一次执行耗时的1/7左右。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用vmap优化编译性能\n",
    "\n",
    "MindSpore当前已知支持vmap特性，在处理无依赖关系的批量数据且相关的算子支持vmap功能时，可以使用vmap替代for循环处理批量数据来优化编译性能。vmap的详细介绍可参考[vmap](https://www.mindspore.cn/docs/zh-CN/r2.0.0-alpha/api_python/mindspore/mindspore.vmap.html#mindspore.vmap)。需要注意的是，vmap不仅能优化编译性能，也能优化运行性能。\n",
    "\n",
    "一个使用vmap替换for循环处理批量数据来优化编译性能的代码样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-09-02T01:56:48.130285Z",
     "start_time": "2022-09-02T01:56:46.129533Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vmap cost time: 0.05766916275024414\n",
      "for loop cost time: 1.9284062385559082\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import time\n",
    "from mindspore import ops, vmap\n",
    "import mindspore as ms\n",
    "\n",
    "def hswish_func(x):\n",
    "    return ops.HSwish()(x)\n",
    "\n",
    "@ms.jit\n",
    "def manually_batched(xs):\n",
    "    output = []\n",
    "    for i in range(xs.shape[0]):\n",
    "        output.append(hswish_func(xs[i]))\n",
    "    return ops.stack(output)\n",
    "\n",
    "shape = (100, 2)\n",
    "prop = 100\n",
    "x_np = (np.random.randn(*shape) * prop).astype(np.float32)\n",
    "x = ms.Tensor(x_np)\n",
    "x = ops.sub(x, 0)\n",
    "\n",
    "start_time = time.time()\n",
    "output_vmap = vmap(hswish_func, in_axes=(0,))(x)\n",
    "end_time = time.time()\n",
    "print(\"vmap cost time:\", end_time - start_time)\n",
    "\n",
    "start_time = time.time()\n",
    "output_manually = manually_batched(x)\n",
    "end_time = time.time()\n",
    "print(\"for loop cost time:\", end_time - start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述样例中，相当于需要批量处理100组Tensor数据，可以看到使用vmap处理的性能超过使用for循环处理性能的30倍。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}