{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 进阶案例:线性拟合\n", "\n", "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r1.8/tutorials/zh_cn/advanced/mindspore_linear_fitting.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_download_code.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/r1.8/tutorials/zh_cn/advanced/mindspore_linear_fitting.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.8/tutorials/source_zh_cn/advanced/linear_fitting.ipynb)\n", "\n", "MindSpore向用户提供了高阶、中阶和低阶3个不同层次的API,详细内容参见[基本介绍-层次结构内容章节](https://www.mindspore.cn/tutorials/zh-CN/r1.8/beginner/introduction.html#层次结构)。\n", "\n", "为方便控制网络的执行流程,MindSpore提供了高阶的训练和推理接口`mindspore.Model`,通过指定要训练的神经网络模型和常见的训练设置,调用`train`和`eval`方法对网络进行训练和推理。同时,用户如果想要对特定模块进行个性化设置,也可以调用对应的中低阶接口自行定义网络的训练流程。\n", "\n", "本章将使用MindSpore提供的中低阶API拟合线性函数:\n", "\n", "$$f(x) = 2x + 3 \\tag {1}$$\n", "\n", "本章将会介绍配置信息和使用MindSpore提供的中低阶API,实现自定义损失函数、优化器、训练流程、Metric、自定义验证流程模块。\n", "\n", "## 配置信息\n", "\n", "初始化网络之前,需要配置`context`参数,用于控制程序执行的策略,如配置静态图或动态图模式,配置网络运行的硬件环境等。本节主要介绍执行模式管理和硬件管理。\n", "\n", "### 执行模式\n", "\n", "MindSpore支持Graph和PyNative两种运行模式。Graph模式是MindSpore的默认模式,而PyNative模式用于调试等用途。\n", "\n", "- Graph模式(静态图模式):将神经网络模型编译成一整张图,然后下发到硬件执行。该模式利用图优化等技术提高运行性能,同时有助于规模部署和跨平台运行。\n", "\n", "- PyNative模式(动态图模式):将神经网络中的各个算子逐一下发到硬件中执行,该模式方便用户编写代码和调试神经网络模型。\n", "\n", "MindSpore提供了静态图和动态图统一的编码方式,大大增加了静态图和动态图的可兼容性,用户无需开发多套代码,仅变更一行代码便可切换静态图/动态图模式。模式切换时,请留意目标模式的[约束](https://www.mindspore.cn/docs/zh-CN/r1.8/note/static_graph_syntax_support.html)。\n", "\n", "设置运行模式为动态图模式:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import mindspore as ms\n", "\n", "ms.set_context(mode=ms.PYNATIVE_MODE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "同样,MindSpore处于动态图模式时,可以通过`set_context(mode=GRAPH_MODE)`切换为静态图模式:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import mindspore as ms\n", "\n", "ms.set_context(mode=ms.GRAPH_MODE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 硬件管理\n", "\n", "硬件管理部分主要包括`device_target`和`device_id`两个参数。\n", "\n", "- `device_target`: 待运行的目标设备,支持`Ascend`、`GPU`和`CPU`,可以根据实际环境情况设置,或者使用系统默认配置。\n", "\n", "- `device_id`: 表示目标设备ID,其值在[0, `device_num_per_host` - 1]范围内,`device_num_per_host`表示服务器的总设备数量,`device_num_per_host`的值不能超过4096,`device_id`默认为0。\n", "\n", "> 在非分布式模式执行的情况下,为了避免设备的使用冲突,可以通过设置`device_id`决定程序执行的设备ID。\n", "\n", "代码样例如下:\n", "\n", "```Python\n", "import mindspore as ms\n", "\n", "ms.set_context(device_target=\"Ascend\", device_id=6)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 处理数据集\n", "\n", "### 生成数据集\n", "\n", "定义数据集生成函数 `get_data` ,生成训练数据集和测试数据集。\n", "\n", "由于拟合的是线性数据,假定要拟合的目标函数为:$f(x)=2x+3$,那么我们需要的训练数据集应随机分布于函数周边,这里采用了$f(x)=2x+3+noise$的方式生成,其中`noise`为遵循标准正态分布规律的随机数值。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "def get_data(num, w=2.0, b=3.0):\n", " for _ in range(num):\n", " x = np.random.uniform(-10.0, 10.0)\n", " noise = np.random.normal(0, 1)\n", " y = x * w + b + noise\n", " yield np.array([x]).astype(np.float32), np.array([y]).astype(np.float32)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "使用get_data生成50组验证数据,并可视化。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEICAYAAAC6fYRZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAq5UlEQVR4nO3dd3wU1frH8c+TUKQXKdKLclWwoAQVK4jXgiJ2wS4lYvtde+NeQUSviiiiFBG9igqKhSLYqCIgJSBIUymigJRQpCole35/7GRZ4m5IyE422Xzfr9e+sjNzdubJZPPs2TNnzjHnHCIikpiS4h2AiIj4R0leRCSBKcmLiCQwJXkRkQSmJC8iksCU5EVEEpiSvEgWZjbFzDrnsGxLM1vjd0wih0tJXgotM1tlZn+a2c6wx2vxjisaM7vNzKbFOw4pWorFOwCRPGrrnJsQ7yBECirV5CXhmFlJM/vDzE4IW1fVq/VXM7NKZjbWzNLNbKv3vHYO913KzN72XrcEaJ5l+2NmtsLMdpjZEjO70lt/PDAIaOF94/jDW3+pmX1vZtvNbLWZ9YjRaRABlOQlATnn9gCfAh3CVl8HfOOc20jwff8/oB5QF/gTyGkzT3fgaO9xEXBrlu0rgHOACsBTwHtmVsM5txToCnznnCvrnKvold8F3AJUBC4F7jSzK3L6u4ocipK8FHajvFp75qOLt34Y0D6s3A3eOpxzm51znzjndjvndgDPAOfl8HjXAc8457Y451YD/cI3Ouc+cs797pwLOOc+BJYBp0XbmXNuinNuoVf+B2B4LmIROSS1yUthd0WUNvnJQGkzOx3YADQFRgKYWWngZeBioJJXvpyZJTvnMg5xvJrA6rDlX8M3mtktwANAfW9VWaBKtJ158T0HnACUAEoCHx0iBpEcU01eEpKXrEcQbLLpAIz1au0ADwLHAqc758oD53rrLQe7XgfUCVuum/nEzOoBbwD3AEd6TTKLwvYbacjXYcAYoI5zrgLBdvucxCGSI0ryksiGAdcDN3rPM5Uj2A7/h5lVJtjOnlMjgMe9i7e1gXvDtpUhmMjTAczsdoI19EwbgNpmViJLLFucc3+Z2WkEm5VEYkZJXgq7z7L0kx+ZucE5N4vghc2awBdhr+kLlAI2ATOBL3NxvKcINtH8AnwNvBt2vCVAH+A7ggn9RGB62GsnAYuB9Wa2yVt3F9DTzHYATxL8EBGJGdOkISIiiUs1eRGRBJbnJG9mdcxssnfjx2Iz+5e3voeZrTWz+d6jTd7DFRGR3Mhzc42Z1QBqOOfmmVk5YC5wBcH+xDudcy/mOUoRETksee4n75xbR7BbGc65HWa2FKiV1/2KiEjexfTCq5nVB6YS7Db2AHAbsB1IAx50zm2N8JpUIBWgTJkyzY477riYxSMiUhTMnTt3k3OuaqRtMUvyZlYW+IbgLd+fmll1gl3UHPA0wSadjtntIyUlxaWlpcUkHhGRosLM5jrnUiJti0nvGjMrDnwCvO+c+xTAObfBOZfhnAsQvAsw6vgdIiLij1j0rjHgTWCpc+6lsPU1wopdSfD2bhERyUexGKDsLOBmYKGZzffWPQF0MLOmBJtrVgF3xOBYIiKSC7HoXTONyAMqfZ7XfYuISN7ojlcRkQSmJC8iksCU5EVEEpiSvIhIHAVcgFtH3crHSz72Zf9K8iIicTJh5QSSeyYzdMFQOo7O9l7Rw6Y5XkVEshMIQHo6VKsGFpuZGfdm7OWYfsewentwuuBmNZoxq/OsmOw7K9XkRUSiCQSgVSuoXRtatgwu59GIxSMo2atkKMHP7DSTtNQ0kpOS87zvSFSTFxGJJj0dZsyA/fuDP9PToXr1w9rVzr07qfhcRTJcBgCXH3s5o64fhcXo20E0qsmLiERTrRqceSYUKxb8Wa3aYe1mwJwBlPtvuVCCX3LXEka3H+17ggfV5EVEojODyZMPu01+8+7NVOldJbScemoqr7d9PdZRZktJXkQkO0lJh9VE02NKD5765qnQ8m/3/UadCnViGVmOKMmLiMTQ6m2rqdu3bmi5+3nd6dGyR9ziUZIXEYmRrmO78vrcA80xmx7exJGlj4xjREryIiJ5tiR9CU0GNAkt92/Tn7ua3xXHiA5QkhcROUzOOS7/4HLG/jwWgGJJxfjj9p8oU6tBnCM7QF0oRURyIxCADRuYufo7knomhRL8h1cNZ9+EMynT4NiY3TgVC3muyZtZHWAoUJ3gLFCDnXOvmFll4EOgPsGZoa5zzm3N6/FEROImECCjVUtOO/5b5nkTnNatUJdl9y6jxKatMOPmmNw4FUuxqMnvBx50zjUGzgDuNrPGwGPAROdcI2CitywiUmh9OfdDip1/IMGPv2wEv973KyWSS8TsxqlYi8X0f+uAdd7zHWa2FKgFtANaesXeAaYAj+b1eCIi+W3P/j3U61uPDbs2ANBiNUxbdg5JT15zoFAeb5zyS0wvvJpZfeAUYBZQ3fsAAFhPsDkn0mtSgVSAunXrRioiIhI3wxYO48ZPbwwtz+k0i5Ti9SIn8sO8ccpPMUvyZlYW+AS4zzm3PXxMBuecMzMX6XXOucHAYICUlJSIZURE8tuOPTso/1z50PLVx1/NR9d+lC/jzcRSTHrXmFlxggn+fefcp97qDWZWw9teA9gYi2OJiPjtlZmvHJTgf7rnJz6+7uNCl+AhNr1rDHgTWOqceyls0xjgVuA57+fovB5LRMRP6bvSqfbigQum9zS/h1fbvBrHiPIuFs01ZwE3AwvNbL637gmCyX2EmXUCfgWui8GxRER80W1iN56d9mxoec39a6hVvlYcI4qNWPSumQZE+w7TOq/7FxHx06o/VtHglQN3qPZq1Ytu53aLY0SxpWENRKTI6jjqdv634O3Q8pZHtlCpVKX4BeQDJXkRKXIWbljISYNOCi0P/rERXd7/MdgFMsEoyYtIkeGc45L3L+GrFV8BUGofbHoBSrtfoG/BGIYg1hLvY0tEJILpv00nqWdSKMF/cu3H7P72XEq7gjUMQaypJi8iCS0jkMEpr5/Cwo0LATi60tEsvXspxZOLw+QrC9wwBLGmJC8iCWvsz2NpO7xtaHnSLZNo1aDVgQIFcBiCWFOSF5GE89f+v6j1Ui22/LkFgPPqncekWyeRZEnBcd4TvPYeTm3yIpJQhi4YSqlnSoUS/LzUeUy5bcqBBN+qFdSuXaAm9vCTavIikhC2/bWNis9XDC13OKEDw64ednCh9PTghB4FbGIPP6kmLyKFXp8ZfQ5K8MvuXfb3BA+HntjDm9oPlzgD4qomLyKF1oadGziqz1Gh5fvPuJ+XLnop+guym9gjsylnxozgB8DkyQlxc5SSvIgUSo+Mf4TeM3qHln9/4HdqlKtx6BdG61GToE05SvIiUqis3LqSo/sdHVp+rvVzPHp2DGYWzWzKyazJJ8jNUUryIlJo3PTpTby/8P3Q8tZHt1LxiIqx2XkBnaM1r5TkRaTAW7B+AU1fbxpafvPyN+l4SsfYHygBb45SkhcR/+TxxiPnHK2HtmbyqskAlC9ZnvUPrqdU8VKxjjRhxWqO17fMbKOZLQpb18PM1prZfO/RJhbHEpFCIo83Hk39dSpJPZNCCX7U9aPY9tg2JfhcilVN/m3gNWBolvUvO+dejNExRKQwOVRvlSi1/P2B/TQZ0ISfN/8MwPFVjueHO3+gWJIaHg5HTGryzrmpwJZY7EtEEkR2Nx5FqeWP+nEUxZ8uHkrwU2+bypK7lyjB54HfZ+4eM7sFSAMedM5tzVrAzFKBVIC6dev6HI6I5JvseqtkqeX/ue43qg09kZ17dwLQukFrxt88HkuQHi7x5OftXAOBo4GmwDqgT6RCzrnBzrkU51xK1apVfQxHRPJdZm+VrMk6rJb/1tUNKT2kQSjBL+i6gAm3TFCCjxHfavLOuQ2Zz83sDWCsX8cSkULGjD++GEml3kcCwaaZm066iXevfDe+cSUg32ryZhZ+f/GVwKJoZUWkaHlu2nNegg9accMs3r0ia78NiYWY1OTNbDjQEqhiZmuA7kBLM2sKOGAVcEcsjiUihde6Heuo+VLN0PIjLR7m+WdmQa+zEmpQsIIkJkneOdchwuo3Y7FvEUkM9395P31n9Q0tr39wPdV3ATNqJ9ygYAWJ+iWJiK+WbV7GP177R2i5z4V9eKDFA8GFMi4hBwUrSJTkRcQXzjk6fNKBDxd/GFq37bFtlC9Z/kChBB0UrCBRkheRmJu3bh7NBjcLLQ+9Yig3n3xz5MIJOChYQaIkLyIxE3ABznv7PKb9Ng2AKqWrsPr+1RxR7Ig4R1Z0KcmLSExM/mUy5w89P7Q8tsNYLv3HpXGMSEBJXkTyaF/GPo597Vh++eMXAE6ufjJzU+eSnJQc58gElORFJA8+WfIJ13x0TWh5esfpnFnnzDhGJFkpyYtIru3et5vKz1dmT8YeAC455hLG3TBO480UQEryIpIrr6e9TtdxXUPLC+9cyAnVTohjRJIdJXkRyZEtf27hyBcOjDfTsWlH3mynG9sLOiV5ETmkXlN78Z/J/wktr/rXKupVrBfHiCSnlORFJLJAgLW//EDt904Jrep2Tjd6nd8rjkFJbinJi8jfBQLcc0cd+tf+PbRq40MbqVpGE/sUNhrTUyTRBQKwYQM4l/06z0+bfsKeTg4l+Fe+SsJ1Xa8EX0gpyYskskgTZkeZRNs5x1UfXsVx/Y8LvXz7C8n8X/GzNTpkIRarSUPeAi4DNjrnTvDWVQY+BOoTnDTkukgTeYuIj7JMmE16enB9+LoNG5izaQGnfXpJ6GXDrhpGhybXQ1eNDlnYxaom/zZwcZZ1jwETnXONgInesojkp7AJs0PjtYetC7Q4g9N7HxtK8DXL1WTPv/fQ4cQO0SfhlkIlJkneOTcV2JJldTvgHe/5O8AVsTiWiORC5njta9bAlCnBZW/d+O/eJ7n1NGZX2AHAl8OSWHvjPEokl4hvzBJTfvauqe6cW+c9Xw9EHDDazFKBVIC6dev6GI5IEZVlvPa9GXs5ut/RrNm+BoCU7WWZ+eqfJLc4S23vCShfLrw65xzBCb0jbRvsnEtxzqVUraqr9yJ++nDRh5TsVTKU4Gd2msmc3ttIXr32QE1fEoqfNfkNZlbDObfOzGoAG308lohkY+fenZT/b3mcV9e6/NjLGXX9qAMDimlmpoTlZ01+DHCr9/xWYLSPxxKRKPrP7k+5/5YLJfgldy1hdPvRGjGyiIhVF8rhQEugipmtAboDzwEjzKwT8CtwXSyOJSI5s2n3Jqr2PtAE2rVZVwZeNjCOEUk8xCTJO+c6RNnUOhb7F5Hc6T65Oz2n9gwt/3bfb9SpUCeOEUm8aOwakQTy27bfqNf3wOiQPc7rQfeW3eMYkcSbkrxIgkj9LJU35r0RWt708CaOLH1kNq+QokBJXqSQW5K+hCYDmoSW+7fpz13N74pjRFKQKMmLFFLOOdoOb8u4ZeMAKJ5UnK2PbqVMiTJxjkwKEiV5kUJo5pqZtHizRWh5xDUjuLbJtXGMSAoqJXmRQiQjkEHzN5rz/frvAahXoR4/3/uzxpuRqJTkRQqJL5Z9QZthbULLE26eQOuG6qUs2VOSF4m3QCA4znuUcdv37N9D3b512bgrODJIi9otmNZxGkmmOX/k0PQuEYmnKLM0ZXr/h/c54pkjQgl+Tpc5zOg0Qwleckw1eZF4ijRzU/Xq7Nizg/LPlQ8Vu6bxNYy4ZoTGm5FcU3VAJJ6qVIHmzQ+auemVma8clOB/uvtHPrr2IyV4OSyqyYvESyAA558Ps2dD8+Zs/OwDqvc8UO+6d3VN+r2zESalwsSJsHmz5luVXFNNXsRPgQBs2AAuwpw5mU01GRk8UW4W1V+uGdq05qbvgwl+/36YPh3OPTdqu71IdpTkRfxyiIuqVKvGqvNPxXrAf88Kfgj0atUL191Rq/6JkJICycnB5pw5cw5utxfJISV5kcORXQ09U6SLqmFuH9ORBmfODi1veWQL3c7tdqAZJy0NTjsNvv022F4f1m4vklNK8iK5dagaeqZq1SIm54UbFmJPGW/PfxuAwZcNxnV3VCpVKfi68A+HOXOCbfGTJ8OaNZqHVXLN9wuvZrYK2AFkAPudcyl+H1PEV1G6Pf6NWTA5ezc6OeDi9y7i6xVfA1CmeBk2PryR0sVLH/y6zA+HGTMOfDiYaR5WOSz5VZNv5ZxrqgQvCSFKDR34ezNOUhJUr8701TNI6pkUSvCfXPcJO5/Y+fcED8HXZn2IHCZ1oRTJrSw19FDzSWYzTmYNfPJk9hOg6aCmLE5fDECjyo1YfNdiiicXj77/9HT47jvIyAj+jPZNQSQH8qMm74CvzWyumaVm3WhmqWaWZmZp6eo1IIWFV0M/qH08SzPO2LT3Kf508VCCn3zrZH6+9+foCT7zW0DVqrrQKjFjzuevgmZWyzm31syqAeOBe51zUyOVTUlJcWlpab7GI+Ib56BlS/6aNZ0aDxl/FN8PwHn1zmPSrZOyH28m67cA3fwkuWBmc6M1h/tek3fOrfV+bgRGAqf5fUyRuDDjnb63UerxjFCCn5c6jym3TTn0gGJZL+Zu3vz3bwoih8HXNnkzKwMkOed2eM8vBHr6eUyReNj21zYqPl8xtNzhhA4Mu3rYwYWyG1I4Uo8akRjw+8JrdWCkN7BSMWCYc+5Ln48pkq9enPEiD49/OLS87N5lHFP5mIMLRbgoS1JY7T7axVyRPPI1yTvnVgIn+3kMkXhZv3M9NfrUCC3ff8b9vHTRS5EL56RvfebFXJEYUhdKkcPw0NcP0ee7PqHl3x/4nRrlakR/gZpjJE6U5EVyYcWWFRzz6oGmmOcveJ5Hznrk0C9Uc4zEiZK8SA7dNuo23lnwTmh566NbqXhExZzvQM0xEgcaoEzkENbvXM81I64JJfj/tfsfrrvLXYIXiRPV5EWicM7xzoJ3eOCrB9i9bzfPnv8s951xH6WKl4p3aCI5piQvEsGqP1Zxx9g7+HrF15xd92yGtB3CsVWOjXdYIrmmJC8SJuAC9J/dn8cnPo6Z8dolr3Fn8zsPfceqSAGlJC/iWZq+lM6fdWbG6hlcfMzFDLp0EPUq1ot3WCJ5oiQvRd6+jH30ntGbp755irIlyjL0iqHcdNJNmLo5SgJQkpcibd66eXQa04n56+dzbeNrefWSV6leVt0cJXEoyUuR9Oe+P+n5TU96z+hN1TJV+fS6T7ny+CvjHZZIzCnJS5Hz7apv6DyqEz9vW0HHph158cIXD0yiLZJg1GVACr6s86ZmXc6hHXt2cPe4uzj3nZbs/XUF478/kTfbvqEELwlNSV4KtswhemvXhpYtg6M4hi8HAjnazRfLvqDJgCYMTBvEfTONRf3hgrFLgmPJiCQwJXkp2LIO0fvjj38fsjcbm3dv5paRt9BmWBvKlijL9Num8vJ35SmzDyhbFqpUOVD4ML8hiBRkSvJSsGUO0Zs5qXXjxjma5No5x0eLP6LxgMYMXzSc/5z7H76/43taHNEIdu0KFtq1CzZtCj7P+o0hh98QRAo63y+8mtnFwCtAMjDEOfec38eUBBJpiN5DDNn7+7Y13D0qlVGrvqBZjWZ8fdPXnHyUN3dNtHHdczKph0gh5Pccr8lAf+CfwBpgjpmNcc4t8fO4kmCyDtEbZche5xxvzRvCgyPvZA8ZvPBrA+7vNoNixUocKBTtQ0KTekiC8rsmfxqw3JsGEDP7AGgHKMlLTK3cupLUz1KZ+MtEzv0dhoyGRttXw4tbczbNnib1kATld5t8LWB12PIab12ImaWaWZqZpaWrp4PkUkYgg74z+3LiwBOZvXY2A9sMYPKKc2i0Pfs2+4gyk78SvCSQuN8M5ZwbDAwGSElJUbcGybEl6UvoNKYTM9fMpE2jNgy6dBB1KtSByXeoRi7i8TvJrwXqhC3X9tZJURQIxCT57s3Yy/PTnqfXt70oV6Ic7135HjeceMOBAcU0zZ5IiN/NNXOARmbWwMxKAO2BMT4fUwqiGHVRTPs9jeZvNOfJKU9y1fFXseTuJdx40o0aMVIkCl+TvHNuP3AP8BWwFBjhnFvs5zGlgIrURTEXdu/bzSPjH+H0IaezafcmRrcfzfCrh1OtjNfmrhuZRCLyvU3eOfc58Lnfx5ECLg9dFL9Z9Q2dP+vM8i3L6XJqF1745wsHT6Kd+S0hc9+TJwebbEQk/hdepYg4jC6K2/ds59HxjzJo7iAaVmrIxFsmcn6D8/9eUDcyiUSl6o7EVnbNJrnoojju53E0GdCEwfMG88AZD7DwzoWREzz8fegD3cgkEqIkL7ETg4urm3Zv4qZPb+Ky4ZdRoWQFZnScQZ+L+lC6eOnoL8r8lrBmDUyZom6TImGU5CV2ol1czcFFUeccHyz6gOP7H8+IxSPofl535t0xj9Nrn36gUIy+JYgUJUryEjuRmk1yULtfu30t7T5oR4dPOtCgYgPmps6lR8selEgOG3NGo0SKHBZdeJXYiXRxdePGqBdFnXMMmTeEh8Y/xL6MffS5sA//Ov1fJCcl/33furgqclhUk5fYytpsEuWi6IotK2g9tDWpY1NpVqMZC+9cyAMtHoic4LPZj4hkTzV58VeW2n2GC/DKzFf496R/Uzy5OIMvG0znUzsf+o5VjRIpcliU5MV/Xu1+0cZFdBrTidlrZ9P2H20ZeOlAapWvdejXZ9mPiOSckrz4bm/GXp799lme/fZZKhxRgeEXDOL6Fl0w3ZUq4jv9l4k/vO6Os9fM4tTXT+Wpb57i2sbXsPSrRrRveQ/WqpV6yIjkA9XkJfb272f3eWfyn3Jp9D3DUaN8LT7r8BmXVWgO7Wurh4xIPlKSl9gKBJh8xcl0TlnCysrQNc14fsAUytc5JngTk+ZRFclXSvISM9v+2sbDY+7hjeZLOGYzTPkfnFfjdKh9dLCAesiI5DsleYmJMT+N4c5xd7J+53oe/rU2PYavo/Qpp8G0aQcnc/WQEclXvl14NbMeZrbWzOZ7jzZ+HUviZ+OujbT/uD3tPmjHkaWOZFbnWbww5FdKr1oL06drXHeROPO7Jv+yc+5Fn48hceCcY9jCYfzry3+xfc92erbsyaNnP3pgvBnV1kUKBDXXSK6t3raaO8fdybhl4zij9hkMaTuEJtWaxDssEYnA7+/S95jZD2b2lplV8vlY4rOACzAobRBNBjRh8qrJ9L2oL9Nun6YEL1KA5akmb2YTgKMibOoGDASeBpz3sw/QMcI+UoFUgLp16+YlHPHRss3L6PJZF7759RtaN2jN4LaDaVipYbzDEpFDMJcPs9ubWX1grHPuhOzKpaSkuLS0NN/jkZzbH9jPy9+9zJNTnqRkckleuuglbm96+6EHFBORfGNmc51zKZG2+dYmb2Y1nHPrvMUrgUV+HUv8sWD9AjqN6cTcdXNpd2w7Blw6gJrlasY7LBHJBT8vvL5gZk0JNtesAu7w8VgSQ3v276HX1F48N/05KpeqzIhrRnBN42tUexcphHxL8s65m/3at/jnu9Xf0WlMJ5ZuWsrNJ93Myxe9zJGlj4x3WCJymNSFUgDYtXcX3SZ1o9+sftQuX5vPb/icSxpdEu+wRCSPlOSFCSsn0OWzLqz6YxV3N7+b/7b+L+VKlot3WCISA0ryRdjWP7fy0NcP8db8t2hUuRFTb5vKOfXOiXdYIhJDSvJF1MilI7nr87tI35XOY2c9xpPnPUmp4qXiHZaIxJiSfBGzYecG7v3iXj5a8hEnVz+ZsR3G0qxms3iHJSI+UZIvIpxzvPvDu9z35X3s2reLZ85/hofPeJDiW/4ITuah7pEiCUnjwBYm3ryp5PIu5d+2/UabYW24ddStHF/1eBZ0XcATZz1G8QsuhNq1oWVLzbcqkqCU5AuLQABatcpVUg64AP1n96fJgCZ8++u39Lu4H9/e/i3HVTkuODvTjBkHz7cqIglHzTWFRaSknM2Y7T9t+onOn3Vm2m/T+GfDfzK47WDqV6x/oEC1appvVaQIUJIvLHKYlPdl7KPPd33oMaUHpYqX4n/t/setJ9/69yEJNN+qSJGgJF9YHCopBwJ8v3QSnaY9wvfrv+eq46+if5v+HFU20kjQHs23KpLwlOQLkyhJ+a+9u3n6juN4vu5qquwrzsc3juDqJtfGIUARKWh04bWQm/7bdJoOOIln66/m5gWw5NUAV1c5N95hiUgBoZp8IbVz706emPgEr81+jbp/luCrj+HCFcA5uogqIgcoyRdCXy3/itSxqazetpp7TujIsze/Tdk/gWLFYMQIXUQVkRA11xQiW3Zt4rbh13Px+xdTqlgpvr39W/pd9QZlm58VTPBnnqkLqSJykDwleTO71swWm1nAzFKybHvczJab2U9mdlHewpRPFn9E42dq8t6PI+i2qg7zU+dxVt2zDvS6WbMGpkxRLV5EDpLX5ppFwFXA6+Erzawx0B5oAtQEJpjZP5xzGXk8XpGzbsc67vniHj5d+imnbIEvR0PTTevguR1QvXSwkLpCikgUearJO+eWOud+irCpHfCBc26Pc+4XYDlwWl6OVdQ453h7/ts0HtCYcT+P47nW/2X2j+fQdFMx3aEqIjnm14XXWsDMsOU13rq/MbNUIBWgbt26PoVTuKz6YxWpn6UyfuV4zq57NkPaDuHYKsfCpEd0h6qI5Mohk7yZTQAi3TbZzTk3Oq8BOOcGA4MBUlJScje8YoLJCGTQf05/npj4BGZG/zb96ZrSlSTzvnCpWUZEcumQSd45d8Fh7HctUCdsuba3rugKBLKthS9NX0rnzzozY/UMLj7mYgZdOoh6FevFIVARSSR+daEcA7Q3s5Jm1gBoBMz26VgFXzbDBO/L2MczU5+h6etN+XHTjwy9Yiif3/C5EryIxESe2uTN7ErgVaAqMM7M5jvnLnLOLTazEcASYD9wd5HuWRNlmOB56+bRcXRHFmxYwHVNrqPfxf2oXlbNMSISO3lK8s65kcDIKNueAZ7Jy/4TRpZhgv+sVI6nJjzGizNepGqZqoy8fiRXHHdFvKMUkQSkYQ3yQ9gwwVP//JHOrzdl2ZZldDqlE73/2ZtKpSrFO0IRSVBK8vlk+76dPJ7WkwFpA6hfsT7jbx7PBQ0P55q2iEjOKcnngy+WfcEdY+9gzfY13Hf6ffQ6vxdlSpSJd1giUgQoyfto8+7N3P/V/bz7w7s0rtqY6R2n06JOi3iHJSJFiEah9IFzjhGLR3B8/+MZvmg4/zn3P8zrnEaLYvVh/XpwRfqeLxHJR6rJx9jvO37nrnF3Mfqn0TSr0YwJt0zgpKonBPvJf/ttMMGfc05wxMgkfcaKiL+U5GPEOcdb37/Fg18/yJ6MPbxwwQvc3+J+iiUVgw0bgt0nM2vwYX3lRUT8pCQfAyu3rqTLZ12Y9Mskzq13LkPaDqHRkY0OFMjsJ59Zk9cokiKST5Tk8yAjkMGrs1+l26RuJFsyAy8dSGqz1AMDimXK7Ce/YUPwefXqGkVSRPKFkvxhWrxxMZ3GdGLW2llc2uhSBl46kDoV6kR/QVIS1KiRfwGKiKAkn2t7M/by/LTneXrq05QvWZ73r3qfDid0wFQzF5ECSEk+F+asnUOnMZ1YuHEh7U9oT7+L+1G1TNV4hyUiEpWS/KEEAuz+/Ve6L+nPSzNf5qiyRzG6/WguP/byeEcmInJISvLZCQSYcuUpdKn/A8srQ5dTOtP7whepcESFeEcmIpIjuhsnim1/baPrJ7fT6tQfCAAT30ti8Om9lOBFpFBRTT6CcT+P446xd7Bu5zoe/K02PYeto/RpZ6lvu4gUOnmqyZvZtWa22MwCZpYStr6+mf1pZvO9x6C8h+q/9F3p3PjpjVw2/DIqlarEd52+48U3fqX0qrXBYQjUg0ZECpm81uQXAVcBr0fYtsI51zSP+4+dbCbSds7xwaIP+L8v/49tf22jx3k9ePycxymRXCJYQMMPiEghlaeavHNuqXPup1gF45tsJtJes30Nl39wOTd8egMNKzVk3h3z6N6y+4EELyJSiPl54bWBmX1vZt+Y2TnRCplZqpmlmVlaenq6P5FEmEg74AIMnjuYJgOaMHHlRPpc2IcZHWdwQrUT/IlBRCQODtlcY2YTgKMibOrmnBsd5WXrgLrOuc1m1gwYZWZNnHPbsxZ0zg0GBgOkpKT4M9B6lom0lxfbTpeh7Zmyagqt6rfijbZvcHTlo305tIhIPB0yyTvncj0RqXNuD7DHez7XzFYA/wDSch1hLHgDhGVsWE/fX4bxn0EnUzy5OIMvG0znUztrSAIRSVi+dKE0s6rAFudchpk1BBoBK/04Vk4tTF9Mp7GdmPP7HNr+oy0DLx1IrfK14hmSiIjv8pTkzexK4FWgKjDOzOY75y4CzgV6mtk+IAB0dc5tyXO0h2HP3j959ut/8+z3/ah0RCU+uPoDrmtynWrvIlIk5CnJO+dGAiMjrP8E+CQv+46FWau/o1O/C1hcdjc3rq9G394LqVJWNzSJSNGRkMMa7Nq7iwe+eoAWb53FtsBuxr4P7w3ZQpVdmkBbRIqWhEvyk36ZxEmDTuLlmS9zx6mpLJ6VwqUrkzXlnogUSQmT5P/46w+6jOlC66GtSbIkptwyiYEvLqX8rPlw2mkwaZKGJRCRIichBihL+z2Ndh+0Y/3O9Txy5iP0aNmDUlu2H7gBas4c2LRJwxOISJGTEEm+YaWGNKnahNHtR5NS0xsnrdoRB90ApaYaESmKEiLJVy5Vma9v/vrgld4NUNEGJRMRKQoSIslHlZSkJhoRKdIS5sKriIj8nZK8iEgCU5IXEUlgSvIiIglMSV5EJIEpyYuIJDAleRGRBGbOFZyRGc0sHfg1D7uoAmyKUTixpLhyR3HljuLKnUSMq55zrmqkDQUqyeeVmaU551LiHUdWiit3FFfuKK7cKWpxqblGRCSBKcmLiCSwREvyg+MdQBSKK3cUV+4ortwpUnElVJu8iIgcLNFq8iIiEkZJXkQkgRWqJG9m15rZYjMLmFlKlm2Pm9lyM/vJzC6K8voGZjbLK/ehmZXwKc4PzWy+91hlZvOjlFtlZgu9cml+xJLleD3MbG1YbG2ilLvYO4/LzeyxfIirt5n9aGY/mNlIM6sYpZzv5+tQv7uZlfT+vsu991J9P+KIcNw6ZjbZzJZ4/wP/ilCmpZltC/v7PplPsWX7d7Ggft45+8HMTs2HmI4NOw/zzWy7md2XpUy+nC8ze8vMNprZorB1lc1svJkt835WivLaW70yy8zs1sMKwDlXaB7A8cCxwBQgJWx9Y2ABUBJoAKwAkiO8fgTQ3ns+CLgzH2LuAzwZZdsqoEo+nr8ewEOHKJPsnb+GQAnvvDb2Oa4LgWLe8+eB5+NxvnLyuwN3AYO85+2BD/Ppb1cDONV7Xg74OUJsLYGx+fV+yunfBWgDfAEYcAYwK5/jSwbWE7xhKN/PF3AucCqwKGzdC8Bj3vPHIr3ngcrASu9nJe95pdwev1DV5J1zS51zP0XY1A74wDm3xzn3C7AcOC28gJkZcD7wsbfqHeAKH8PNPOZ1wHA/jxNjpwHLnXMrnXN7gQ8Inl/fOOe+ds7t9xZnArX9PF42cvK7tyP43oHge6m193f2lXNunXNunvd8B7AUqOX3cWOkHTDUBc0EKppZjXw8fmtghXMuL3fTHzbn3FRgS5bV4e+jaLnoImC8c26Lc24rMB64OLfHL1RJPhu1gNVhy2v4+z/AkcAfYckkUplYOwfY4JxbFmW7A742s7lmlupzLJnu8b4yvxXlK2JOzqWfOhKs9UXi9/nKye8eKuO9l7YRfG/lG6+J6BRgVoTNLcxsgZl9YWZN8imkQ/1d4v2eak/0ilY8zhdAdefcOu/5eiDSPKUxOW8Fbo5XM5sAHBVhUzfn3Oj8jieaHMbZgexr8Wc759aaWTVgvJn96H3q+xIXMBB4muA/5dMEm5I65uV4sYgr83yZWTdgP/B+lN3E/HwVNmZWFvgEuM85tz3L5nkEmyR2etdbRgGN8iGsAvt38a67XQ48HmFzvM7XQZxzzsx868te4JK8c+6Cw3jZWqBO2HJtb124zQS/JhbzamCRyuTYoeI0s2LAVUCzbPax1vu50cxGEmwuyNM/R07Pn5m9AYyNsCkn5zLmcZnZbcBlQGvnNUhG2EfMz1cWOfndM8us8f7GFQi+t3xnZsUJJvj3nXOfZt0envSdc5+b2QAzq+Kc83Uwrhz8XXx5T+XQJcA859yGrBvidb48G8yshnNundd0tTFCmbUErxtkqk3wemSuJEpzzRigvdfzoQHBT+PZ4QW8xDEZuMZbdSvg5zeDC4AfnXNrIm00szJmVi7zOcGLj4silY2VLO2gV0Y53hygkQV7IpUg+FV3jM9xXQw8AlzunNsdpUx+nK+c/O5jCL53IPhemhTtQymWvHb/N4GlzrmXopQ5KvP6gJmdRvD/29cPoBz+XcYAt3i9bM4AtoU1Vfgt6rfpeJyvMOHvo2i56CvgQjOr5DWtXuityx2/ryzH8kEwMa0B9gAbgK/CtnUj2DPiJ+CSsPWfAzW95w0JJv/lwEdASR9jfRvommVdTeDzsFgWeI/FBJst/D5/7wILgR+8N1mNrHF5y20I9t5YkU9xLSfY9jjfewzKGld+na9IvzvQk+AHEMAR3ntnufdeauj3+fGOezbBZrYfws5TG6Br5vsMuMc7NwsIXsA+Mx/iivh3yRKXAf29c7qQsJ5xPsdWhmDSrhC2Lt/PF8EPmXXAPi9/dSJ4HWcisAyYAFT2yqYAQ8Je29F7ry0Hbj+c42tYAxGRBJYozTUiIhKBkryISAJTkhcRSWBK8iIiCUxJXkQkgSnJi4gkMCV5EZEE9v/VbjQbmzqBIgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "train_data = list(get_data(50))\n", "x_target_label = np.array([-10, 10, 0.1])\n", "y_target_label = x_target_label * 2 + 3\n", "x_eval_label, y_eval_label = zip(*train_data)\n", "\n", "plt.scatter(x_eval_label, y_eval_label, color=\"red\", s=5)\n", "plt.plot(x_target_label, y_target_label, color=\"green\")\n", "plt.title(\"Eval data\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "上图中绿色线条部分为目标函数,红点部分为验证数据`train_data`。\n", "\n", "### 加载数据集\n", "\n", "加载`get_data`函数所产生的数据集到系统内存里面,并进行基本的数据处理操作。\n", "\n", "- `ds.GeneratorDataset`:将生成的数据转换为MindSpore的数据集,并且将生成的数据的x,y值存入到`data`和`label`的数组中。\n", "- `batch`:将`batch_size`个数据组合成一个batch。\n", "- `repeat`:将数据集数量倍增。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from mindspore import dataset as ds\n", "\n", "def create_dataset(num_data, batch_size=16, repeat_size=1):\n", " input_data = ds.GeneratorDataset(list(get_data(num_data)), column_names=['data', 'label'])\n", " input_data = input_data.batch(batch_size, drop_remainder=True)\n", " input_data = input_data.repeat(repeat_size)\n", " return input_data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "使用数据集增强函数生成训练数据,通过定义的`create_dataset`将生成的1600个数据增强为100组shape为16x1的数据集。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The dataset size of ds_train: 100\n", "dict_keys(['data', 'label'])\n", "The x label value shape: (16, 1)\n", "The y label value shape: (16, 1)\n" ] } ], "source": [ "data_number = 1600\n", "batch_number = 16\n", "repeat_number = 1\n", "\n", "ds_train = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number)\n", "print(\"The dataset size of ds_train:\", ds_train.get_dataset_size())\n", "step_size = ds_train.get_dataset_size()\n", "dict_datasets = next(ds_train.create_dict_iterator())\n", "\n", "print(dict_datasets.keys())\n", "print(\"The x label value shape:\", dict_datasets[\"data\"].shape)\n", "print(\"The y label value shape:\", dict_datasets[\"label\"].shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 定义网络模型\n", "\n", "`mindspore.nn`类是构建所有网络的基类,也是网络的基本单元。当用户需要自定义网络时,可以继承`nn.Cell`类,并重写`__init__`方法和`construct`方法。\n", "\n", "`mindspore.ops`模块提供了基础算子的实现,`nn.Cell`模块实现了对基础算子的进一步封装,用户可以根据需要,灵活使用不同的算子。\n", "\n", "如下示例使用`nn.Cell`构建一个简单的全连接网络,用于后续自定义内容的示例片段代码。在MindSpore中使用`nn.Dense`生成单个数据输入,单个数据输出的线性函数模型:\n", "\n", "$$f(x)=wx+b\\tag{2}$$\n", "\n", "并使用Normal算子随机初始化公式 (2) 中的参数$w$和$b$。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from mindspore import nn\n", "from mindspore.common.initializer import Normal\n", "\n", "class LinearNet(nn.Cell):\n", " def __init__(self):\n", " super(LinearNet, self).__init__()\n", " self.fc = nn.Dense(1, 1, Normal(0.02), Normal(0.02))\n", "\n", " def construct(self, x):\n", " fx = self.fc(x)\n", " return fx" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "初始化网络模型后,接下来将初始化的网络函数和训练数据集进行可视化,了解拟合前的模型函数情况。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd4VNXWx/HvSkJRQKkJSFcBAVGQwEUQKRYQC8K91msvqFe8FLuoAUVRroINUbBgh1cUsSCKUoQACkg30gSRHqUq0jL7/WMmwxAmBTKTM0l+n+fJkzn7tMXJMGv22fvsbc45RERE4rwOQEREYoMSgoiIAEoIIiISoIQgIiKAEoKIiAQoIYiICBCBhGBmNc1sipmlmdlSM+sVKO9vZuvNbEHgp0v+wxURkWix/D6HYGbVgGrOuR/NrBwwD7gUuBz40zn3TP7DFBGRaEvI7wGccxuBjYHXu8wsDaie3+OKiEjByncN4ZCDmdUBvgNOBfoCNwA7gbnA3c65bWH26QH0AChTpkzzU045JWLxiIgUB/PmzfvdOVclv8eJWEIws7LANOAJ59zHZpYE/A444HH8t5VuyukYycnJbu7cuRGJR0SkuDCzec655PweJyK9jMysBPAR8J5z7mMA59xm51yGc84HjARaRuJcIiISHZHoZWTA60Cac25ISHm1kM26AUvyey4REYmefDcqA22Aa4HFZrYgUPYQcJWZNcV/y2gNcFsEziUiIlESiV5GMwALs2pCfo8tIiIFR08qi4gIoIQgIiIBSggiIgIoIYiIFFrb/t5GnefqROx4kehlJCIiBcg5x1UfXcWYpWMielzVEERECpFRC0YR91hcMBmktEuJ2LFVQxAR8ZLPB+npkJgIFq4Hv99P6T/R+OXGweVmVZsx+5bZlIwvyQAGRCQUJQQREa/4fNChA8ycCa1bw5QpEHfojZu/9v1Fw2EN+W3nb8Gy1b1WU6d8nYiHo1tGIiJeSU/3J4MDB/y/09MPWd1zQk/KDiobTAbjrhiHS3FRSQagGoKIiHcSE/01g8waQmIiAON/Hs+lYy4NbtazRU9e7PJi1MNRQhAR8YqZ/zZRoA1hzY5fqft83eDqmsfVJO3ONMqULFMg4SghiIh4KS6OfZUr0GpEc+Zvmh8sXnLHEhonNs5hxyiEUqBnExEprnw+2LwZskxKNmDqAEoNLBVMBm92fROX4go8GYBqCCIikZNdF9IwvYmm/DqNjm93DG5yReMr+OCfH2A5dD2NNiUEEZFIyKkLaUhvos0LU6n6eHxwt1Lxpdh490YqHFPBo8APisSMaTXNbIqZpZnZUjPrFSivaGaTzGxF4Lf3/1oRkWjJqQtpYiIZrc+k8zVQtU9GsHj2zbPZ8/CemEgGEJk2hAPA3c65hkAr4E4zawQ8AHzrnKsHfBtYFhEpmjK7kCYkHNKFFOClOcNI6Didr072Lw9ZcSLukQz+UeMfHgUbXiRmTNsIbAy83mVmaUB1oCvQPrDZW8BU4P78nk9EJCZl6UKKGfM2zCN5ZHJwkw5rjK/fdiTErYVn0yEpycOADxfRNgQzqwM0A74HkgLJAufcRjNLzGFXEZHCLy4OkpLYsWcHNYfWZNe+XcFVG/qsp9rFV0HczMNqELEiYgnBzMoCHwG9nXM789pSbmY9gB4AtWrVilQ4IiIFzjnHteOu5b3F7wXLJl07iXNPPNe/kKUGEWsi8hyCmZXAnwzec859HCjebGbVAuurAVvC7eucG+GcS3bOJVepUiUS4YiIFLj3Fr1H3GNxwWTw0FkP4VLcwWQAwRpELCYDiEANwfxVgdeBNOfckJBVnwLXA08Ffo/P77lERGLNz7//TMNhDYPLTRKbMOfWOZRKKOVhVEcnEreM2gDXAovNbEGg7CH8ieD/zOxmYC1wWQTOJSISE3bv382pL5/K6u2rg2Ur71rJSRVP8jCq/IlEL6MZQHb1n3Pye3wRkVjTZ2Ifnvv+ueDy2MvG8s9G//QwosjQk8oiInn0+fLPufiDi4PLtzW/jeEXDvd0uIlIUkIQEcnF2h1rqf1c7eByUpkkVv53JWVLlvUwqshTQhARycb+jP2c9eZZ/LD+h2DZwtsXclrSaR5GFT0a/lpEJIwnvnuCkgNLBpPByItH4lJckU0GoBqCiMghpv86nbNHnR1c7nZKN8ZePpY4K/rfn5UQRESA9L/SSXzm4HAScRbH5ns2U/nYyh5GVbCUEESkWPM5H5d8cAlfrPgiWJZ6Uyqta7b2MCpvKCGISLH1ytxXuOOLO4LLT53zFPefVXwHZVZCEJFiZ/7G+Zwx4ozgcttabZl8/WQS4or3R2Lx/teLSLGyc+9O6jxXh217tgXLfuvzGzWOq+FhVLGj6Debi0ix55zjxvE3cvxTxweTwZdXT8ClOCWDEEoIIlKkjV4ymrjH4hi1YBQA9/5aAzcwgc63PAU+n7fBxRjdMhKRwsHnO6LJZVb8sYL6L9UPLjeo1IAF3b6idJ2T4cABmDnTf7wYm8bSS6ohiEjs8/mgQweoUQPat8/xm/3f+/+m/ov1D0kGy3su5+eeP1P6hFr+6SsTEmJ2GksvKSGISOxLT/d/ow/9Zh/GfZPu49gnj2XF1hUAjP7naFyKo16lev4NzPzTWK5bB1OnxuzMZV7RLSMRiX2Jif5v9DPDT1A/ceVELnjvguDyjU1v5PVLXj84LHXo7abMaSzlMBFJCGb2BnARsMU5d2qgrD9wK5CZyh9yzk2IxPlEpJjJ/GafpQ1h/c711Bh6sJdQxWMqsrrXao4rddzBfTNvN2UmkylT/ElBDhOpqzIK6BymfKhzrmngR8lARI5eyAT1B3wHOOuNsw5JBj/2+JE/7vvj0GQAeb7dJBFKCM6574CtkTiWiEhOBs94mhKPlyD1t1QAhl84HJfiaFatWfgdMm83qSE5V9FuQ+hpZtcBc4G7nXPbsm5gZj2AHgC1atWKcjgiUij5fKQu+oKzxl8SLLro94qMf24zcfG5fIxlc7tJDmfOucgcyKwO8HlIG0IS8DvggMeBas65m3I6RnJysps7d25E4hGRouGPP9Op8kwiLuRzfMtgqLIvwd9bSA3EmNk851xyfo8TtZYV59xm51yGc84HjARaRutcIlL0+JyPbmO6UfnZg8lg2ltxuK/O9CcD3f6JuKjdMjKzas65jYHFbsCSaJ1LRIqWkfNG0uPzHsHlJ1bV4aEP1vmTwDffwLJl0KiRbv9EWKS6nX4AtAcqm9k6IAVob2ZN8d8yWgPcFolziUjRtWjzIk5/5fTg8j+q/4PpN06nhMXD/9KhcmXo2FFdSKMkIgnBOXdVmOLXI3FsESn6du3dxckvnsyWv7YEy37t/Su1jg/paJKUBJs3H96FVG0IEaPUKiKecc7R47MeHPfUccFk8NlVn+FS3KHJIJO6kEaVhq4QEU+M/Wksl314WXC59z96M7Tz0Jx3UhfSqFJCEJECtWrrKk5+8eTg8kkVTmLxHYs5psQxeTuAxiKKGiUEESkQew/spfmI5ixNXxosS7tiOqc0aKNv+jFCbQgiEhk+n7/RN8zDrv2+7UfpJ0oHk8E7Xd/CTT6bU5p0yHV+Ayk4Sggikn/ZTGAzadUkbIDx5IwnAbjmtGvwPerjmmqdNOBcDNItIxHJvywjim5Ys5jq7zQNri5Xshxr+6ylfOny/oJc5jcQbyghiEj+BT7gD8xK5bz/lGFqSDKYc+sckk/IMsyOegvFJN0yEpH8M2PIoEso0S+DqRV2APBC5xdwKe7wZJApZH4DiQ2qIYhIvny/7ntavd4quNzppE58cfUXxMfFexiVHA0lBBE5Klv/3kq1Z6uxL2NfsGzT3ZtIKqtnBAor3TISkSPinOPyDy+n0uBKwWQw+brJuBSnZFDIqYYgInk2asEobhx/Y3C5f7v+pLRP8TAiiSQlBBHJ1dItSzl1+KnB5ebVmjPz5pmUjC/pYVQSaUoIIpKtv/b9RYOXGrB+1/pg2epeq6lTvo53QUnURKQNwczeMLMtZrYkpKyimU0ysxWB3xUicS4RKRg9J/Sk7KCywWQw7opxuBSnZFCERapReRTQOUvZA8C3zrl6wLeBZRGJcZ/8/Ak2wBg2ZxgAPVv0xKU4Lj3lUo8jk2iL1Ixp35lZnSzFXfFPqwnwFjAVuD8S5xORo+Dz5fhk8OptqznxhRODyzWPq0nanWmUKVmmIKMUD0Wz22mSc24jQOB32MFKzKyHmc01s7npGuBKJDqyDj63cWNwVNJ9Gfs449UzDkkGS+5Ywto+a5UMihnPn0Nwzo1wziU755KrVKnidTgiRVPo4HPTp0PNmtC+PSmTH6XUwFLM3zQfgFFdR+FSHI0TG3scsHghmr2MNptZNefcRjOrBmzJdQ8RiY7Q0UUPHGBKzQw6dvwOpn8HwBUndeWDqz/G4jz/jigeiuZf/1Pg+sDr64HxUTyXiOQkMLropmXzsP7Q8QZ/cemE0mz97kxG3/gF1qGDJqop5iLV7fQDYBbQwMzWmdnNwFPAeWa2AjgvsCwi0ZLDjGUZvgzOf68z1d45PVg2+6ZZ/H3LGip8N0cT1QgQuV5GV2Wz6pxIHF9EcpHZaJw54cyUKf7hpYEXv3+R/078b3DTIecPoc+ZffwLzmmiGgnSk8oiRUGWGcvYsoW5vnW0GNkiuEnHuh35+pqvDx2WWhPVSAglBJGiILPRePp0ticcoMaw6vyVcLA9YEPfDVQrVy38vpkT1Uixpy4FIkWBGe6DD7j6n1DhAYLJYNK1k3ApLvtkIBJCCUGkCHhn4TvEjazOB6f6G5T7ramJe9THuSee63FkUpjolpFIIZaWnkajlxsFl5skNmFO188pVa2m2gPkiCkhiBRCu/fv5tSXT2X19tXBspV3reSkiid5GJUUdrplJFLI9J7YmzJPlgkmg7GXjcWlOCUDyTfVEEQKic+Xf87FH1wcXL6t+W0Mv3A4pltDEiFKCCIxbu2OtdR+rnZwOalMEiv/u5KyJct6GJUURUoIIjFqf8Z+2rzRhjkb5gTLFt2+iCZJTTyMSooytSGIxKCB3w2k5MCSwWQw8uKRuBSnZCBRpRqCSAz57tfvaDeqXXC5e8PufHjZh8SZvrtJ9CkhiMSALX9tIemZg8NHxFs8m+/ZTKVjK3kYlRQ3SggiHvI5Hxe9fxFfrvwyWJZ6Uyqta7b2MCoprpQQRDwyfM5w/jPhP8Hlp899mvva3OdhRFLcRT0hmNkaYBeQARxwziVH+5winvL5chxOev7G+Zwx4ozgcttabZl8/WQS4vT9TLxVUO/ADs653wvoXCLeyWGimh17dlD7udrs2LsjuPm6Puuoflx1r6IVOYS6LohEUtaJatLTcc5xwyc3UP7p8sFkMPHfE3EpTslAYkpBJAQHfG1m88ysR9aVZtbDzOaa2dx0zecqhV3mRDUJCdC6NaO3TCbusTjeWvgWAPe1vg+X4uh0ciePAxU5nLkwE3JH9ARmJzjnNphZIjAJuMs59124bZOTk93cuXOjGo9I1Pl8LF8xmwaj2wSLGlRqwIIeP1J62y5NVSkRZ2bzItE+G/UagnNuQ+D3FmAc0DLa5xTxyt/7/6b+sFMOSQbLey7n5//8ROnzLoAaNaB9e39bg0iMiWpCMLMyZlYu8zVwPrAkmucU8cq9X9/LsU8ey4qtKwAY/c/RuBRHvUr1wrYtiMSaaPcySgLGBYbnTQDed85NjPI5RQrUlyu+pMv7XYLLNzW9idcuee3QYakz2xYyex8lJnoQqUjOopoQnHO/AKdH8xwiXlm3cx01h9YMLlc6phK/9PqF40odd/jGZv4uqDk8nyDiNT0JI3KE9mfsp92odsxaNytYNv+2+TSt2jTnHePiICkp521EPKTnEESOwNMznqbkwJLBZDD8wuG4FJd7MhApBFRDEMmDrLOWXVz/Yj658hMNSy1FihKCSA5279/N4NTBDE4dDECcxbHp7k1UKVPF48hEIk8JQSQM5xxjlo7hvkn38dvO37ii8RU8fe7T1C5fO/edRQopJQSRLOZtmEevib1I/S2VZlWb8V7392hbu63XYYlEnRKCFA+5DEkNsOnPTfT7th9vLniTKmWq8NrFr3FD0xuIj4sv4GBFvKGEIEVfDkNSA+w9sJcXvn+Bx797nD0H9nD3mXfz8NkPc3zp4z0MWqTgKSFI0ZO1NhBu2IikJJxzfL78c/p+3ZeVW1dyUf2LePb8Z6lfqb7X/wIRT6jPnBQtmbWB0EHksgxJTWIiS7cspdO7nbhk9CWUiCvBxH9P5LOrPlMykGJNNQQpWrKpDWQOG7G1XAL9J/bi5TkvU65UOZ7v/Dx3JN9BifgSXkcu4jklBClashlE7gA+Xv11LI9OfZTte7ZzW/PbeKzDY1Q+trLHAYvEDiUEKVrCDCL37S/f0mtiL5amL6VDnQ483/l5miQ18TpSkZijhCBFT2AQuVVbV3HPpHv45OdPqFu+Lh9f/jGXnnLpocNSi0iQEoIUObv27uKJ6U8wdPZQSsSVYNA5g+jdqjelE0of3CgPzyWIFDdR72VkZp3NbJmZrTSzB6J9Pim+fM7HqAWjqP9SfZ5OfZqrTr2K5Xct54GzHjg8GWTtiSQi0a0hmFk8MAw4D1gHzDGzT51zP0XzvFL8zPxtJr0m9mLuhrm0qtGK8VeOp2X1bKbvzq4nknjGOX9ezvqTkeH/yWv50exT2Msj+X0m2reMWgIrAzOnYWajga5AoUgImW9S5w7/8ao8FmPy8lpsy1jH1777WcT7lOMEurl3OXXtVXz2ShyfZncsXyKu6ju49RtxSdVxgxLx5eFaz54Nixd7/a4UiZ5oJ4TqwG8hy+uAf4RuYGY9gB4ACQlNadAgNj50JMYl/A2tn4GznoK4DEh9mF2p9zNuX1nG4W8W8P+44Ou4OAu8NsyuwI5x2A4j7k0L2Z6Q7Q9d3rTJ63900WYG8fH+6x76Ex+v8pzK4+KgS5fcr29eRDshhGutO+Tj1jk3AhgBULFismvWLPf/mMW1PBZjKtBy54MLOvHJjhk8eCH8dsweup/yL54+53/U7V/nkP2AQ8cwOjPrGEaWzdtTpPiKdkJYB9QMWa4BbMhu4xNPhNGjoxyRFFrzl06mV4NvmF4LTt8E71zxMe2adct+B7UViByRaPcymgPUM7O6ZlYSuBL4NMrnlCJmy19b6PFZD5qPPZ+0qiV49Ys45v3UlnZNL815xzBjGIlI9qJaQ3DOHTCznsBXQDzwhnNuaTTPKUXHvox9vPTDSwyYNoDd+3fTu1VvHm37MOX/uz9vzw+YHfbUsohkL+oPpjnnJgATon0eKTqcc0xYMYG+X/dl+R/L6VKvC0POH0KDyg38Gxx7BAcLPLUsIrnTk8oSU9LS0+j7dV8mrpxIg0oN+OLqL+hSL0JdKEQkR0oIEhO2/b2NAdMGMGzOMMqUKMOQ84dwZ8s7KRlf0uvQRIoNJQTxVIYvg5E/juThyQ+z9e+t9Gjeg8c7PE6VMlW8Dk2k2FFCEM9MWT2F3l/1ZtHmRbSr3Y7nOj9H06pNvQ5LpNhSQpACt3rbau6ZdA8fp31M7eNrM/aysXRv2B0NSy3iLSUEibxshpb+c9+fDJo+iGdnPUt8XDwDOwyk75l9OabEMR4GKyKZlBAkskKHi2jRAqZPxxdnvLvoXR745gE2/rmRa0+7lkHnDKL6cdW9jlZEQighSGSFDhcxaxbfX3g6//1nGX7Y8AMtq7fk4ys+plWNVl5HKSJhKCFIZCUmQosWrF8yiwfPhXdOX0q17Um8delbXHPaNcRZ1OdkEpGjpIQgeZeHaSf3ZOxlyKAuPPnN9+zHx4NravLg8CWUK31cAQcrIkdKX9ckb3KZdtI5x0c/fUTDYQ3pN/UROp3albTrvufJN35VMhApJFRDkLzJOpT0Tz9B48ZgxsJNC+n9VW+mrplKk8QmfHvdt3Ss29HriEXkCCkhSN5kDiU9cyaUKQNNm5LevgWP9DqNkfNfo0LpCrzc5WVubX4rCXF6W4kURvqfK3mTOZT0kiXsb96UYS0c/VvM5s/5c7ir5V2ktEuhwjEVvI5SRPJBCUGOyJcp/6bvbY6fq0Cn38szpM8MGiU29josEYmAqDUqm1l/M1tvZgsCPxrDuBBb9vsyLnzrfLo0XUJGHHz+Hnw5twGNKjf0OjQRiZBo1xCGOueeifI5JIq279nO49Me54UfXuDYhGN45uda3PXhWkpmAKvnaZ5ikSJEt4wkrAxfBq/Pf52HJz/M77t/55b1SQwck07iaadDixNg7lzNUyxSxEQ7IfQ0s+uAucDdzrltWTcwsx5AD4BatWpFORwBcn3AbNqaafSa2IuFmxfStlZbnm/5KM2aXgAHMmD2bFi71j81peYpFilS8tWGYGbfmNmSMD9dgeHASUBTYCPwbLhjOOdGOOeSnXPJVapoUpSoy+EBszXb13D5h5fT/q32bNuzjTH/GsO0G6bRrNE5/tpAQoL/d9Wq/ttESgYiRUq+agjOuXPzsp2ZjQQ+z8+55AiEqwFkljl36ANm6en8VaEsT814imdmPYNhDGg/gHta38OxJUJms58yJddhK0SkcItmL6NqIYvdgCXROpeECFcDCC274go480xISMC1PpP3Nk+iwUsNGDh9IN0bdmdZz2U82u7RQ5MB+G8RqVYgUqRFsw1hsJk1BRywBrgtiueSTFmHmEhP9yeE0LK1a5nz+0J6/fAYs8ZdS/NqzRnzrzG0qdXG6+hFxENRSwjOuWujdWzJQegQE61bQ+XK/trBgQNgxsb2zXnw+wd5a+FbJJVJ4o1L3uD6ptdrWGoRUbfTIidziInM+/1btsCsWexJgOfOhCc6LGHfkvnc3+Z+Hmr7EMeV0kikIuKnhFAUZd7vB1yVKoy/pD531/yJXyo4up54Ls+c/wwnVzzZ4yBFJNYoIRRhizcvpvdXvZl82k80rtCASRe+yLknned1WCISo5QQiqA/dv/Bo1Me4ZV5r3J8qeN56YKXuC35Ng1LLSI50idEEbI/Yz/D5w6n/9T+7Ny9jf/MM/r/fQqV7r3DfxtJRCQHSghFxNervqb3xN6k/Z7GudXP5rnnUmm8KQMS5mgAOhHJE31tLORW/LGCS96/mE7vdmJfxj7GXzmer2+aQuP6bfxDTbRoARoSRETyQAmhkNqxZwf3fn0vjV9uzNS0Lxn8TRxLP67GJfUuwuLi4NtvITkZfvgBzjoLMjK8DllEYpwSQiGT4cvg9R9fp/5L9Xl21rNcW/9fLH8R7p3ho1TqbP/tIYA//vAPUZ2RAbNm+ZNCyEB2IiJZKSEUIjPWzqDlyJbc8tktnFzhZH649Qdev/w9qp7e5uBIpJnzEyQm+m8XZZoz52CyEBEJQwnBKz4fbN7sH300F2t3rOXKsVfS9s22bFmzhPfHxTHj7XiSq55x8Mnkdetg6tSDg8+ZwfTp0KoVxMdDmzaazEZEcqSE4IUc5iQItXv/bvpP7c8pL53C+GXjSWl+Nz8/n8FVC33YzFkHv/FnNxJpfDykpsL69YcmCxGRMNTt1AvhRiQN6RbqnGPM0jHcO+le1u1cxxWNr2DweYOpdVxNaDHn4MB1efnGHzKMhYhITpQQvJB1RNKQD/Z5G+bRa2IvUn9LpVnVZrzf/X3a1m57cF9NVCMiUaKEEA25zFl82IikZmz6cxP9vu3HmwvepEqZKrx28Wvc0PQG4uPiD91X3/hFJEryO6fyZWa21Mx8ZpacZd2DZrbSzJaZWaf8hVmI5LF9IPODfW/GPganDqb+i/V5Z9E73H3m3SzvuZybz7j58GQgIhJF+a0hLAG6A6+GFppZI+BKoDFwAvCNmdV3zhX9p6NyaR/I5Jzjs+Wf0fervqzatoqL61/Ms+c/S71K9Q72QNJtIREpQPmqITjn0pxzy8Ks6gqMds7tdc6tBlYCLfNzrkIjs30g63MBIZZuWUqndzvRdXRXSsaXZOK/J/LpVZ8eTAZ5qWGIiERYtNoQqgOzQ5bXBcoOY2Y9gB4AtWrVilI4BShr+4Bz/lnLEhPZumcbKVNSGD53OOVKleOFzi9we/LtlIgvcXD/PNYwREQiLdeEYGbfAFXDrOrnnBuf3W5hysI+geWcGwGMAEhOTs79Ka3CILPhN/Bt/8CsVF79V10ebbqV7Xu2c3vz2xnQYQCVj618+L459EASEYmmXBOCc+7cozjuOqBmyHINYMNRHKdwS0/nm40z6H2rj6WJK+lY8Syeu/hlmiQ1yX6fMD2QREQKQrSeVP4UuNLMSplZXaAe8EOUzhWTVm1dxaVTbuO8f/vYXQLGLWrENzdNyzkZZMruyWMRkSjKVxuCmXUDXgSqAF+Y2QLnXCfn3FIz+z/gJ+AAcGex6GEE7Nq7iyemP8HQ2UMpEVeCQR2fpPeJV1P6hFr6gBeRmJavhOCcGweMy2bdE8AT+Tl+YeJzPt5e+DYPfvsgm/7cxPWnX8+T5zzJCeVO8Do0EZE80ZPKETDzt5n0mtiLuRvm0qpGK8ZfOZ6W1YtHL1sRKTqUEPJh3c513P/N/by/+H1OKHcC73Z7l6ubXI3p1pCIFEJKCEdh9/7dPDPzGZ5OfRqf8/Fw24e5/6z7KVuyrNehiYgcNSWEI+Cc48OfPuTeSfeydsdaLmt0GYPPG0yd8nW8Dk1EJN+UEPJo/sb59JrYi+lrp3N60um8fenbtKvV1v+8gHPqQSQihZ5mTMvFlr+2cOunt9J8RHPSfk/j1YteZV6Pef5koDGHRKQIUQ0hG/sy9vHi9y/y2HePsXv/bvq06sMj7R6hfOny/g02b9aYQyJSpCghZOGc44sVX9D3q76s2LqCLvW6MOT8ITSo3ODQDTXmkIgUMUoIIdLS0+jzVR++WvUVDSo1YMLVE7ig3gXhN9aYQyJSxCghANv+3saAaQN46YeXKFuyLEM7DeXOFnceOix1OJrOUkSKkGKdEA74DjBy3kgemfII2/Zs49YzbuXxDo9TpUwVr0MTESlwxTYhTF49md4Te7N4y2La1W7H852f5/Sqp3sdloiIZ4pdQvhl2y/cO+lePk77mDrl6zD2srF0b9hdw02ISLFXtBOCzxds9N21708GzRjEkFnubzSqAAAKT0lEQVRDiI+LZ2CHgfQ9sy/HlDjG6yhFRGJC0U0IgekrfTNTebf7yTzQcicb/9zItaddy6BzBlH9uLBTPIuIFFv5elLZzC4zs6Vm5jOz5JDyOmb2t5ktCPy8kv9Qj1B6OrN/TeXMGzK4vtEyah5blVk3z+Ltbm8rGYiIhJHfGsISoDvwaph1q5xzTfN5/KOyfud6Hkh9gHdvzKDaLnjrpwZc88gc4uLivQhHRKRQyO+MaWlAzDTI/r3/b4bMGsKTM54kw5fBQ20e5MFTbqFs9bp6cExEJBfRbEOoa2bzgZ3Aw8656dE6kXOOj9I+4t5J97Jm+xq6N+zO/877HydWODFapxQRKXJyTQhm9g1QNcyqfs658dnsthGo5Zz7w8yaA5+YWWPn3M4wx+8B9ACoVavW4UcK6SkU7lv+wk0L6TWxF9N+nUaTxCZMvm4yHep2yO2fJSIiWeSaEJxz5x7pQZ1ze4G9gdfzzGwVUB+YG2bbEcAIgOTkZHfIygMH4OyzYc4c/wByU6b4h4sA0v9K55EpjzDyx5FUKF2B4RcO55YzbiEhruh2nBIRiaaofHqaWRVgq3Muw8xOBOoBvxzRQXw+aNsWZs/2LweGmN5XuQLDfhjGgGkD+Gv/X9zV8i5S2qVQ4ZgKkf5niIgUK/lKCGbWDXgRqAJ8YWYLnHOdgLOBx8zsAJAB3O6c23pEB09P99cMMrVowZc75tHnw74s+2MZnU7qxNBOQ2lYpWF+/gkiIhKQ315G44BxYco/Aj7Kz7FJTIQ2bSA1lWXtT6XvjeWZ8MGF1KtYj8+v+pwu9brk3Lspl7YHERE5VOxOoWnG9i/H0ffDWzi13VJm/JbKM+c9w5L/LOHC+hfmngw0vaWIyBGJyRbYDF8Gr89/nX6T+/HH7j+45YxbGNhxIIll8jgrWXq6prcUETlCMZcQpq2ZRq+JvVi4eSFta7Xl+c7P06xasyM7iKa3FBE5Yuacy32rAlLxpIpu23XbqHV8Lf533v+4rNFlR/8UtNoQRKSYMLN5zrnk3LfMWUzVELbv2c5j7R/jntb35H9Yak1vKSJyRGKqhnBas9PcovmLvA5DRKRQiVQNIaZ6GZWML+l1CCIixVZMJYQc+XyweTPEUI1GRKQoKRwJQc8ViIhEXeFICOGeKxARkYgqHAkh87mChAQ9VyAiEiUx1e00W2b+oa/1XIGISNQUjoQAeq5ARCTKCsctIxERiTolBBERAZQQREQkIF8Jwcz+Z2Y/m9kiMxtnZuVD1j1oZivNbJmZdcp/qCIiEk35rSFMAk51zp0GLAceBDCzRsCVQGOgM/CymcXn81wiIhJF+UoIzrmvnXMHAouzgRqB112B0c65vc651cBKoGV+ziUiItEVyW6nNwFjAq+r408QmdYFyg5jZj2AHoHFvWa2JIIxRUtl4Hevg8gDxRlZijNyCkOMUHjibBCJg+SaEMzsG6BqmFX9nHPjA9v0Aw4A72XuFmb7sKPSOedGACMCx5kbiSFco01xRpbijKzCEGdhiBEKV5yROE6uCcE5d24ugVwPXASc4w5OrrAOqBmyWQ1gw9EGKSIi0ZffXkadgfuBS5xzu0NWfQpcaWalzKwuUA/4IT/nEhGR6MpvG8JLQClgUmDu49nOududc0vN7P+An/DfSrrTOZeRh+ONyGc8BUVxRpbijKzCEGdhiBGKWZwxNYWmiIh4R08qi4gIoIQgIiIBBZ4QzOwyM1tqZj4zS86yLtfhLsysrpl9b2YrzGyMmZUsgJjHmNmCwM8aM1uQzXZrzGxxYLuIdAM7wjj7m9n6kFi7ZLNd58A1XmlmD3gQZ7ZDnmTZrsCvZ27XJtBRYkxg/fdmVqcg4soSQ00zm2JmaYH/S73CbNPezHaEvBceLeg4A3Hk+Dc0vxcC13ORmZ3hQYwNQq7TAjPbaWa9s2zjyfU0szfMbEvo81lmVtHMJgU+AyeZWYVs9r0+sM2KQG/Q3DnnCvQHaIj/IYqpQHJIeSNgIf5G6rrAKiA+zP7/B1wZeP0KcEcBx/8s8Gg269YAlQv6moacvz9wTy7bxAeu7YlAycA1b1TAcZ4PJARePw08HQvXMy/XBvgP8Erg9ZXAGA/+ztWAMwKvy+EfNiZrnO2Bzws6tiP9GwJdgC/xP7vUCvje43jjgU1A7Vi4nsDZwBnAkpCywcADgdcPhPv/A1QEfgn8rhB4XSG38xV4DcE5l+acWxZmVa7DXZi/K1NHYGyg6C3g0mjGG+b8lwMfFNQ5o6AlsNI594tzbh8wGv+1LzAu+yFPvJaXa9MV//sO/O/DcwLviwLjnNvonPsx8HoXkEY2IwEUAl2Bt53fbKC8mVXzMJ5zgFXOuV89jCHIOfcdsDVLceh7MLvPwE7AJOfcVufcNvzjznXO7Xyx1IZQHfgtZDnccBeVgO0hHybZDokRJW2Bzc65Fdmsd8DXZjYvMCSHF3oGqt5vZFOVzMt1Lkg34f+GGE5BX8+8XJvgNoH34Q7870tPBG5ZNQO+D7P6TDNbaGZfmlnjAg3soNz+hrH2fryS7L/wxcL1BEhyzm0E/5cDINwk80d1XaMyhablYbiLcLuFKcvaJzbPQ2IcqTzGfBU51w7aOOc2mFki/mczfg5k+IjJKU5gOPA4/mvyOP7bWzdlPUSYfSPe9zgv19MOH/Ikq6hfzyw8fQ8eKTMrC3wE9HbO7cyy+kf8tz3+DLQlfYL/AdGCltvfMJauZ0ngEgKjNmcRK9czr47qukYlIbhchrvIRl6Gu/gdf5UyIfDtLGJDYuQWs5klAN2B5jkcY0Pg9xYzG4f/FkREP8Dyem3NbCTweZhVBTKsSB6uZ7ghT7IeI+rXM4u8XJvMbdYF3hPHc3iVPurMrAT+ZPCec+7jrOtDE4RzboKZvWxmlZ1zBTpQWx7+hrE0zM0FwI/Ouc1ZV8TK9QzYbGbVnHMbA7fXtoTZZh3+do9MNfC32+Yolm4Z5TrcReCDYwrwr0DR9UB2NY5IOxf42Tm3LtxKMytjZuUyX+NvOC3QkVuz3Hvtls355wD1zN9bqyT+KvKnBRFfJst+yJPQbby4nnm5Np/if9+B/304ObuEFi2BNovXgTTn3JBstqma2bZhZi3x/1//o+CizPPf8FPgukBvo1bAjszbIR7I9g5ALFzPEKHvwew+A78CzjezCoFbx+cHynLmQat5N/zZay+wGfgqZF0//L08lgEXhJRPAE4IvD4Rf6JYCXwIlCqguEcBt2cpOwGYEBLXwsDPUvy3Rgr62r4DLAYWBd401bLGGVjugr9nyiqP4lyJ//7mgsDPK1nj9Op6hrs2wGP4kxdA6cD7bmXgfXiiB9fvLPzV/0Uh17ALcHvmexToGbhuC/E33Lf2IM6wf8MscRowLHC9FxPS87CAYz0W/wf88SFlnl9P/AlqI7A/8Ll5M/42q2+BFYHfFQPbJgOvhex7U+B9uhK4MS/n09AVIiICxNYtIxER8ZASgoiIAEoIIiISoIQgIiKAEoKIiAQoIYiICKCEICIiAf8PlKm6QJSDsSYAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import mindspore as ms\n", "\n", "net = LinearNet() # 初始化线性回归网络\n", "\n", "model_params = net.trainable_params() # 获取训练前的网络参数 w 和 b\n", "\n", "x_model_label = np.array([-10, 10, 0.1])\n", "y_model_label = (x_model_label * model_params[0].asnumpy()[0] + model_params[1].asnumpy()[0])\n", "\n", "plt.axis([-10, 10, -20, 25])\n", "plt.scatter(x_eval_label, y_eval_label, color=\"red\", s=5)\n", "plt.plot(x_model_label, y_model_label, color=\"blue\")\n", "plt.plot(x_target_label, y_target_label, color=\"green\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 自定义损失函数\n", "\n", "损失函数(Loss Function)用于衡量预测值与真实值差异的程度。深度学习中,模型训练就是通过不停地迭代来缩小损失函数值的过程,因此在模型训练过程中损失函数的选择非常重要,定义一个好的损失函数可以帮助损失函数值更快收敛,达到更好的精度。\n", "\n", "[mindspore.nn](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/mindspore.nn.html#损失函数)提供了许多通用损失函数供用户选择, 也支持用户根据需要自定义损失函数。\n", "\n", "自定义损失函数类时,既可以继承网络的基类`nn.Cell`,也可以继承损失函数的基类`nn.LossBase`。`nn.LossBase`在`nn.Cell`的基础上,提供了`get_loss`方法,利用`reduction`参数对损失值求和或求均值,输出一个标量。下面将使用继承LossBase的方法来定义平均绝对误差损失函数(Mean Absolute Error,MAE),MAE算法的公式如下所示:\n", "\n", "$$ loss= \\frac{1}{m}\\sum_{i=1}^m\\lvert y_i-f(x_i) \\rvert \\tag{3}$$\n", "\n", "上式中$f(x)$为预测值,$y$为样本真实值,$loss$为预测值与真实值之间距离的平均值。\n", "\n", "使用继承LossBase的方法来自定义损失函数时,需要重写`__init__`方法和`construct`方法,使用`get_loss`方法计算损失。示例代码如下:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "from mindspore import nn, ops\n", "\n", "class MyMAELoss(nn.LossBase):\n", " \"\"\"定义损失\"\"\"\n", " def __init__(self):\n", " super(MyMAELoss, self).__init__()\n", " self.abs = ops.Abs()\n", "\n", " def construct(self, predict, target):\n", " x = self.abs(target - predict)\n", " return self.get_loss(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 自定义优化器\n", "\n", "优化器在模型训练过程中,用于计算和更新网络参数,合适的优化器可以有效减少训练时间,提高模型性能。\n", "\n", "[mindspore.nn](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/mindspore.nn.html#优化器)提供了许多通用的优化器供用户选择,同时也支持用户根据需要自定义优化器。\n", "\n", "自定义优化器时可以继承优化器基类`nn.Optimizer`,重写`__init__`方法和`construct`方法实现参数的更新。\n", "\n", "如下示例实现自定义优化器Momentum(带动量的SGD算法):\n", "\n", "$$ v_{t+1} = v_t × u+grad \\tag{4}$$\n", "\n", "$$p_{t+1} = p_t - lr × v_{t+1} \\tag{5}$$\n", "\n", "其中,$grad$ 、$lr$ 、$p$ 、$v$ 和 $u$ 分别表示梯度、学习率、权重参数、动量参数(Momentum)和初始速度。" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:20.992077Z", "start_time": "2021-12-22T06:58:20.933414Z" } }, "outputs": [], "source": [ "import mindspore as ms\n", "from mindspore import nn, ops\n", "\n", "class MyMomentum(nn.Optimizer):\n", " \"\"\"定义优化器\"\"\"\n", "\n", " def __init__(self, params, learning_rate, momentum=0.9):\n", " super(MyMomentum, self).__init__(learning_rate, params)\n", " self.moment = ms.Parameter(ms.Tensor(momentum, ms.float32), name=\"moment\")\n", " self.momentum = self.parameters.clone(prefix=\"momentum\", init=\"zeros\")\n", " self.assign = ops.Assign()\n", "\n", " def construct(self, gradients):\n", " \"\"\"construct输入为梯度,在训练中自动传入梯度gradients\"\"\"\n", " lr = self.get_lr()\n", " params = self.parameters # 待更新的权重参数\n", " for i in range(len(params)):\n", " self.assign(self.momentum[i], self.momentum[i] * self.moment + gradients[i])\n", " update = params[i] - self.momentum[i] * lr # 带有动量的SGD算法\n", " self.assign(params[i], update)\n", " return params" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 自定义训练流程\n", "\n", "`mindspore.Model`提供了`train`和`eval`的接口方便用户在训练过程中使用,但此接口无法适用于所有场景,比如多数据多标签场景,在这些场景下用户需自行定义训练过程。\n", "\n", "本节主要使用线性回归的例子来简单介绍自定义训练流程。首先定义损失网络,将前向网络与损失函数连接起来;然后定义训练流程,训练流程一般继承`nn.TrainOneStepCell`,`nn.TrainOneStepCell`封装了损失网络和优化器,用来实现反向传播网络以更新权重参数。\n", "\n", "### 定义损失网络\n", "\n", "定义损失网络`MyWithLossCell`,将前向网络与损失函数连接起来。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:21.023142Z", "start_time": "2021-12-22T06:58:21.008460Z" } }, "outputs": [], "source": [ "class MyWithLossCell(nn.Cell):\n", " \"\"\"定义损失网络\"\"\"\n", "\n", " def __init__(self, backbone, loss_fn):\n", " \"\"\"实例化时传入前向网络和损失函数作为参数\"\"\"\n", " super(MyWithLossCell, self).__init__(auto_prefix=False)\n", " self.backbone = backbone\n", " self.loss_fn = loss_fn\n", "\n", " def construct(self, data, label):\n", " \"\"\"连接前向网络和损失函数\"\"\"\n", " out = self.backbone(data)\n", " return self.loss_fn(out, label)\n", "\n", " def backbone_network(self):\n", " \"\"\"要封装的骨干网络\"\"\"\n", " return self.backbone" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 定义训练流程\n", "\n", "定义训练流程`MyTrainStep`,该类继承`nn.TrainOneStepCell`,`nn.TrainOneStepCell`封装了损失网络和优化器,在执行训练时通过`ops.GradOperation`算子来进行梯度的获取,通过优化器来实现权重的更新。" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:21.040020Z", "start_time": "2021-12-22T06:58:21.025185Z" } }, "outputs": [], "source": [ "class MyTrainStep(nn.TrainOneStepCell):\n", " \"\"\"定义训练流程\"\"\"\n", "\n", " def __init__(self, network, optimizer):\n", " \"\"\"参数初始化\"\"\"\n", " super(MyTrainStep, self).__init__(network, optimizer)\n", " self.grad = ops.GradOperation(get_by_list=True)\n", "\n", " def construct(self, data, label):\n", " \"\"\"构建训练过程\"\"\"\n", " weights = self.weights\n", " loss = self.network(data, label)\n", " grads = self.grad(self.network, weights)(data, label)\n", " return loss, self.optimizer(grads)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "下面定义绘图函数`plot_model_and_datasets`来绘制测试数据、目标函数和网络模型拟合函数,并查看损失值。" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "from IPython import display\n", "import matplotlib.pyplot as plt\n", "import time\n", "\n", "def plot_model_and_datasets(net, data, loss):\n", " weight = net.trainable_params()[0]\n", " bias = net.trainable_params()[1]\n", " x = np.arange(-10, 10, 0.1)\n", " y = x * ms.Tensor(weight).asnumpy()[0][0] + ms.Tensor(bias).asnumpy()[0]\n", " x1, y1 = zip(*data)\n", " x_target = x\n", " y_target = x_target * 2 + 3\n", "\n", " plt.axis([-11, 11, -20, 25])\n", " plt.scatter(x1, y1, color=\"red\", s=5) # 原始数据\n", " plt.plot(x, y, color=\"blue\") # 预测数据\n", " plt.plot(x_target, y_target, color=\"green\") # 拟合函数\n", " plt.title(f\"Loss:{loss}\") # 打印损失值\n", "\n", " plt.show()\n", " time.sleep(0.2)\n", " display.clear_output(wait=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 执行训练\n", "\n", "使用训练数据`ds_train`对训练网络`train_net`进行训练,并可视化训练过程。" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:22.681175Z", "start_time": "2021-12-22T06:58:21.041577Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd4FNXXwPHvSULvLfRioSOiRKSLghRREQR/YFcgdESQoiBiQRAQlE4QBKUISO8dBAJI6KFJxwCG0Dsh2fv+MRPdN24gmGw22ZzP8+zD7k47mSxnb+7cOVeMMSillPJ+Pp4OQCmlVNLQhK+UUqmEJnyllEolNOErpVQqoQlfKaVSCU34SimVSmjCV0qpVEITvkpSInJCROp44Li1ReSgiNwUkbUiUvQe61YQkQ0ickVEwkSkr9OyYiJiROS60+NTp+WTRCQy1nLfeG77mogE2zGucxHXcyKyQ0SuisgxEQmMtTyPiEwTkcsicklEpibwtCkv4+fpAJRyNxHJDcwBWgELgS+BGUDlODaZBswFagHFgI0isssYs8BpnezGmKg4th9kjOlzj5Di2vYi8B1QCngu1s+Qxo6pBxAEBABrRWSrMWa3vdocYBtQFLgJlLtHDCoV0ha+ShZEpLWIHBGRiyKyQEQK2O+LiAwTkXN2i3uPiJSzl70gIvtF5JqInBaRj+LYfRNgnzFmljHmNtAPeFxESsWxfjFgqjEm2hhzFNgIlE3Mn9cVY8wqY8xM4IyLxTmBrMDPxrINOACUARCRukBhoLsx5oox5q4xZqe7Y1YpiyZ85XEi8hwwAHgNyA+cBH6xF9cFagIlgOzA/4AL9rIJQBtjTBas1uwap31eFpHq9suyQEwrGGPMDeAocSfx74C3RSSNiJQEqgCrYq1z0u7u+dH+C8JZe/uLa7uIvOpi//fa1iVjTDgwHXhPRHxFpApWS36jvUpl4BAwWUQuiMg2EXkmPvtWqYcmfJUcvAFMNMbsMMbcAT4GqohIMeAukAWrm0OMMQeMMWft7e4CZUQkqzHmkjFmR8wOjTHZjTExyTAzcCXWMa/Y+3VlEdAUuAUcBCbYLWqA88BTWMm2or0P577y4UBxwB/4FJgkItXiue39TAf6AneADUBvY8yf9rJCWF+Oa4F8wLfA/Ph+oajUQRO+Sg4KYLXqATDGXMdqxRc0xqwBRgKjgHARCRKRrPaqrwIvYLWY19utXleuY3WHOMsKXIu9oojkBJYBXwDpsbpJ6olI+5jYjDEhxpgou9XdEagbE5P9pXXBXr4EK6E3ic+292J3P80A3gbSYv110kNEGtqr3AJOGGMm2N05vwB/AtVc7lClSprwVXJwBqvVC4CIZAJyAacBjDHDjTEVsZJcCaC7/f42Y0wjrNb0PGBmHPvfBzwea/+P2O/H9jAQbYz5yU7MYVjdSy/Ese+YcrNyj+X3WnavbZ2VAw4ZY5YbYxzGmEPAYqCBvXyP0/6UckkTvvKENCKSPuaBlajfs4dDpgO+BrYaY06IyFMi8rQ9SuUGcBuIFpG0IvKGiGQzxtwFrgLRcRxvLlBORF61j9cX2GOMOehi3T+wrhW/LiI+IpIP67rBbqwFT4tISXtZLqwunHXGmCv28qYiktleXhd4E1gQz2197fj8AB/7/KSx49oJFLeHZoqIPAK8yD/XJuYCOUTkHXs/TYGCwKYH+s0o72aM0Yc+kuwBnMBqiTo/vgLaYl1IvYjVh17IXr82Vuv1OlYf+FSsPvm0WF0vl7CS/TagutNxrgM1nF7XweqPvwWsA4o5LRsLjHV6/Zy9vyvAX8B4IKO9rAVwHOvL5yzwE5DPadsN9nZXsZJxc6dl99v2XRfnZpLT8teAUKyuqDDgG8DHaXkNYK/9s4c4//z60IcxBjFG/wpUSqnUQLt0lFIqlUhwwheRwmLdqn5ARPaJyAf2+/3sm2F22Y+4LnoppZRKAgnu0hGR/EB+Y8wOEckCbAdewepvvG6MGZLwMJVSSiVUgmvpGOsmmLP282sicgBrdIBSSqlkJFEv2tp3Rv6GNWa4K9aog6tYIwa6GWMuudgmEAgEyJQpU8VSpeIqb6KUUsqV7du3nzfG5LnfeomW8EUkM7Ae6G+MmSMiebGG0Rms6oT5jTHv32sfAQEBJiQkJFHiUUqp1EJEthtjAu63XqKM0rFvDpmNVWFwDljFnoxVbdCBNY65UmIcSyml1H+TGKN0BKtq4QFjzFCn9/M7rdYY64YRpZRSHpIYE6BUA94C9orILvu9T4AWIlIBq0vnBNAmEY6llFLqP0qMUTobcV38aUlC962UUirx6J22SimVSmjCV0qpVEITvlJKpRKa8JVSyoOSsmKxJnyllPIAYwwzQmfwZNCTXL59OUmOqQlfKaWS2Kkrp3hp+ks0n90cX/Hlws0LSXLcxBiHr5RSyuGAiAjw9wdxPU1xtCOakb+PpPea3hgMQ+sOpdPTnfDzSZpUrAlfKaUSyuGAZ5+F4GCoWhXWrgWf/9+Bsid8D60Xtub3079T/9H6jGk4hmLZiyVpmNqlo5RSCRURYSX7qCjr34iIvxfdunuLj1d9TMWgihy/dJxpTaax5PUlSZ7sQRO+UkolnL+/1bL387P+9fcHYPWx1Tw25jEGbhrIW+Xf4mDHg7R4rAUigjEwfz5Urw7XrydNmJrwlVIqoUSsbpywMFi3jgu3LvLuvHep83MdRITVb69mYqOJ5MyQE4CjR+HFF+GVV+DyZThzJmnC1D58pZRKDD4+GH9/podOp8uyLly6fYlPqn9Cn5p9yJAmAwC3bsHAgfDNN5A2LQwdCh07Qpo0SROiJnyllHpQLkbknLh8gnaL27HsyDIqFazEqpdWUT5v+b83WbgQOneGEyfg9ddh8GAoUCBpw9YuHaWUcsXhgPBwiH0nbMyInEKFoFYtoqIi+Tb4W8qOLsvGUxsZXn84we8H/53sjx2Dl16Cl1+GjBmtnp+pU5M+2YO28JVS6t/uNczSaUTOzqObaB30FNsj9vBiiRcZ/cJoCmcrDFjdN4MGwYABVpfNkCFWCz+pum9cSXDCF5HCwE9APsABBBljvheRnMAMoBjWBCivuZrEXCmlkh1Xwyzz5rWW+ftzs/rTfJY2mGGVo8l9M5yZTWfStExTxO7eWbzYSu7HjkHz5layL1jQgz+PLTG6dKKAbsaY0kBloIOIlAF6AauNMcWB1fZrpZRK/uIYZgmw4thKyr1yhiFVDe/vEg4sfZhmpV9FRDh+HBo1skbgpEsHq1fD9OnJI9lD4sx4dRY4az+/JiIHgIJAI6CWvdpkYB3QM6HHU0opt4sZZul0YTbiRgRdV3Rlyp4plMj2COt+8uWZY9Hgt43bf0YwaHJeBgwAX1+rK+eDD6yROMlJovbhi0gx4AlgK5DX/jLAGHNWRPzvsalSSiUvPj6QNy/GGH7e/RNdl3fl6p2rfFrzUz6p/jHp59aHU8EsLfUhnWr7c/QovPYafPutdT03OUq0hC8imYHZQBdjzFWJo3iQi+0CgUCAIkWKJFY4SimVYEcvHqXt4rasOraKqoWrEvRiEGX9ywJw4se1fNgxknlL01OyJKxcCXXqeDjg+0iUYZkikgYr2U81xsyx3w4Xkfz28vzAOVfbGmOCjDEBxpiAPHnyJEY4SimVIHej7zJo0yAeG/MYW8O2MvqF0Wx4bwNl/cty5w707w9lyvmwYn16Bg6EPXuSf7KHxBmlI8AE4IAxZqjTogXAO8BA+9/5CT2WUkq5W8iZEFotaMXu8N28UuoVRjYYScGs1lXXZcugUyc4cgSaNrXulC1c2MMBP4DEaOFXA94CnhORXfbjBaxE/7yIHAaet18rpVSydD3yOl2Xd+XpH57m3I1zzHltDnP/N5eCWQty8iQ0aQINGlhd+8uXw6xZKSvZQ+KM0tkIxNVhXzuh+1dKKXdbengp7Ra34+SVk7QLaMeA2gPIlj4bd+5YF2G/+spa7+uvoWtXa8hlSqR32iqlUq3w6+F0Wd6FX0J/oXTu0mx8byPVilQDYMUKq7DZ4cNW637YMEjp40o04SulUh1jDD/u+pGPVnzEjbs3+LzW5/Ss1pN0fun480/48EOYPRuKF7f67evV83TEiUMTvlIqVTl84TCBiwJZd2IdNYrUIOilIErlLkVkpFW6+MsvrXpp/ftDt24pt/vGFU34SqlUITI6kiHBQ/hi/Rek90tP0ItBtHyyJT7iw6pVVvfNoUPQuLHVfVO0qKcjTnya8JVSXm9r2FZaLWxF6LlQmpZpyvD6w8mfJT9hYdZF2Fmz4JFHYMkSaySOt9J6+Eopr3XtzjU6L+1MlQlVuHz7MvObz2dWs1nkSpefQYOgVClrYpIvvoDQUO9O9qAtfKWUl1p4aCHtl7Tn9NXTdHiqA/1r9ydruqysXm113xw8aFW2HDYMHnrI09EmDW3hK6W8ytlrZ3lt1mu8/MvLZE+fneCWwYx4YQTXzmeleXOrBEJkJCxaBPPmpZ5kD9rCV0p5CYdx8MOOH+ixsge3o27T/7n+fFT1I8SRliFD4PPPrflMPv8cevSA9Ok9HXHS04SvlErxDp4/SODCQDac2kCtYrUY9+I4SuQqwdq1VvfN/v3WpCTffw8PP+zpaD1Hu3SUUilWZHQkX6z/gsfHPk7ouVAm1BrKmrdWk/lOCV5/HZ57zppbdsEC6+Jsak72oC18pVQKtenUJgIXBbI/Yj/Ny/6P78aeJGfvTxhWNC2fnWvP3bvCZ59Bz56QIYOno00etIWvlEoeHA4ID7duc72HK7ev0H5xe6r/WJ3rkddZ/Ppiptf4noPrMvBE9Da6HetAzacj2bcP+vXTZO9ME75SyvMcDnj2WWtuwFq1rNcuzD0wlzKjyzBu+zi6PN2Ffe338UTmF3izmz+1otdwnczML/sxi1ak5ZFHkvZHSAk04SulPC8iAoKDrWE0wcHWayenr56myYwmNJnZhDwZ87Cl5RYG1xnGD6MzU7IkzJolfNrHsP9YBl7e+zXiE78pVlMb7cNXSnmevz9UrWol+6pVrddYQy3HhYyj1+peREZHMrD2QLpW6cqW4DQ82QD27oX6z95hxLi0PFpcgLye/TmSucSa03aiiJwTkVCn9/qJyOlYs2AppdS/icDatRAWBuvWgQj7zu2jxo81aL+kPZUKViK0XSjvPNqTlu+loWZNuHLFMLdMb5b8lplHW9WKsxtI/SOxunQmAfVdvD/MGFPBfixJpGMppbyRjw/kzcvt6Dv0XduXJ8Y9waHzh5j8ymSWNF/B4imPULIkzJgBvXvDgXXneOWPQUi0624g9W+J0qVjjPlNRIolxr6UUqnXb8fXEbigNYcuH+HN8m8ytO5QDu3MQ0AA7NkDdevCiBFQogRgXHcDqbi5+6JtRxHZY3f55HC1gogEikiIiIRE6De0UqmPw8HlU38QuKA1z/z0LHdOHGHZrnIMqTyZ7h3yUKMGXLpkzUC1bJmd7MFlN5C6N3cm/DHAI0AF4CzwrauVjDFBxpgAY0xAnjx53BiOUiq5MdHRzHqtHKWHl2TCjh/4aLOwa5QPhxc8R8lSMG0afPwxHDhgzSv7r5xudwNpso8ft43SMcaExzwXkfHAIncdSymV8vx55U86zG3FwscO8OQZWPyLD7dztKTW3fbsogLPP20YMQJKlvR0pN7DbQlfRPIbY87aLxsDofdaXymVOkQ7ohm9bTSfrPkEh3Ew5PDDtJhxkz65JvBj6AsUynuXWcMdvNrMRxvuiSyxhmVOBzYDJUUkTERaAoNEZK+I7AGeBT5MjGMppVKuveF7qTaxGp2XdaZa4WrsDgwlQ5XDlM10mp8vNKBn4akcOJ+HpqOeRYwOs0xsiTVKp4WLtyckxr6VUinfrbu3+PK3LxkcPJgc6XMwtclUHrregtfqCjt3Qu3aMOKz85R+7l1wHmaZV2+kSkxaWkEp5VZrj6+l/NjyDNg4gDfLv8mG5gdY8/3rVK0qhIdb4+pXroTS1XNZwyv9/HSYpZtoaQWllFtcvHWRj1Z8xI+7fuSRHI+w/I1VHF1ZmyrvwbVr0L07fPopZMkSs4U9zDIiwkr22oGf6DThK6USlTGGGftm8MGyD7hw8wK9qvWiQaa+dG2ege3braKYI0dCmTIuNo4ZZqncQrt0lFKJ5uTlkzSc1pAWs1tQNFtRVtVZyYUZX1OrWgbOnIHp02H16jiSvXI7beErpR6Mw/GvbpdoRzTDtw6nz9o+CMLQOsPI8H4kr/5RnqsSTdcPffmsnzh13yhP0Ba+Uir+XExUsuuvXVSeUJmuK7pSq1gtplXfx/SOHWj3Rw8eYy+7fCoypMc5TfbJgCZ8pVT8OU1UcvP3TfRc2JmAoAD+vPInP9SbQcH1i3ilVlH+/MuPqaW+ZK3v85Stll1H3CQT2qWjlIo/e6KSlWc30rZJGo7tGkXLJ1pR7uwgejTMwZUr8OGH8NlnQtbMvSEiUEfcJCOa8JVS8Xb+1gW6flCEn/c6KJGrCGOLj2Ni31pM+B1q1oRRo6BcuZi1dcRNcqMJXyl1X8YYpu6dyofLP+Ty7ct0e6oPVxb2pl3n9Pj7w88/wxtvaEM+udOEr5S6p2OXjtFucTtWHF1B5YKVaRA1nhHvlOPiRejcGT7/HLJl83SUKj70oq1SyqUoRxRDgodQbnQ5Nv+5mZ7lRuL4YSOftStHqVKwcyd8950m+5REW/hKqX/ZfmY7rRe2ZudfO2nw0Mvk2TaKQb0LkScP/PQTvPmmdt+kRJrwlVJ/uxF5g75r+/Ld1u/wz+RP+9y/MvOjJly8IHTqZHXfZM/u6SjVf6UJXykFwLIjy2i7qC0nr5zk1aJtODVxIKM3ZKdqVRi1AipU8HSEKqESawKUiSJyTkRCnd7LKSIrReSw/a/LScyVUp517sY53pjzBg2mNiCdTwaaXN7A3JZjOXEwO5MmwYYNmuy9RWJdtJ0E1I/1Xi9gtTGmOLDafq2USiaMMUzaNYnSo0oza98sXsnej0sDdjFveHXat4dDh+Cdd6wClso7JNaMV7+JSLFYbzcCatnPJwPrgJ6JcTyl1H24KHDm7MjFI7RZ1IY1x9dQIWc1ZPV45q0oTZUqsHwJPPGEB2JWbufO7+68MZOY2/+6LKYhIoEiEiIiIREREW4MR6lUInaBs6goCA8HY7gbfZeBGwfy2JjHCDkdwrM3xrK7y2/8uaM0EyfCxo2a7L2Zxy/aGmOCgCCAgIAA4+FwlEr5nAqcERwMNWpASAi/1ytH64YO9pzbQ0DGJpwYM4J1JwrQrh18+SXkzOnpwJW7ubOFHy4i+QHsf8+58VhKqRh2gTP8/OCpp7i2+3e61ImicsAu/rocQentMwnpMZtH/AuwbZtV/0aTfergzhb+AuAdYKD973w3HkspFUP+mRt20aWttJ/8GmHp7lB+fz32LphG9B0HPxT/hvc2dsfHT6/IpiaJNSxzOrAZKCkiYSLSEivRPy8ih4Hn7ddKqcTicPzdNx/bXzfP8b8NnXlpRiNMlkfJMWcDe35dSmDkLP6gBC2P98Hngl4zS20Sa5ROizgW1U6M/SulYom5MBscbHXfrF0LPj4YY5iwcwLdV3bnRuRNih77kpNTe/DUk2kZvdUQ8NE0CL5mbaOTkqQ6Hr9oq5T6D2JfmD13jkN+V2izqA3rT66nUNQzXA0ax7WokgSNgZYtwcfnn64enZQkddKEr1RKFHNhdsMGIk0Ug3pU5qsSf+FrMpBt/XjC1r1PYGsfvv4acuVy2s5HJyVJzTThK5USicAvv7C5ciFav2jY53+SPKebEjFlBAGl8jFqC1Sq5OkgVXKjCV+pFOjqnat8svMrRr/nIPPV7PhMm0RUxMuMHSK0agW+vp6OUCVHmvCVSmHmHZxHxyUdOXPtDBlDO3FtYX9avZWZAQOF3Lk9HZ1KzjThK5VCnLl2hk5LOzHnwBwyXy+PmT6HknkrMWo9VK7s6ehUSqAJX6lkzmEcBG0PoufKnty8E4nP2gH4hnZj9FdpCAzU7hsVf5rwlUrGDkQcIHBhIBv/3Ei6088RNXsc77/yKAN/hTx5PB2dSmk04SuVDN2JusOAjQP4esPXcCcLLP6RUrzDmEVClSqejk6lVJrwlUpmNp7aSMv5rfnj4kEk9HUybxzGsD7+tG2r3TcqYTThK5VMXL59mZ6rehG0fRy+14rC/CW8U70B3+zWKggqcWjCV8rDjDHMOTCHdos6EXEzHDZ3pfT5Lxg7KRPVqnk6OuVNNOEr5UFhV8Nou6ADi48ugL+eINOqhQzoXJF27axy9kolJv1IKeUB0Y5oRm8bQ88Vn3A7MgrWDObNR7swZKOflrpRbqMJX6kkFnoulLdmtmbXhS1w9HlKHh7LD4Mfpnp1T0emvJ3bE76InACuAdFAlDEmwN3HVMpjHI44yw/fjrrNZ6u/Ysjmb3Dcyk769T8zsMUbdJgo2n2jkkRSfcyeNcacT6JjKeUZcUxKArDu+HremBHImTt/wO63aZbtW4bPy02+fB6OWaUq2q5QKrHEnpQkIoJLWdPSZnZ3Zh2dABcfpljoCiZ/9jw1a3o6WJUaJcUMxgZYISLbRSQw9kIRCRSREBEJiYjQOTZVChYzKYmfH6ZqFX4+vZbC35Rm1uFJpP29BwOL7uWPZZrsleeIcTEBcqIeQKSAMeaMiPgDK4FOxpjfXK0bEBBgQkJC3BqPUm7lcHDq2E5eXfgZIVcXw5mKNLgbxIQuhcn/WG6dVlC5hYhsj8/1Ube38I0xZ+x/zwFzAZ2HR3mlaEc0ny4dwcM/PUPI+XXk2zWMVf8LZsnRD8lfsQDUqmX18yvlIW5N+CKSSUSyxDwH6gKh7jymUp6w9eRuinxRha9CusDJmvTOsY9TM7tQu+ylf/XrK+Up7r5omxeYK9afsX7ANGPMMjcfU6kkc+vuLd6e+AW/nhkMN3NR9ep0Zn79PwoWtLtuYvr1Y0buaFEc5UFuTfjGmGPA4+48hlKeMiV4NW0WtuFm+qNkO/E+k98YTKO6Of//SiLW8Mw4xuYrlZR0WKZSD+j0pQu8OLwbu5iM3HqUNtnXMHz8s6RNG8cGPj5ovQSVHGjCVyqejDH0mDKNofu74EhzmdKXPmF+tz4UL5bB06EpFS+a8JWKhw17j9N0UjvOZV1O+huVGF5nPK1fLu/psJR6IJrwlbqH6zejaDbke5bd6QvpfXg5zXB+GdSeDOl16imV8mjCVyoOI2fv4KP1rbmTawf5b7/E3FajeLp0YU+HpdR/pglfqVgOHLnBy8P6cST3MHwz5qFPiVl80fxVREfYqBROE75KPe5Ruhjgzh1oO3gFky+0xfgf5ymf1izo+g35suXwQLBKJb6kKJ6mlOfFlC4uVMhliYOZiyPI0+YtJkXXI1P6tMxssJ7fPw3SZK+8irbwlfdybtG7KF1M3rycPGlo+uXPhOTqCkWu8kbBvvzw7sek90vv6eiVSnTawlfeKXaLPnfuv0sXU7Uqkdn96T7gKA/3q0tI4XcokqkkOwJ3MqXV55rsldfSFr7yTrFb9OfP/13iYNmuHLzVfBDny/bDr3AaPq88mk/qtsFHtP2jvJsmfOWdXBQtCzstvPvJKVZnqgcVdlM1R2NmvjuCglkLejpapZKEJnzlnZyKlkVm9+ebQTf4YuOnRD05nCw++RjfeA7/K9/Y01EqlaQ04Svv5ePD6tC8vNt/CWGPt4OAU7xZsh0jXxlAtvTZPB2dUklOE77ySqdPQ4ce4cy/0wWe+YUi6cswrcVGqhWp9s9K9xmXr5S3cftVKhGpLyKHROSIiPRy9/FU6nb3LgwebHj41YnML1wa38fm8Gn1z/mj245/J/t7jMtXyhu5tYUvIr7AKOB5IAzYJiILjDH73XlclTqtXQute/3B0dJtoME6nvKvwU/NgiiVu9S/V45jXL5S3szdLfxKwBFjzDFjTCTwC9DIzcdUqcyZM/C/1yN57rP+HKtXnkyP7iToxSC2tF3nOtnDP6N47HH5OvWgSg3c3YdfEPjT6XUY8LTzCiISCAQCFClSxM3hKG9y946DEQNv0GdqKLefD4SSoTQu2YyRDb8nf5b8/3/l2P31OvWgSoXc3cJ39b/I/L8XxgQZYwKMMQF58uRxczjKW6xf66B8vlC6bf2EW69XI2+xyyxovoDZzWe6Tvau+utjph7UZK9SCXe38MMA5wLihYAzbj6m8mJnz0L37jA1ZBG+77VFsp6lwzbh67HryFL4EdcbaX+9UoD7W/jbgOIi8pCIpAWaAwvcfEzlhaKi4LvvoPiTZ5kW1QxaNKKkzzWCJ/ky4noNshR6OO6Ntb9eKcDNLXxjTJSIdASWA77ARGPMPnceU3mfDRugfQcHoWl/wK9VD9Kkvc1ntfrTvXI30nS/fP8+eO2vVwpIghuvjDFLgCXuPo7yPn/9BT16wM9LD5KuaSDk20CNYs8y7sVxFM9V3Fopvl0zMf31SqViWh5QJTtRUfD991CizB2mnf4C3w6Pk7FYKBNensDqt1f/k+yVUg9ESyuoZGXjRujQAfZc2kSm1q2JzniA5uWa812978ibWVvoSiWEJnyVLISHQ8+eMPmXK2R+pReUHkuubEWY2XAxLxR/wdPhKeUVNOErj4qKgrFjoU8fuF54Lpl7deSmz198+PSHfPHsF2ROm9nTISrlNTThK48JDra6b3YdPU2edzsSnWsej+R9nPEvzeOpgk95OjylvI5etFXu53BYfTbGusn63Dl47z2oVt3BiTyjydC9NNfyLuObOt+wrfU2TfZKuYm28JV7xZQ1CA4mukp1xtafR5/BWbmeYT8FPw3ktE8wdYrVYWzDsTySM447ZZVSiUITvnIvu6zB5qgAOmz4lp2b0/FQndbcqPITt9NnZXK9ybxV/i1Eb4ZSyu004Su3ihB/euWax8TwhuQu+isFXqrF8dzXePPRpgxtNJo8mbRgnlJJRfvwVcLF6qMHiI6GMWOgRElh8o0qlO/5Luffa0Y632ss31WOn1vM1GSvVBLThK8SxkXp4a1boVIlaN/eUPD5WWTvU4Z9GafQvcpH7P3oKHXn7NF6Nkp5gHbpqIRxKj18ftMhPn7tMj/Mzkne4n/y5OD27Li42LTPAAAWWUlEQVSxiCdzPMnKl5byRP4nPB2tUqmatvBVwvj7E12lOuN82lHSsZ8f52TkubrvcuPdMhyMXMO3db9la6utmuyVSga0ha8SZFuI0P7WGkIcQoD/BCJf7sSaQreol+9ZxjSewEM5HvJ0iEopm7bw1X9y4QK0aQNPPw1hf93i5WY12NWmFWdz3GJqaEmWvrtKk71SyYzbWvgi0g9oDUTYb31i18ZXKZjDARMmQK9ecOUKvPrRGnbkbcWC68d5dycMWQG5jqyx6s8rpZIVd3fpDDPGDHHzMVQSCQmxat/8/jtUee4Ced/qzq8nf+QRU5hVP0HtY1ijbzTZK5Us6f9MdV8XL0K7dtZQy5OnDO3HTOdI/dIsPPUTvU4UZm+f09Q+n9WaM7ZGDZ1ZSqlkyt0Jv6OI7BGRiSKSw9UKIhIoIiEiEhIREeFqFZWUnG6iium+KVECxo+Hdz88QfmBDRkd/jrFshdjR9OVDJhylgx3HHDjBuzcCevW6Rh7pZKpBCV8EVklIqEuHo2AMcAjQAXgLPCtq30YY4KMMQHGmIA8efTOS49yuolqR8XWVK1qaNUKSpWJ4sOZw5iRqyzBp3/j+/rfs7nlZsqXrgVVq1ot+2rVoGxZTfZKJWMJ6sM3xtSJz3oiMh5YlJBjqUTkcFg3TPn7Wwk65rUxXNq0nz7R3zFmZzv88zj4cvxe5jlaMWTvdhoWb8johqMpkq3IP/tau/b/70splWy5c5ROfmPMWftlYyDUXcdSD8CpXDFVq8Lq1VC7No5Nm5n8yBf0kMNcJAttC08nzejd9NsxlNwZczOj6QyalWn276qWPj7aZ69UCuHOUTqDRKQCYIATQBs3HkvFl1MpBIKD4eBBdm66SYfodWz+oyrVnorkzZ7zGHy6L8e2H6PVE60Y9PwgcmRweQlGKZWCuC3hG2Pecte+VQL4+1st++BgLleqy6djyjA6egu5uMCInO+ytWc07UKnUCJXCda+s5ZaxWp5OmKlVCLR0gqpjQiO1Wv5efQ1un+VlQtboJ2MotxjPelT/xZX9vvRp0YfetfsTXq/9J6OVimViDThpzK7d0OHDj5s2pSNKlVg4pyjDJ/Vl1E5b1HlSlaCAjdSLu9jng5TKeUGmvBTiStXoG9fGDkScuaE8ROiuFhiGK+t/wy/fH6MrPQ17Z7tgY+Pr6dDVUq5iSZ8L2cMTJkC3btb12vbtoWmnbfTdW0rdq3eRaOSjRj5wkgKZS3k6VCVUm6mpRW82J49ULMmvP02FCsGv22+QfqXu1FnRiXCT+1n9iwf5o65SKHMBTwdqlIqCWgL3wtduQL9+sGIEZA9O/zwA+SvsYw3lrTl5JWTtCnzNgPfm0r2Gw7w22w1/XUsvVJeT1v4XsQYmDoVSpWC77+HwEDYtOsca7K/TsPpDciQJgMb3tvA2KaTyF6xmlUSoWpVa6imUsrraQvfS4SGWqWLf/vNqmq5YIEh1G8SVaZ24/rNy/Tb4EMvR27StatqfTPEPCIjrbtvffVirVLeTlv4KdzVq9CtG1SoAPv2QVAQ/Lz4CL0O1OH9Be9TNnsJdo/z4bO1DtJt2mJ130REwObNEB0NW7ZA9epW0ldKeTVN+CmUMTBtmtV9M2wYtGwJofvvcr7UAB4f9xghp7cxtuEY1rfeROmSsbpv/P3hqaf+2dm2bdaXgFLKq2mXTnIXu7IlVku+Y0er9HxAAMybB6bAVurObs3ec3t59Vxuhk+7RIFl02FtoOuKlhs2WC37bdus0sbaj6+U19MWfnLmVJ+eWrW4dsVB9+5W982ePTBuHKxcf42pFz6gyoQqXLx1kXn1JvFr0GUKXI62iqNFRPxT0dK50qWvL2zaBKdP66QlSqUS2sJPzuzKliYqipkbC9K1lOHMX9C6NXz9NWy5uIjy49oTdjWM9k+15+vaX5M1bRaoOvGf8sf3arlraWOlUhVN+MmZvz8HKrSg4/Z3WeN4jooFDXPmQdGyf9Fh2QfM3DeTsnnKsun9TVQpXOWf7XRSEqWUC9ql42lOc8g6u34devQUyu+azM5stRgz2rB5i2GP33hKjyrNvIPz+PLZL9nRZsf/T/bgugtHKZXqJXRO22Yisk9EHCISEGvZxyJyREQOiUi9hIXppWL10eNwYAzMnGmNvhk8GN55Rzj0hw/PNvuDOlOeJXBRII/nfZw9bffQp2Yf0vqm9fRPoZRKIRLawg8FmgC/Ob8pImWA5kBZoD4wWkT0zp7YYs0+dTD4InXrwv/+Z/XGBAfD6HGRjN3/JeXHlmdP+B5+eOkH1r6zlpI5i7v8y0AppeKSoIRvjDlgjDnkYlEj4BdjzB1jzHHgCFApIcfySvbsU9d9s9Er/yTKP5eLkBAYNcoaLUmhzTw57kn6rutL41KNOdDhAC2fbIkY86+/DJRS6n7cddG2ILDF6XWY/d6/iEggEAhQpEgRN4WTPBmE2R3W8uERQ9ifvrzX/BYDv0tP+uzX6LzsY8aEjKFQ1kIsarGIhiUa/rNh7HlptfiZUioe7pvwRWQVkM/Fot7GmPlxbebiPZd9D8aYICAIICAgINX0Txw6BJ06wcqVPlSoYJiRpz1Vfx3PvMgSdKx2hTPXztD56c589dxXZE6b+f9v7DQvrRY/U0rF130TvjGmzn/YbxhQ2Ol1IeDMf9iP17lxA/r3hyFDIGNGq4Rx28bnOFcuiFebRDOnzH7Kpy3DnFZzqFQwjl4wER16qZR6YO4alrkAaC4i6UTkIaA48LubjpUiGANz5kDp0jBgALz+utXKb9/BwQ9n5lC6IywpDgOOFiOk7c64k30MHXqplHpACR2W2VhEwoAqwGIRWQ5gjNkHzAT2A8uADsaY6IQGm1IdPgwNGsCrr0KOHFYZm0mT4ILPfmr+WJN2S9oT8Ogz7H17M70mHyONnw61VEolvgRdtDXGzAXmxrGsP9A/IftP6W7etEogDB4M6dNbk5K0bw/R3KHfugF8veFrsqTLwqRGk3j78bcRba0rpdxISyu4gTEwfz506QInT8Jbb8GgQZAvH2w4uYHARYEcPH+QNx57g6H1huKfSS+6KqXcT0srJLIjR6BhQ2jcGLJkgfXr4aefIH32y7RZ2Iaak2pyO+o2S99YypQmUzTZK6WSjLbwE8nNmzBwIHzzDaRLZ01K0qED+PkZZu+fQ6elnQi/EU63Kt34vNbnZEqbydMhK6VSGU34CWQMLFwIH3wAJ07AG29Yffb580PY1TA6LOnAgkMLeCLfEyxssZCKBSp6OmSlVCqlXToJcPQovPQSNGoEmTJZ84hMmQL+eaMZ+ftIyowqw8qjKxn8/GB+b7mFir6FtPaNUspjNOH/B7duQb9+ULas1Uf/7bewcyc88wzsDd9L9R+r02lpJyoXqkxo+1A+qtwVv9rPa+0bpZRHaZfOA1q0CDp3huPHoUUL647ZAgXgdtRtPl/zFd9s+obs6bMzpfEUXn/sdWuoZXi41r5RSnmcJvx4On7c6qdfuNC6W3bNGqtgJcC6E+sIXBjI4YuHefvxt/m27rfkzpj7n4219o1SKhnQhH8ft29bY+gHDAA/P+uC7AcfQJo0cPHWRXqs7MGEnRN4OMfDrHxrJXUedlF6SGvfKKWSAU3497BkiVXR8tgxa1KSb7+FggXBGMOM0Jl0XtaZCzcv0LNaT/o+05eMaTLGvTOdMFwp5WGa8F04ccK6S3b+fGuqwVWroHZta9mpK6dov7g9iw8vJqBAAMvfXE6FfBU8Gq9SSsWHJnwnt29bF2H79wdfX+smqi5dIG1aiHZYQy17r+kNwLB6w+hUqRO+Pjpzo1IqZdCEb1u2zOq+OXIEmjWzum8K2xX9d/+1m9YLW7PtzDYaPNqAMQ3HUDR7Uc8GrJRSDyjVj8M/eRKaNLHKF/v6wsrlDmaOCKdwIcOtu7fotaoXFYMqcvLKSaa/Op3Fry/WZK+USpFSbQv/zh2rFf/VV9agmYED4cMPHKSt9ywEB7OqYWnaPneTo5eO8n6F9xlcdzA5M+T0dNhKKfWfJXQClGYisk9EHCIS4PR+MRG5JSK77MfYhIeaeJYvh8ceg969rcqWBw5Az56Q9koEF3Zu4t0Xo3j+ib34OAxr3l7DhEYTNNkrpVK8hLbwQ4EmwDgXy44aY5LV8JVTp6BrV5g9G4oXtxJ/3brWMmMM08JX0qWTD5d9o/nkRGH6jNtLhrT3GGqplFIpSEJnvDoAJPuZmiIjYehQ+PJLq3ZZ//7QrZtVxhjg+KXjtFvcjuVHl/P0Q08zvupAHiv9jN4gpZTyKu7sw39IRHYCV4E+xpgNbjxWnFautEbfHDpkXZwdOhSK2tdcoxxRfL/le/qu64uP+DCiwQjaBbTToZZKKa9034QvIquAfC4W9TbGzI9js7NAEWPMBRGpCMwTkbLGmKsu9h8IBAIUKVIk/pHfR9gpB107RjJrYXoefRSWLoX69f9ZvuPsDlovbM2Oszt4qcRLjHphFIWzFU604yulVHJz34RvjHFRHOa+29wB7tjPt4vIUaAEEOJi3SAgCCAgICDBxeIjI+G7YQ6++OQ2Dgd8VWw83Xa3JH1G6/r0jcgbfLbuM4ZtGYZ/Jn9mNZvFq6VfTfbdUkoplVBu6dIRkTzARWNMtIg8DBQHjrnjWM5Wr4aOHeHgQR9eYTnD+JBiJ0/BlRchY36WH1lO28VtOXH5BIFPBjKwzkByZMjh7rCUUipZSOiwzMYiEgZUARaLyHJ7UU1gj4jsBn4F2hpjLiYs1LidOQPNm0OdOnD3LiyecpG5NKEYJ8EYIm5f4M05b1J/an3S+aZj/bvrGffSOE32SqlURUwymnIvICDAhIT8q9fnvkJDrTLz3btbj/TpDDzzDCZ4Ez81eZSuAee5ducaH1f/mE9qfEI6v3Sud+RwaAljpVSKIyLbjTEB91vPK+60LVcO/vwTsmWLeUc4OmcCbea1ZPXpDVTNXZXxL42nTJ4yce/E4bBmNImZpGTtWquksVJKeQmvSPjwT7K/G32XoZuH0m99P9L6pmX0C6NpE9AGH7lP8o6I0GkIlVJezWsSPsC209tovbA1u8N307hUY0Y0GEHBrAXjt7FOQ6iU8nJekfCvR17n0zWfMvz34eTLnI85r82hcenGD7YTnYZQKeXlvCLh7wnfw/Dfh9OmYhsG1B5AtvTZ7r+RKzoNoVLKi3lFwq9auCpHOh3hoRwPeToUpZRKtrxmGIome6WUujevSfhxcjggPNwqk6mUUqmYdyf8mLH1hQpBrVrWa6WUSqW8O+G7GluvlFKplHcn/Jix9X5+OrZeKZXqecUonTjp2HqllPqbdyd80LH1Sill8+4uHaWUUn/ThK+UUqmEJnyllEolEjrj1WAROSgie0Rkrohkd1r2sYgcEZFDIlIv4aEqpZRKiIS28FcC5Ywx5YE/gI8BRKQM0BwoC9QHRouIbwKPpZRSKgESlPCNMSuMMVH2yy1AIft5I+AXY8wdY8xx4AhQKSHHUkoplTCJOSzzfWCG/bwg1hdAjDD7vX8RkUAg0H55XUQOJSCG3MD5BGzvLhrXg9G4HozG9WC8Ma6i8VnpvglfRFYB+Vws6m2MmW+v0xuIAqbGbOZifZfVy4wxQUBQfIK9HxEJic9EvklN43owGteD0bgeTGqO674J3xhT517LReQd4EWgtjF/l6QMAwo7rVYIOPNfg1RKKZVwCR2lUx/oCbxsjLnptGgB0FxE0onIQ0Bx4PeEHEsppVTCJLQPfySQDlgpVp2aLcaYtsaYfSIyE9iP1dXTwRgTncBjxUeidA25gcb1YDSuB6NxPZhUG5cYnRhEKaVSBb3TVimlUglN+EoplUqkqIQvIs1EZJ+IOEQkINay+5ZyEJGHRGSriBwWkRkiktZNcc4QkV3244SI7IpjvRMistdeL8QdscQ6Xj8ROe0U2wtxrFffPo9HRKRXEsQVZ4mOWOu5/Xzd72e3ByLMsJdvFZFi7ojDxXELi8haETlg/x/4wMU6tUTkitPvt28SxXbP34tYhtvnbI+IPJkEMZV0Og+7ROSqiHSJtU6SnC8RmSgi50Qk1Om9nCKy0s5FK0UkRxzbvmOvc9geEZkwxpgU8wBKAyWBdUCA0/tlgN1YF5AfAo4Cvi62nwk0t5+PBdolQczfAn3jWHYCyJ2E568f8NF91vG1z9/DQFr7vJZxc1x1AT/7+TfAN544X/H52YH2wFj7eXNgRhL97vIDT9rPs2CVMokdWy1gUVJ9nuL7ewFeAJZi3Z9TGdiaxPH5An8BRT1xvoCawJNAqNN7g4Be9vNerj7zQE7gmP1vDvt5joTEkqJa+MaYA8YYV3fi3reUg1jDiJ4DfrXfmgy84s547WO+Bkx353ESWSXgiDHmmDEmEvgF6/y6jYm7REdSi8/P3gjrswPWZ6m2/Xt2K2PMWWPMDvv5NeAAcdy9ngw1An4yli1AdhHJn4THrw0cNcacTMJj/s0Y8xtwMdbbzp+juHJRPWClMeaiMeYSVu2y+gmJJUUl/HsoCPzp9NpVKYdcwGWnxBJnuYdEVAMIN8YcjmO5AVaIyHa7xERS6Gj/WT0xjj8j43Mu3el9rNagK+4+X/H52f9ex/4sXcH6bCUZuxvpCWCri8VVRGS3iCwVkbJJFNL9fi+e/kw1J+5GlyfOF0BeY8xZsL7MAVcTbif6eUt2UxxKPEo5uNrMxXuxx5vGu9xDfMQzzhbcu3VfzRhzRkT8se5lOGi3Bv6ze8UFjAG+xPq5v8Tqbno/9i5cbJvgsbvxOV/y7xIdsSX6+Yodpov33Po5elAikhmYDXQxxlyNtXgHVrfFdfv6zDysmx7d7X6/F4+dM/s63cvYlXxj8dT5iq9EP2/JLuGb+5RyiEN8Sjmcx/pT0s9umSWo3MP94hQRP6AJUPEe+zhj/3tOROZidSkkKIHF9/yJyHhgkYtFbimLEY/z5apER+x9JPr5iiU+P3vMOmH27zgb//5z3S1EJA1Wsp9qjJkTe7nzF4AxZomIjBaR3MYYtxYKi8fvxZOlVhoAO4wx4bEXeOp82cJFJL8x5qzdvXXOxTphWNcZYhTCun75n3lLl859SznYSWQt0NR+6x0grr8YEkMd4KAxJszVQhHJJCJZYp5jXbgMdbVuYonVb9o4juNtA4qLNaIpLdafwwvcHFdcJTqc10mK8xWfn30B1mcHrM/Smri+oBKTfZ1gAnDAGDM0jnXyxVxPEJFKWP+/L7g5rvj8XhYAb9ujdSoDV2K6M5JAnH9le+J8OXH+HMWVi5YDdUUkh939Wtd+779z9xXqxHxgJakw4A4QDix3WtYba4TFIaCB0/tLgAL284exvgiOALOAdG6MdRLQNtZ7BYAlTrHsth/7sLo23H3+fgb2AnvsD1z+2HHZr1/AGgVyNIniOoLVV7nLfoyNHVdSnS9XPzvwBdaXEUB6+7NzxP4sPezu82MftzrWn/N7nM7TC0DbmM8Z0NE+N7uxLn5XTYK4XP5eYsUlwCj7nO7FaYSdm2PLiJXAszm9l+TnC+sL5yxw185fLbGu+6wGDtv/5rTXDQB+cNr2ffuzdgR4L6GxaGkFpZRKJbylS0cppdR9aMJXSqlUQhO+UkqlEprwlVIqldCEr5RSqYQmfKWUSiU04SulVCrxf/SAE5mobR3bAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "loss_func = MyMAELoss() # 损失函数\n", "opt = MyMomentum(net.trainable_params(), 0.01) # 优化器\n", "\n", "net_with_criterion = MyWithLossCell(net, loss_func) # 构建损失网络\n", "train_net = MyTrainStep(net_with_criterion, opt) # 构建训练网络\n", "\n", "for data in ds_train.create_dict_iterator():\n", " train_net(data['data'], data['label']) # 执行训练,并更新权重\n", " loss = net_with_criterion(data['data'], data['label']) # 计算损失值\n", " plot_model_and_datasets(train_net, train_data, loss) # 可视化训练过程" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 自定义评价指标\n", "\n", "当训练任务结束,常常需要评价指标(Metrics)评估函数来评估模型的好坏。常用的评价指标有混淆矩阵、准确率 Accuracy、精确率 Precision、召回率 Recall等。\n", "\n", "[mindspore.nn](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/mindspore.nn.html#评价指标)模块提供了常见的评估函数,用户也可以根据需要自行定义评估指标。自定义Metrics函数需要继承`nn.Metric`父类,并重新实现父类中的`clear`方法、`update`方法和`eval`方法。平均绝对误差(MAE)算法如下式所示,下面以简单的MAE为例,介绍这三个函数及其使用方法。\n", "\n", "$$ MAE=\\frac{1}{n}\\sum_{i=1}^n\\lvert y\\_pred_i - y_i \\rvert \\tag{6}$$\n", "\n", "- `clear`:初始化相关的内部参数。\n", "- `update`:接收网络预测输出和标签,计算误差,并更新内部评估结果。一般在每个step后进行计算,并更新统计值。\n", "- `eval`:计算最终评估结果,一般在一个epoch结束后计算最终的评估结果。" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:22.707013Z", "start_time": "2021-12-22T06:58:22.683370Z" } }, "outputs": [], "source": [ "class MyMAE(nn.Metric):\n", " \"\"\"定义metric\"\"\"\n", "\n", " def __init__(self):\n", " super(MyMAE, self).__init__()\n", " self.clear()\n", "\n", " def clear(self):\n", " \"\"\"初始化变量abs_error_sum和samples_num\"\"\"\n", " self.abs_error_sum = 0\n", " self.samples_num = 0\n", "\n", " def update(self, *inputs):\n", " \"\"\"更新abs_error_sum和samples_num\"\"\"\n", " y_pred = inputs[0].asnumpy()\n", " y = inputs[1].asnumpy()\n", "\n", " # 计算预测值与真实值的绝对误差\n", " error_abs = np.abs(y.reshape(y_pred.shape) - y_pred)\n", " self.abs_error_sum += error_abs.sum()\n", " self.samples_num += y.shape[0] # 样本的总数\n", "\n", " def eval(self):\n", " \"\"\"计算最终评估结果\"\"\"\n", " return self.abs_error_sum / self.samples_num" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 自定义验证流程\n", "\n", "mindspore.nn模块提供了评估网络包装函数[nn.WithEvalCell](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/nn/mindspore.nn.WithEvalCell.html#mindspore.nn.WithEvalCell),由于`nn.WithEvalCell`只有两个输入`data`和`label`,不适用于多数据或多标签的场景,所以需要自定义评估网络。多标签场景下自定义评估网络可参考[自定义评估与训练章节](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/train/train_eval.html#自定义训练和评估)。\n", "\n", "如下示例实现简单的自定义评估网络`MyWithEvalCell`,输入传入数据`data`和标签`label`:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:22.726360Z", "start_time": "2021-12-22T06:58:22.709549Z" } }, "outputs": [], "source": [ "class MyWithEvalCell(nn.Cell):\n", " \"\"\"定义验证流程\"\"\"\n", "\n", " def __init__(self, network):\n", " super(MyWithEvalCell, self).__init__(auto_prefix=False)\n", " self.network = network\n", "\n", " def construct(self, data, label):\n", " outputs = self.network(data)\n", " return outputs, label" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "执行推理并评估:\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "ExecuteTime": { "end_time": "2021-12-22T06:58:22.935251Z", "start_time": "2021-12-22T06:58:22.729350Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MAE: 0.9605088472366333\n" ] } ], "source": [ "data_number = 160\n", "batch_number = 16\n", "repeat_number = 1\n", "\n", "# 获取验证数据\n", "ds_eval = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number)\n", "\n", "eval_net = MyWithEvalCell(net) # 定义评估网络\n", "eval_net.set_train(False)\n", "mae = MyMAE()\n", "\n", "# 执行推理过程\n", "for data in ds_eval.create_dict_iterator():\n", " output, eval_y = eval_net(data['data'], data['label'])\n", " mae.update(output, eval_y)\n", "\n", "mae_result = mae.eval()\n", "print(\"MAE: \", mae_result)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "输出评估误差,MAE与模型在训练集上效果大致相同。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 保存及导出模型\n", "\n", "将上述训练好的模型参数保存到CheckPoint(简称ckpt)文件中,然后将CheckPoint文件导出为MindIR格式文件用于跨平台推理使用。" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fc.weight : [[1.894384]]\n", "fc.bias : [3.0015702]\n" ] } ], "source": [ "import numpy as np\n", "import mindspore as ms\n", "\n", "ms.save_checkpoint(net, \"./linear.ckpt\") # 将模型参数保存在ckpt文件\n", "param_dict = ms.load_checkpoint(\"./linear.ckpt\") # 将模型参数存入param_dict字典中\n", "\n", "# 查看模型参数\n", "for param in param_dict:\n", " print(param, \":\", param_dict[param].asnumpy())\n", "\n", "net1 = LinearNet()\n", "input_np = np.random.uniform(0.0, 1.0, size=[1, 1]).astype(np.float32)\n", "ms.export(net1, ms.Tensor(input_np), file_name='linear', file_format='MINDIR')" ] } ], "metadata": { "kernelspec": { "display_name": "MindSpore", "language": "python", "name": "mindspore" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0" } }, "nbformat": 4, "nbformat_minor": 4 }