{ "cells": [ { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "# 自定义损失函数\n", "\n", "[![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.3/tutorials/source_zh_cn/intermediate/custom/loss.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.3/tutorials/zh_cn/mindspore_loss.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.3/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9tb2RlbGFydHMvcXVpY2tfc3RhcnQvbWluZHNwb3JlX2xvc3MuaXB5bmI=&imageid=65f636a0-56cf-49df-b941-7d2a07ba8c8c)\n", "\n", "损失函数用于衡量预测值与真实值差异的程度。深度学习中，模型训练就是通过不停地迭代来缩小损失函数值的过程。因此在模型训练过程中损失函数的选择非常重要，定义一个好的损失函数，可以有效提高模型的性能。\n", "\n", "MindSpore提供了许多通用损失函数供用户选择，但这些通用损失函数并不适用于所有场景，有时需要用户自定义所需的损失函数。因此，本文介绍损失函数的自定义构建方法。\n", "\n", "## 自定义损失函数示例\n", "\n", "`Cell`是MindSpore的基本网络单元，可以用于构建网络，损失函数也可以通过`Cell`来定义。使用`Cell`定义损失函数的方法与定义一个普通的网络相同，差别在于，其执行逻辑在于计算前向网络输出与真实值之间的误差。\n", "\n", "下面使用`Cell`定义损失函数`L1Loss`，用于计算两个输入数据的绝对值误差：" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "import mindspore.nn as nn\n", "import mindspore.ops as ops\n", "\n", "class L1Loss(nn.Cell):\n", " def __init__(self):\n", " super(L1Loss, self).__init__()\n", " self.abs = ops.Abs()\n", " self.reduce_mean = ops.ReduceMean()\n", "\n", " def construct(self, predict, target):\n", " x = self.abs(predict - target)\n", " return self.reduce_mean(x)\n" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "首先，`__init__`方法实例化所需的算子，并在`construct`中调用这些算子。这样一个用于计算`L1Loss`的损失函数就定义好了。\n", "\n", "代码中使用`nn.Cell`作为`L1Loss`的基类，最后在`construct`中调用基类提供的`predict`, `target`方法。`reduction`的合法参数有`mean`、`sum`和`none`，分别表示求均值、求和与输出原值。\n", "\n", "在定义损失函数时还可以继承损失函数的基类`LossBase`。`LossBase`提供了`get_loss`方法，用于对损失值求和或求均值，输出一个标量。`L1Loss`使用`LossBase`作为基类的定义如下：" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "class L1Loss(nn.LossBase):\n", " def __init__(self, reduction=\"mean\"):\n", " super(L1Loss, self).__init__(reduction)\n", " self.abs = ops.Abs()\n", "\n", " def construct(self, base, target):\n", " x = self.abs(base - target)\n", " return self.get_loss(x)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "首先使用`LossBase`作为`L1Loss`的基类，然后给`__init__`增加一个参数`reduction`，并通过`super`传给基类，最后在`construct`中调用基类提供的`get_loss`方法。`reduction`的合法参数有`mean`、`sum`和`none`，分别表示求均值、求和与输出原值。\n", "\n", "## 损失函数与模型训练\n", "\n", "接下来使用通过基类`LossBase`定义的`L1Loss`进行模型训练。以简单的线性拟场景作为样例。\n", "\n", "> 线性拟合详细介绍可参考教程[实现简单线性函数拟合](https://www.mindspore.cn/docs/programming_guide/zh-CN/r1.3/quick_start/linear_regression.html)。\n", "\n", "### 定义数据集与网络\n", "\n", "定义训练数据集生成函数，并增强为MindSpore可训练的数据类型。\n", "\n", "- `get_data`：数据生成函数。\n", "- `create_dataset`：将numpy数据转换为MindSpore可训练的函数，并构造数据集的batch增强方法。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "import numpy as np\n", "from mindspore import dataset as ds\n", "\n", "# 生成随机数\n", "def get_data(num, w=2.0, b=3.0):\n", " for _ in range(num):\n", " x = np.random.uniform(-10.0, 10.0)\n", " noise = np.random.normal(0, 1)\n", " y = x * w + b + noise\n", " yield np.array([x]).astype(np.float32), np.array([y]).astype(np.float32)\n", "\n", "def create_dataset(num_data, batch_size=16):\n", " dataset = ds.GeneratorDataset(list(get_data(num_data)), column_names=['data', 'label'])\n", " dataset = dataset.batch(batch_size)\n", " return dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "定义网络，`nn.Dense`将数据集定义为所有的函数。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "from mindspore.common.initializer import Normal\n", "\n", "class LinearNet(nn.Cell):\n", " def __init__(self):\n", " super(LinearNet, self).__init__()\n", " self.fc = nn.Dense(1, 1, Normal(0.02), Normal(0.02))\n", "\n", " def construct(self, x):\n", " return self.fc(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 使用Model进行模型训练\n", "\n", "`Model`是MindSpore提供的用于模型训练、评估和推理的高阶API。创建数据集并定义一个`Model`就可以使用`train`接口进行模型训练。接下来我们使用`Model`来训练`LinearNet`，选择MindSpore提供的`Momemtum`作为优化器，并采用之前定义好的`L1Loss`作为此次训练的损失函数。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "net = LinearNet()\n", "loss = L1Loss()\n", "opt = nn.Momentum(net.trainable_params(), learning_rate=0.005, momentum=0.9)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "定义`Model`时需要指定前向网络、损失函数和优化器，`Model`内部会将它们关联起来，组成一张训练网。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "from mindspore import Model\n", "\n", "model = Model(net, loss, opt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "创建数据集，并调用`train`接口进行模型训练。\n", "\n", "参数解释：\n", "\n", "- `epoch`：训练数据集的迭代次数。\n", "- `train_dataset`：训练数据集。\n", "- `callbacks`：`model.train`的回调函数参数，可监控训练过程中参数的变化情况。\n", "- `LossMonitor`：损失函数值监视器，用于打印训练过程中的模型损失值。\n", "- `dataset_sink_mode`（bool）：数据下沉模式，可加快训练。Ascend和GPU平台支持开启该功能（True）。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch: 1 step: 1, loss is 7.9989004\n", "epoch: 1 step: 2, loss is 10.780429\n", "epoch: 1 step: 3, loss is 12.113883\n", "epoch: 1 step: 4, loss is 8.991383\n", "epoch: 1 step: 5, loss is 9.228854\n", "epoch: 1 step: 6, loss is 7.8051577\n", "epoch: 1 step: 7, loss is 9.328038\n", "epoch: 1 step: 8, loss is 7.665909\n", "epoch: 1 step: 9, loss is 4.955902\n", "epoch: 1 step: 10, loss is 7.7147307\n" ] } ], "source": [ "from mindspore.train.callback import LossMonitor\n", "\n", "ds_train = create_dataset(num_data=160)\n", "model.train(epoch=1, train_dataset=ds_train, callbacks=[LossMonitor()], dataset_sink_mode=False)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" } }, "nbformat": 4, "nbformat_minor": 4 }