{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 量子神经网络在自然语言处理中的应用\n", "\n", "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.8/mindquantum/zh_cn/mindspore_qnn_for_nlp.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.8/mindquantum/zh_cn/mindspore_qnn_for_nlp.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.8/docs/mindquantum/docs/source_zh_cn/qnn_for_nlp.ipynb)\n", "[![在线运行](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/resource/_static/logo_run_notebook.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9yMS44L21pbmRxdWFudHVtL3poX2NuL21pbmRzcG9yZV9xbm5fZm9yX25scC5pcHluYg%3D%3D&imageid=9549a798-7cce-42b2-a2ae-dcb864f122df)\n", "\n", "## 概述\n", "\n", "在自然语言处理过程中，词嵌入（Word embedding）是其中的重要步骤，它是一个将高维度空间的词向量嵌入到一个维数更低的连续向量空间的过程。当给予神经网络的语料信息不断增加时，网络的训练过程将越来越困难。利用量子力学的态叠加和纠缠等特性，我们可以利用量子神经网络来处理这些经典语料信息，加入其训练过程，并提高收敛精度。下面，我们将简单地搭建一个量子经典混合神经网络来完成一个词嵌入任务。\n", "*提示：由于HiQ量子云平台JupyterLab环境中svg图片暂时无法显示，请开发者自行打印量子线路。*\n", "\n", "## 环境准备\n", "\n", "设置系统所使用的线程数，当您的服务器CPU较多时，如果不设置，系统默认调用所有CPU，反而会导致模拟变慢甚至卡住。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "os.environ['OMP_NUM_THREADS'] = '1'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "导入本教程所依赖模块" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import time\n", "import mindspore as ms\n", "import mindspore.ops as ops\n", "import mindspore.dataset as ds\n", "from mindspore import nn\n", "from mindquantum.framework import MQLayer\n", "from mindquantum.core.gates import RX, RY, X, H\n", "from mindquantum.core.circuit import Circuit, UN\n", "from mindquantum.core.operators import Hamiltonian, QubitOperator" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "本教程实现的是一个[CBOW模型](https://blog.csdn.net/u010665216/article/details/78724856)，即利用某个词所处的环境来预测该词。例如对于“I love natural language processing”这句话，我们可以将其切分为5个词，\\[\"I\", \"love\", \"natural\", \"language\", \"processing”\\]，在所选窗口为2时，我们要处理的问题是利用\\[\"I\", \"love\", \"language\", \"processing\"\\]来预测出目标词汇\"natural\"。这里我们以窗口为2为例，搭建如下的量子神经网络，来完成词嵌入任务。\n", "\n", "![quantum word embedding](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/docs/mindquantum/docs/source_zh_cn/images/qcbow.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "这里，编码线路会将\"I\"、\"love\"、\"language\"和\"processing\"的编码信息编码到量子线路中，待训练的量子线路由四个Ansatz线路构成，最后我们在量子线路末端对量子比特做$\\text{Z}$基矢上的测量，具体所需测量的比特的个数由所需嵌入空间的维数确定。\n", "\n", "## 数据预处理\n", "\n", "我们对所需要处理的语句进行处理，生成关于该句子的词典，并根据窗口大小来生成样本点。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "def GenerateWordDictAndSample(corpus, window=2):\n", " all_words = corpus.split()\n", " word_set = list(set(all_words))\n", " word_set.sort()\n", " word_dict = {w: i for i, w in enumerate(word_set)}\n", " sampling = []\n", " for index, _ in enumerate(all_words[window:-window]):\n", " around = []\n", " for i in range(index, index + 2*window + 1):\n", " if i != index + window:\n", " around.append(all_words[i])\n", " sampling.append([around, all_words[index + window]])\n", " return word_dict, sampling" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'I': 0, 'language': 1, 'love': 2, 'natural': 3, 'processing': 4}\n", "word dict size: 5\n", "samples: [[['I', 'love', 'language', 'processing'], 'natural']]\n", "number of samples: 1\n" ] } ], "source": [ "word_dict, sample = GenerateWordDictAndSample(\"I love natural language processing\")\n", "print(word_dict)\n", "print('word dict size: ', len(word_dict))\n", "print('samples: ', sample)\n", "print('number of samples: ', len(sample))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "根据如上信息，我们得到该句子的词典大小为5，能够产生一个样本点。\n", "\n", "## 编码线路\n", "\n", "为了简单起见，我们使用的编码线路由$\\text{RX}$旋转门构成，结构如下。\n", "\n", "![encoder circuit](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/docs/mindquantum/docs/source_zh_cn/images/encoder.png)\n", "\n", "我们对每个量子门都作用一个$\\text{RX}$旋转门。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def GenerateEncoderCircuit(n_qubits, prefix=''):\n", " if prefix and prefix[-1] != '_':\n", " prefix += '_'\n", " circ = Circuit()\n", " for i in range(n_qubits):\n", " circ += RX(prefix + str(i)).on(i)\n", " return circ.as_encoder()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": "\n\n\nq0:\n \n\nq1:\n \n\nq2:\n \n\n\n\n\n\n\nRX\n \n\ne_0\n \n\n\n\n\nRX\n \n\ne_1\n \n\n\n\n\nRX\n \n\ne_2\n \n\n", "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "GenerateEncoderCircuit(3, prefix='e').svg()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "我们通常用$\\left|0\\right>$和$\\left|1\\right>$来标记二能级量子比特的两个状态，由态叠加原理，量子比特还可以处于这两个状态的叠加态：\n", "\n", "$$\n", "\\left|\\psi\\right>=\\alpha\\left|0\\right>+\\beta\\left|1\\right>\n", "$$\n", "\n", "对于$n$比特的量子态，其将处于$2^n$维的希尔伯特空间中。对于上面由5个词构成的词典，我们只需要$\\lceil \\log_2 5 \\rceil=3$个量子比特即可完成编码，这也体现出量子计算的优越性。\n", "\n", "例如对于上面词典中的\"love\"，其对应的标签为2，2的二进制表示为`010`，我们只需将编码线路中的`e_0`、`e_1`和`e_2`分别设为$0$、$\\pi$和$0$即可。下面来验证一下。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Label is: 2\n", "Binary label is: 010\n", "Parameters of encoder is: \n", " [0. 3.14159 0. ]\n", "Encoder circuit is: \n", " q0: ──RX(e_0)──\n", "\n", "q1: ──RX(e_1)──\n", "\n", "q2: ──RX(e_2)──\n", "Encoder parameter names are: \n", " ['e_0', 'e_1', 'e_2']\n", "Amplitude of quantum state is: \n", " [0. 0. 1. 0. 0. 0. 0. 0.]\n", "Label in quantum state is: 2\n" ] } ], "source": [ "from mindquantum.simulator import Simulator\n", "\n", "n_qubits = 3 # number of qubits of this quantum circuit\n", "label = 2 # label need to encode\n", "label_bin = bin(label)[-1: 1: -1].ljust(n_qubits, '0') # binary form of label\n", "label_array = np.array([int(i)*np.pi for i in label_bin]).astype(np.float32) # parameter value of encoder\n", "encoder = GenerateEncoderCircuit(n_qubits, prefix='e') # encoder circuit\n", "encoder_params_names = encoder.params_name # parameter names of encoder\n", "\n", "print(\"Label is: \", label)\n", "print(\"Binary label is: \", label_bin)\n", "print(\"Parameters of encoder is: \\n\", np.round(label_array, 5))\n", "print(\"Encoder circuit is: \\n\", encoder)\n", "print(\"Encoder parameter names are: \\n\", encoder_params_names)\n", "\n", "# quantum state evolution operator\n", "state = encoder.get_qs(pr=dict(zip(encoder_params_names, label_array)))\n", "amp = np.round(np.abs(state)**2, 3)\n", "\n", "print(\"Amplitude of quantum state is: \\n\", amp)\n", "print(\"Label in quantum state is: \", np.argmax(amp))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "通过上面的验证，我们发现，对于标签为2的数据，最后得到量子态的振幅最大的位置也是2，因此得到的量子态正是对输入标签的编码。我们将对数据编码生成参数数值的过程总结成如下函数。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "def GenerateTrainData(sample, word_dict):\n", " n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n", " data_x = []\n", " data_y = []\n", " for around, center in sample:\n", " data_x.append([])\n", " for word in around:\n", " label = word_dict[word]\n", " label_bin = bin(label)[-1: 1: -1].ljust(n_qubits, '0')\n", " label_array = [int(i)*np.pi for i in label_bin]\n", " data_x[-1].extend(label_array)\n", " data_y.append(word_dict[center])\n", " return np.array(data_x).astype(np.float32), np.array(data_y).astype(np.int32)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":2: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\n", "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n", " n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n" ] }, { "data": { "text/plain": [ "(array([[0. , 0. , 0. , 0. , 3.1415927, 0. ,\n", " 3.1415927, 0. , 0. , 0. , 0. , 3.1415927]],\n", " dtype=float32),\n", " array([3], dtype=int32))" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "GenerateTrainData(sample, word_dict)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "根据上面的结果，我们将4个输入的词编码的信息合并为一个更长向量，便于后续神经网络调用。\n", "\n", "## Ansatz线路\n", "\n", "Ansatz线路的选择多种多样，我们选择如下的量子线路作为Ansatz线路，它的一个单元由一层$\\text{RY}$门和一层$\\text{CNOT}$门构成，对此单元重复$p$次构成整个Ansatz线路。\n", "\n", "![ansatz circuit](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/docs/mindquantum/docs/source_zh_cn/images/ansatz.png)\n", "\n", "定义如下函数生成Ansatz线路。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "def GenerateAnsatzCircuit(n_qubits, layers, prefix=''):\n", " if prefix and prefix[-1] != '_':\n", " prefix += '_'\n", " circ = Circuit()\n", " for l in range(layers):\n", " for i in range(n_qubits):\n", " circ += RY(prefix + str(l) + '_' + str(i)).on(i)\n", " for i in range(l % 2, n_qubits, 2):\n", " if i < n_qubits and i + 1 < n_qubits:\n", " circ += X.on(i + 1, i)\n", " return circ" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": "\n\n\nq0:\n \n\nq1:\n \n\nq2:\n \n\nq3:\n \n\nq4:\n \n\n\n\n\n\n\n\n\nRY\n \n\na_0_0\n \n\n\n\n\nRY\n \n\na_0_1\n \n\n\n\n\nRY\n \n\na_0_2\n \n\n\n\n\nRY\n \n\na_0_3\n \n\n\n\n\nRY\n \n\na_0_4\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\nRY\n \n\na_1_0\n \n\n\n\n\nRY\n \n\na_1_1\n \n\n\n\n\nRY\n \n\na_1_2\n \n\n\n\n\nRY\n \n\na_1_3\n \n\n\n\n\nRY\n \n\na_1_4\n \n\n\n\n\n\n\n\n\n\n\n\n", "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "GenerateAnsatzCircuit(5, 2, 'a').svg()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 测量\n", "\n", "我们把对不同比特位上的测量结果作为降维后的数据。具体过程与比特编码类似，例如当我们想将词向量降维为5维向量时，对于第3维的数据可以如下产生：\n", "\n", "- 3对应的二进制为`00011`。\n", "- 测量量子线路末态对$Z_0Z_1$哈密顿量的期望值。\n", "\n", "下面函数将给出产生各个维度上数据所需的哈密顿量（hams），其中`n_qubits`表示线路的比特数，`dims`表示词嵌入的维度：" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "def GenerateEmbeddingHamiltonian(dims, n_qubits):\n", " hams = []\n", " for i in range(dims):\n", " s = ''\n", " for j, k in enumerate(bin(i + 1)[-1:1:-1]):\n", " if k == '1':\n", " s = s + 'Z' + str(j) + ' '\n", " hams.append(Hamiltonian(QubitOperator(s)))\n", " return hams" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1 [Z0] , 1 [Z1] , 1 [Z0 Z1] , 1 [Z2] , 1 [Z0 Z2] ]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "GenerateEmbeddingHamiltonian(5, 5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 量子版词向量嵌入层\n", "\n", "量子版词向量嵌入层结合前面的编码量子线路和待训练量子线路，以及测量哈密顿量，将`num_embedding`个词嵌入为`embedding_dim`维的词向量。这里我们还在量子线路的最开始加上了Hadamard门，将初态制备为均匀叠加态，用以提高量子神经网络的表达能力。\n", "\n", "下面，我们定义量子嵌入层，它将返回一个量子线路模拟算子。" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "def QEmbedding(num_embedding, embedding_dim, window, layers, n_threads):\n", " n_qubits = int(np.ceil(np.log2(num_embedding)))\n", " hams = GenerateEmbeddingHamiltonian(embedding_dim, n_qubits)\n", " circ = Circuit()\n", " circ = UN(H, n_qubits)\n", " encoder_param_name = []\n", " ansatz_param_name = []\n", " for w in range(2 * window):\n", " encoder = GenerateEncoderCircuit(n_qubits, 'Encoder_' + str(w))\n", " ansatz = GenerateAnsatzCircuit(n_qubits, layers, 'Ansatz_' + str(w))\n", " encoder.no_grad()\n", " circ += encoder\n", " circ += ansatz\n", " encoder_param_name.extend(encoder.params_name)\n", " ansatz_param_name.extend(ansatz.params_name)\n", " grad_ops = Simulator('projectq', circ.n_qubits).get_expectation_with_grad(hams,\n", " circ,\n", " parallel_worker=n_threads)\n", " return MQLayer(grad_ops)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "整个训练模型跟经典网络类似，由一个嵌入层和两个全连通层构成，然而此处的嵌入层是由量子神经网络构成。下面定义量子神经网络CBOW。" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "class CBOW(nn.Cell):\n", " def __init__(self, num_embedding, embedding_dim, window, layers, n_threads,\n", " hidden_dim):\n", " super(CBOW, self).__init__()\n", " self.embedding = QEmbedding(num_embedding, embedding_dim, window,\n", " layers, n_threads)\n", " self.dense1 = nn.Dense(embedding_dim, hidden_dim)\n", " self.dense2 = nn.Dense(hidden_dim, num_embedding)\n", " self.relu = ops.ReLU()\n", "\n", " def construct(self, x):\n", " embed = self.embedding(x)\n", " out = self.dense1(embed)\n", " out = self.relu(out)\n", " out = self.dense2(out)\n", " return out" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "下面我们对一个稍长的句子来进行训练。首先定义`LossMonitorWithCollection`用于监督收敛过程，并搜集收敛过程的损失。" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "class LossMonitorWithCollection(ms.train.callback.LossMonitor):\n", " def __init__(self, per_print_times=1):\n", " super(LossMonitorWithCollection, self).__init__(per_print_times)\n", " self.loss = []\n", "\n", " def begin(self, run_context):\n", " self.begin_time = time.time()\n", "\n", " def end(self, run_context):\n", " self.end_time = time.time()\n", " print('Total time used: {}'.format(self.end_time - self.begin_time))\n", "\n", " def epoch_begin(self, run_context):\n", " self.epoch_begin_time = time.time()\n", "\n", " def epoch_end(self, run_context):\n", " cb_params = run_context.original_args()\n", " self.epoch_end_time = time.time()\n", " if self._per_print_times != 0 and cb_params.cur_step_num % self._per_print_times == 0:\n", " print('')\n", "\n", " def step_end(self, run_context):\n", " cb_params = run_context.original_args()\n", " loss = cb_params.net_outputs\n", "\n", " if isinstance(loss, (tuple, list)):\n", " if isinstance(loss[0], ms.Tensor) and isinstance(loss[0].asnumpy(), np.ndarray):\n", " loss = loss[0]\n", "\n", " if isinstance(loss, ms.Tensor) and isinstance(loss.asnumpy(), np.ndarray):\n", " loss = np.mean(loss.asnumpy())\n", "\n", " cur_step_in_epoch = (cb_params.cur_step_num - 1) % cb_params.batch_num + 1\n", "\n", " if isinstance(loss, float) and (np.isnan(loss) or np.isinf(loss)):\n", " raise ValueError(\"epoch: {} step: {}. Invalid loss, terminating training.\".format(\n", " cb_params.cur_epoch_num, cur_step_in_epoch))\n", " self.loss.append(loss)\n", " if self._per_print_times != 0 and cb_params.cur_step_num % self._per_print_times == 0:\n", " print(\"\\repoch: %+3s step: %+3s time: %5.5s, loss is %5.5s\" % (cb_params.cur_epoch_num, cur_step_in_epoch, time.time() - self.epoch_begin_time, loss), flush=True, end='')\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "接下来，利用量子版本的`CBOW`来对一个长句进行词嵌入。运行之前请在终端运行`export OMP_NUM_THREADS=4`，将量子模拟器的线程数设置为4个，当所需模拟的量子系统比特数较多时，可设置更多的线程数来提高模拟效率。" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "scrolled": true, "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":2: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\n", "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n", " n_qubits = np.int(np.ceil(np.log2(1 + max(word_dict.values()))))\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "epoch: 25 step: 20 time: 0.351, loss is 3.154\n", "epoch: 50 step: 20 time: 0.362, loss is 3.023\n", "epoch: 75 step: 20 time: 0.353, loss is 2.948\n", "epoch: 100 step: 20 time: 0.389, loss is 2.299\n", "epoch: 125 step: 20 time: 0.392, loss is 0.810\n", "epoch: 150 step: 20 time: 0.389, loss is 0.464\n", "epoch: 175 step: 20 time: 0.384, loss is 0.306\n", "epoch: 200 step: 20 time: 0.383, loss is 0.217\n", "epoch: 225 step: 20 time: 0.387, loss is 0.168\n", "epoch: 250 step: 20 time: 0.382, loss is 0.143\n", "epoch: 275 step: 20 time: 0.389, loss is 0.130\n", "epoch: 300 step: 20 time: 0.386, loss is 0.122\n", "epoch: 325 step: 20 time: 0.408, loss is 0.117\n", "epoch: 350 step: 20 time: 0.492, loss is 0.102\n", "Total time used: 138.5629165172577\n" ] } ], "source": [ "import mindspore as ms\n", "ms.set_context(mode=ms.PYNATIVE_MODE, device_target=\"CPU\")\n", "corpus = \"\"\"We are about to study the idea of a computational process.\n", "Computational processes are abstract beings that inhabit computers.\n", "As they evolve, processes manipulate other abstract things called data.\n", "The evolution of a process is directed by a pattern of rules\n", "called a program. People create programs to direct processes. In effect,\n", "we conjure the spirits of the computer with our spells.\"\"\"\n", "\n", "ms.set_seed(42)\n", "window_size = 2\n", "embedding_dim = 10\n", "hidden_dim = 128\n", "word_dict, sample = GenerateWordDictAndSample(corpus, window=window_size)\n", "train_x, train_y = GenerateTrainData(sample, word_dict)\n", "\n", "train_loader = ds.NumpySlicesDataset({\n", " \"around\": train_x,\n", " \"center\": train_y\n", "}, shuffle=False).batch(3)\n", "net = CBOW(len(word_dict), embedding_dim, window_size, 3, 4, hidden_dim)\n", "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n", "net_opt = nn.Momentum(net.trainable_params(), 0.01, 0.9)\n", "loss_monitor = LossMonitorWithCollection(500)\n", "model = ms.Model(net, net_loss, net_opt)\n", "model.train(350, train_loader, callbacks=[loss_monitor], dataset_sink_mode=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "打印收敛过程中的损失函数值：" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxTUlEQVR4nO2de3gc9XX3P2d3JdmyZVu+4AtGNgbHMbca28FAUkoupEDTcEnCxXlD0lwMhPQtTfMUEnhJ6oaGpG2e0IY3xrk0IcWEBGNCeEkDSSmQYIQtc7GNYzDGMrLxXTKyZeu25/1jZ8RovXfN7Mxqz+d59Gh3Znb2SFrNd87ld46oKoZhGEb1EgvbAMMwDCNcTAgMwzCqHBMCwzCMKseEwDAMo8oxITAMw6hyEmEbUCwTJ07UmTNnhm2GYRhGRdHS0rJPVSdl2ldxQjBz5kzWrl0bthmGYRgVhYi0ZttnoSHDMIwqx4TAMAyjyjEhMAzDqHJMCAzDMKocEwLDMIwqx4TAMAyjyqm48tHhxIrm7dz1xKvsO9RD0ukCm1RFFWICSU19xQUQBrYrkEymzhGPpZ4Xug+gvibO5LEj+fS7T2Txoqay/syGYUQPqbQ21AsXLtQoryNY0bydH/1+K2++dZSjPf1IlotzfzJMK98mBkgsv7CIwKjaOHW1ccaNrDURMYwKQ0RaVHVhxn0mBKWTfkff219Zv8uhIs73ETUxRo1ImEAYRoQxIfCROx7dxD2rt9HVG5Fb+ggSFxhVl2DxWU3cfPHcsM0xDAMTgiHj3vm/efAoyYB/XXEnHON3jkCAMByWGDBmZIKJDSPMWzCMEAlVCEQkDqwFdqjqh9L21QH3AAuA/cCVqrot1/nKJQRurL+t4whHh3D3717EM12ca+MxTpw4ihPG1zOpoY7L509nwYxGX+zPREtrOyvXtbGvs5vNuzrZ0dEF5BcWP/MZU8bU8f65kwP/WQ3DGEzYQvBFYCEwJoMQfB44Q1WvE5GrgMtU9cpc5wtaCFpa27nhP1vY1dld9GtjAALHNQy/i50rIlt2d7J132E6j/bR05dkKJ+eS+dN4ztXnembjYZhZCc0IRCR6cBPgNuBL2YQgt8AX1PV1SKSAHYBkzSHUUEJwYrm7Xz78c3sO9RT8Gtq48JJk0Yzf0bjsLroF0O6QBw80ltU0jwRg/lNjdx00dyq/P0ZRrnIJQRBryP4DvD3QEOW/ccDbwCoap+IHAQmAPu8B4nIEmAJQFOTvzHmYgWgNi7MO2GcXbgcFsxoPOb30NLazq2r1vPK7s68eYm+JDy3rZ2PfO8Zjmuo5cYPzLE8gmGUmcA8AhH5EHCxqn5eRM4HvpTBI9gAXKiqbc7z14BFqrov/XwufnkExYaA7CJVGi2t7Sx78jWe2bKPwz39Bb1mwqgall/zLhNaw/CRUEJDIvIN4BNAHzACGAM8qKr/y3NMWUND7kXpudf3c/BIX97j62tivGf2JK79s5PsouQDxXpfp0xt4B8vPd1+94bhA6GXj+bwCG4ATvckiy9X1StynatUIbjj0U0se2prwcfXxYXuiCwQE6AuEeOM6WOHRUiqpbWdO369iXWt7QWVtJ4103IIhjFUIiUEIrIUWKuqD4vICOCnwJnAAeAqVc15tS5FCFY0b+crq9aXZHs1EBeIxYSYCGNGJDhx4igADhzuYfyoWsbV1w4c63eZazEL9EwQDKN0QhcCPylFCD7xw2aefjVr2sEogSlj6jhj+rhB2zq6eujuS3Llu5qKzqW0tLbzuZ+s4UBXb95jP3jKZAvXGUaRVL0QmEdQfuZOaeCE8fVAcV7Eiubt3P7/Xs6bWI4JfP3S0y15bxgFUvVCADD3//yaI9YfKDQEmNY4klOnjinobv7Gnz3PQy/szHteCxcZRmGYEFC6VxAXmDp2BJ9/7+zA7z69LSA6unoGVvD29icD73FUTgS4oIDwjrseYdOuzrzntHCRYeTGhMDhou88VdBFJRMxgYUzGhlXX0tHV8+gRGo5+gT5hVtC+/z2dtq7ekKdixAT+MDc/BfwQiu+LFxkGNkxIXBoaU2tYA0CVyggc7VNunh4KWVfvteUmrR1cZvu7T3cTVd3f6CzFkTg2j+dlbNltVtyumZbe+5zAbdfZmJgGOmYEHhYcs9aHnt5t48WRRtv0hZKExavx1NM+4hiKSTev6J5O7esWp+32Z3lDgxjMCYEHgr1CuoSMU5oHElNPMbm3Z3DKkZfKumi4grE5l2dLP3VRo72DT3OFI8J/3jJaTnv6Fc0b+f/PLS+ICGy3IFhpDAhSKOYXMGl86bxiXNmDiRxYfDdM8Da1vaqFQq3Guj4sSMG8icbd75VcF+hbOfMl0x2cx2PF+DdxWPw82vPNTEwqhoTgjRKzRXEYzBn8rH9b9KrfcLOEbxxoKvkpPhQcXMlnUf7hmxDIcnfFc3buXXVevL5IpNG17LsEwtNDIyqxYQgAx/49pNs2XPIB4uihwCx2OCpYyKpFhI1MSFJqp32qBE1g+7kMwnLUEVlSkNdSUN+vFx3Xv5EcqHeQb5zGcZwxYQgA7bauHASzui1hhFxZh/XMCAca7a1FzShbO6UBtoP9wxJEAqZZlZomal1NTWqEROCLLzr64+zt4iJZEYKcb8LjEjEQIQYcChPXmDcyAS9SeVwd2n5g0Iu4Hc8uom7n9pakECZd2BUE7mEIFZuY6KE22UzGw118TJZUlmo85VU6OpN0tXTn1cEADqO9HG4u7/kD93Lb3byke89wxXLnqGlNfN6gpsvnsvtl51OXDLuHsSyp7Zy48+eL9Eawxg+VLUQzJ6cbYJmis7ufq47bxYrrz+XC06ZTG0hVxcjL0MtMnVHW97x6KaM+xcvauLn16X+ZpLnT/bQCztNDIyqp6pDQy2t7Xx02TPk+xXkqkX3Vgy5BL3qN5MNy558jdf3HqImHuOto7109yfp7U9ytCc57HoVecm3cKzQBWiF5CAMo5KxHEEOCl1pXGhfnEolXdC8Te+SqvQnNbJiku9vU2gS2eZSG8OZsGYWjwCeAuqABPCAqn417ZhPAf8M7HA2fVdVf5DrvH4LQUtrOx/73jMFhysEOOm40Xz63SdW3QXD27Auil1RcwlCMUlk8w6M4UhYQiDAKFU9JCI1wO+Bv1HVZz3HfApYqKpfKPS8fgsBwC2r1nNv8/aiXzd3SgPzZzRWTOfRIHCbwb30RkdkZjxnW4i2onk7tz60viDxOm/2RO75zKKALDSM8hN6aEhE6kkJwfWq2uzZ/ikiIAQtre1cefczZGuVI5DzTjImqRXHvf1JZk0aPWzDR/kotENoOcjWhdTEwKhWQhMCEYkDLcDJwF2qelPa/k8B3wD2Aq8Af6uqb2Q4zxJgCUBTU9OC1tZW323N5xWcNbOx4AVUArxrZuNAojiIBHGUcfMNj2/cFfo6jUyJ/mKa1lmYyBguRMEjGAesAv5aVTd4tk8ADqlqt4hcC1ypqu/Lda4gPAJIXbyuXr6anixXhxnj6/n2lfOGfMdbbFvoSvYwWlrbuXL5avpCDhkJcG3a4rFi2lLYSmRjOBC6EDhG3AZ0qeq/ZNkfBw6o6thc5wlKCCC/VzBzQj3/esU8AFaua2PL7s6ydB7N5GGU0qQuDK/EveD+9uXdBXlTQZKpKqjQqjERuN2mnxkVTFjJ4klAr6p2iMhI4DHgm6r6iOeYqar6pvP4MuAmVT0713mDFIJUrmA1fXmu7N7adTcMsmV3Jzs6jrCz42joF7x85PNKghi9GaX8gfcOv6W1nSvuXk1/gWpu8w2MSiUsITgD+AkQJ7WC+eequlRElgJrVfVhEfkG8GGgDzhAKpn8x1znDVIIYOjx40wLzMJsC10qmUZvzp7cMGSBKLRtdNB4k8nF/M3B5hsYlUkkQkN+EbQQQHHlpPU1Md4ze1Leu8RMAgG5QzlR9DC8AlFqqKmltZ2bVr4UiTbgrncHpEJYm3bnXWkOqVDdL647N2DrDMM/TAiKpKW1nSuWPVP0TN5Jo2s5s6nR19BBKS0sgpgrkAs31FRMSCkqiWSXs2Y2MntyAw11CZY/vbWgvI+FiYxKwoSgBNKTiGfNbESh4Bh3bVyor40PDH/xI6wyVArxSmBoozddj6GQnzdKiWQXEfjTkyfy+y37CvodFDJj2TCigAlBCaTfsbr/8HOmNHDDf7aUPGSlJp6aFDayJkYiNfFloDkcMGi7d188JswYX1+WJG+m0Zs18Ribd3cWJRCFjJp03y8qiWSX82ZP5OlX9xW8biTT4jXDiBImBCWSnivwXtjueHQT9za30lnikJUgGV0bp9e5YmcTlkL2ud4McEy4qZhQkxt2ySdSUZsaN25kDR1Hegs+3gbdGFHGhKBEMpWTpt/lehux7auSaWfjRiYYNaKGMXUJ9h7qpqunv6AJZYV0cC20U2hUMTEwoooJwRDI1JsmW8jDDXFs3HGQfoWamBQ0uavayBdXr3Qx+CcLExkRxIRgiBQjBul4F5y5/f2BonIEXd399EakusZPcrVuiMp6g1KxiiIjapgQ+EAmMcjUwyYoCq342dFxhEM9fQXnAbLtK6c3k62dd0trO1+8/wVaD3SVxQ6/sYoiI0qYEPhEthbGwzUunF49tKPjyMAIzINdfb6/n9vOuzYRG7RQ7ZofNvPUq/t8f79yUKjnaBhBY0LgI9lm4FZbKMArEkEtVvN6Co9v3FWxeQMrLzWigAmBz2RLZg73uca58OZC8i1Ii8egv4jgv/t7ra+N89ALO4dubEhYEtkIExOCAMg1A1cErv3T4RkuKgRXFJ5vbc/qKQgwpj5RVIiplNdEDRMDIyxMCAIiX9dKm2lc3ND4amG45pSMaGNCECDugrLfbdqdNRzi9t8Jqtd/1Cnkd5RvLvRwI9OQHMMIEhOCMlBMv5z0iWPVIg5RbDIXNjMn1POOyQ1V8xkwwiOswTQjgKeAOiABPKCqX007pg64B1gA7Cc1s3hbrvNGVQhcSg2FCDCtcSTHjx0x7GYXp5OtDNclk3cgQNP4el/WFIysiXGkN3pL1dzy2d7+5LD6exvRICwhEGCUqh4SkRrg98DfqOqznmM+D5yhqteJyFXAZap6Za7zRl0IoLgKmmLIN7u4ku4qB7yDAgfBwNsL+F7bdzhnmClIBHjnlIayTJxL/3tD9XiPhv+EHhoSkXpSQnC9qjZ7tv8G+JqqrhaRBLALmKQ5jKoEIfDiFYU129oDD4mkj5iM+p1lsWMi3SHyc6Y0+BJmmlvCRd1P76QUst0QRP1vbYRLaEIgInGgBTgZuEtVb0rbvwG4UFXbnOevAYtUdV/acUuAJQBNTU0LWltbA7M5SLJNGwtaILwXjijeUebyDupr43SltboQ4AJnAR8w5FkG86aP5cW2g8Mib5HPawTzKqqVKHgE44BVwF+r6gbP9oKEwEuleQSF4F4IX997qCyzi6O6ziHbqu1seBfwAdy6an3JIZsxI+K8c8oYnitRUPysemqoi3Oouz9QYfLmI4oZeVrKmFQvmUSopbWdZ7fu5+xZEwBYua4NARMrnwldCBwjbgO6VPVfPNuGfWjIL/LNLobiR0y6g9uj9M+2onk7tzy0PmPeINvF1tvcbagtrOdOaaD9cE/RE+iOdxL9pQpJOufNnsj08fUDf+9yeI7lwltO/caBLv64uxNViJH6+7o/Y7YCCsicHzt12lie2Lwn4w2VSzV7Q2EliycBvaraISIjgceAb6rqI55jbgBO9ySLL1fVK3Kdt1qFoBAyNYnL50lEsS1GsXkDGNzPJ33edClMHzeCto6jRb0mHhM+954T+fWGXb7kD2ZOqOdfr5g38HfJ1YHWb69xOJMpCQ/DXyTCEoIzgJ8AcVJi/3NVXSoiS4G1qvqwU2L6U+BM4ABwlarmvJ0zISiOQpvDRU0Q8lUVZSwx9SSSr7h7Nf1DLCuqiUGxVaZu/uJob78vHVMFeOD6cwv6m+TzGsfV19LR1eNrJdtwI1OOpbsvOagbbqUSidCQX5gQDI186xyi1kM/15qDSaNr2Zs2HtS9EM+aOIrvP721KK/CT0Tgkj+Zxo6OI0NKZAOMqo1zy1+c4tvfxFvJVmy8v5QcQbHiE9VV5nOnNHDC+PqB55UmEiYExiDytXyIWtvkXKGimJDxZ3DDNGGuOYBUX6ELTp3iy4CdXBPdoo6bEG6sr2XDzoODch/dfUnOmTWBt7r7BpLEQNYCCvd13vxYy/YOkklFgKlZFmYGmWNxRSLK4SUTAiMjuQTBDbNERQxaWtu5aeVLbNlz6Jh9Extq2d/Zk3E18rXnzaKzu497m7eXxc5MuLMq7vztK76Ei6pt9kUheCuPsv1ecuVY/BSJbAOWwsaEwMhJtots1KZrtbS2c+Xy1fRlcA3E+Z7p0zx3SgOv7j2U8XVzpzQMVK0EiZuDmTVxFA+sa2NfWkirWMo5JrUayCYSfgxdikp1ngmBkZdsF9koikGuJPKMLCt+3bu09H/qKWPquHTe8WXLJ/gtCFG5yAxn/BAJt2R29uSG0EJHJgRGQWS7yCZiwv3XnhOpi02uvMG86WN5oe3gMduztYZw8wlvdffx1OY9RZeNlkIiLty/5Bw27+rkm/+1iYNHSh+2411pHaW/UTWQXrJdaFI8jFklJgRGUaRX6ghw9aIm/umy00O1Kx1XuB5PWzMgwJ/OnsjTr+4rOO7rXkzPn3Mctz28IWMYyW9OPm403/zIGSyY0cg1P2z2JX9gHkK4FDKdz0s5PQUTAqNo0u+4o1ZW6iXTamK3HhxgTWt7wTmAclcbee/mN+/q5K4nXmWHDx6JTUELn0IGMnkR4KTjRvPpd58YyP+ZCYFREresWj+o2iZq+QIv2dZHxGPCX54xlV+9uLPgHEBc4OfXnQuQ0eMIAr8b6QE01id418wJFjIKmWK9BAimHNWEwCiJltZ2rrx7NX2e25ko5gtcsi0+G+QdFFgm6A3bFDJ9ri4Ro7vPn2E3l86bxneuOpOW1nZf1h9AavHdmU2NJgohs6J5O/ev2U5PX5LNuzsL8hT8Ch+ZEBglkylf8KU/n8MN7z05VLuykSuJnIgLn333idz99NaCQkXp4bAbf/Y8D72wM+drRtXFOdzdn/OYQqhLxPjqX57K4kVNrGjeztJfbeSoT0JjeYRoUMqsErfIoJS/nQmBMSTSY/D/FKFVx5nIVWJ68nGj+cA7jyu4XDS9IqfQpK5fglAbF+adMI6bLprLT1dvyytExWBeQnQoZub54hILN0wIjCFx1xNb+NfHNpPU6FYQZSLbfAPvFLdCy/1KbXU9MhHjiE938nGB8aNqqa9N+D4drb42zmnTxpinEDKFlKN+fFETt5sQmBCUm5bWdq5evpoe5xY6yhVE6eSab1CsIHhX8xZzBzdjfD07Dx6h18eS1CAbs/nhKWRbhAWlN67LNfTGm1R131uAhroEq7fupy4RGzShzd0+ecyIgn7OQlpYBIG38kgVahIx7vvc2RYaMiEIh/QKoignjdPJN9/A7RS6YedbGXsZpeONsa9o3s5XVq3P+5qodtTMhwA1cWHsyBpOnDiqoAt3WEN0BHjnlIaCk7De1+Ua79nR1VNQU7t84jZr0mjOn3PcQNM9V7yAgvokDVWITAiMIZNeQVRJISIorKZ77pQGXtnTSX8BkRyvV1SoGBhGJtybhLjAAmdyWxCzpk0IDF9Iv7OuHYKbGhaFtOCekKWbaaZjvYvBbl21Hn+yAYaRmaH8z+USgsSQLcv+picA9wCTSQneclW9M+2Y84FfAq87mx5U1aVB2WQMjcWLmti48yArmrejQH9/kme37q8oIVgwo5HvX7Mwa4xfgX2dPQjZ21t7j33s5d38dtNuPjB3Ml+/7HSe2LyH3768u+CwyOjaOId6Cqsu8qsSqZxkGgvpZ44gWxhKSIX83AKHdzoLtNwwT7/Hs62kW+HevmD+5wITAqAP+DtVXSciDUCLiDyuqi+nHfe0qn4oQDsMH7l8/nRWrmujpzeJiNCYIXFXCSyY0cgvrjs364pkVxAAxtYnONiVvSlcUt8WhK9fejrvnXNc1qlq6RQqAgCHu/uZO6WBt472Vsx84ngMXtndycSGEYG1TkhPTOeLvXvj7UDe8Z7uOU+dNpYnNu/JOCwnn7jt6DjCmwePopoqUnjH5Ab+uKuz6L9hTSI2YLeflC00JCK/BL6rqo97tp0PfKkYIbDQUPisaN7Obb/cQFKV2kSMez9bWeGhdIrtCZMLd6APkDNBPVSKCWFFjZikxm/W1cYBqEvEj0m+QuWNgsxHesLX/dzteevowIQ27+S2YZcjEJGZwFPAaar6lmf7+cBKoA3YSUoUNmZ4/RJgCUBTU9OC1tbWwG02slOp6wryMbAQrYjQTjY+6HQyXfV825B7BuVj3MgEB4/0VZwgFMPMCfW8Y3IDcOxFMsrjIaNEqEIgIqOBJ4HbVfXBtH1jgKSqHhKRi4E7VXV2rvOZRxA+6esKKjFpnItsPYuKxU0m19fG+eULOwO/UE9sSN05VqKXMFS8a0IyhWhMLEIUAhGpAR4BfqOq3y7g+G3AQlXNuobfhCAa3LJq/UDSeDh5BS7ePjCFrj7OhrtOoZgOqEOlvjZOV4H5h7iTVB3u4iHAtLQ1AG7sv72rp+yLxcpNKEIgIgL8BDigqjdmOWYKsFtVVUTOAh4AZmgOo0wIosFw9wq8+JVDmDG+nr5ksqyJ3oa6VCy+0HLYeAx86ohRkbi/L3g7dxHmeEk/CUsI3gM8DayHgfLqrwBNAKq6TES+AFxPqsLoCPBFVX0m13lNCKKD1yuIC3zxg9HtSuoHfuYQxo1M0DGE8ZSlvqcKOSugjOyki8SYugS1iRjnzJpAw8iayHsUoSeL/cSEIDq4XkFvv1ITF+4rsT1upeFn2Cgs4kLZwlTVRENdnDH1tYypS9Dbn8zaGwkGl7l6S1hzlaIOJdcRyoIyo0oQZ0mOSNiWlI0FMxoH/hHTu0WG0WOnFLwikKjycJCfdHb309l9hB3uhr2Hcx7v7d91DFle+4uWNt/DsCYERsk8u3U/ff3Jil1l7AdeUYBjheG5gEtH/cBEoLIIYnWxCYFRMmfPmkBtIlbxq4z9JJswPN/aTuv+w3T12lXXGBqxmPi+utiEwCiZBTMaue1Dpw6sMl76yEbmTGmoOq8gF5mEwY/B9Eb1cuKEet//x2K+ns2oOtq7ekiqktS3XVYjO26Po5XXn8viRU3MndIQtklGicRILWQrNzVx/y/bBXkEIjIKOKKqSRF5B/BO4Neq2uu7RUZFYeGh0khPOJuXUHkkIZRVeJt3d9LS2u6rV1CotDwFjBCR44HHgE8AP/bNCqNiccNDsZgMhIdaWu2CVgzpXsKUhrqwTRqWiPNV6ajiu+ddaI5AVLVLRD4D/F9V/ZaIvOCrJUbFkik8ZHmC4hnwEi47fWDx2vPb29l3qCds04YFlVDWWwhBtKIuWAhE5Bzg48BnnG1xXy0xKhYLD/mPO0AH3q48+sOr+2g90BWyZUaYCPC1vzw1tME0NwJfBlap6kYRmQU84aslRsVi1UPBkmkB25bdnWzdd9i8hSpDgY07D/p+3oKEQFWfJNVKGhGJAftU9X/7bo1RsVh4qDxkW6fwh1f3saOjyxaHVQGv7u70/ZyFVg2tAK4D+oE1wBgRuVNV/9l3i4yKxMJD4ZBJGJY9+Rov7zxId3+SQ0f7OBriIja3o2lSGdSTKS6QiMeoiQm9SaXbFKxg1mxr971qqKCmcyLygqrOE5GPA/OBm4EWVT3DN0sKxJrORZfhNsJyuHDHo5v4+do3SKqSSKQKBXv7k5HpQhrj7fbEg7YLjEjE6FdMKNJYXML8Dz+aztU4Q2YuJTV3uFdEhksS3vAJCw9Fk5svnsvNF889ZntLazu3rlrPa3sPUZeIDbRYLrdIZLvEJxVryZEFt1OpXxQqBHcD24AXgadEZAbwVs5XGFWHhYcqiwUzGvn1jedl3Ocucvvjm28Rj8kgT6LzSF9Ftt4eTkz0ea1JyfMIRCShqllvG0TkBOAeYDKpZPdyVb0z7RgB7gQuBrqAT6nqulzva6GhaGPhoerAW720o+MIh3r6ONqTpLc/SVJTuYGYzTwIBAEeuP7cov+vhhwaEpGxwFcB9/bhSWApkKuOqQ/4O1VdJyINQIuIPK6qL3uOuQiY7XwtAr7nfDcqFAsPVQfpSepseNtyQ2royo6OI3T3p0TDKx5GfiaMqmH5Ne8KbR3Bj4ANwBXO808A/wFcnu0Fqvom8KbzuFNENgHHA14huAS4x5lR/KyIjBORqc5rjQrEDQ/19iUDWQFpVBaFCgakROPZrfsHPjPuyurOo6nAw8iaGL1J5XB3f2D2Rp32rmDauxUqBCep6kc8z/+hmBYTIjITOBNoTtt1PPCG53mbs22QEIjIEmAJQFNTU6Fva4TAghmN3PvZs1m5rm1Y9HUxyke6aLgrq9Pxlsge6umjt0+pjQ/OY/T2KTGg13E13DLVSg9dJRUeXNcWmkdwRETeo6q/BxCRd5MaNp8XERkNrARuVNWSEsyquhxYDqkcQSnnMMrLg+va6OlLsnJdm+UJDF/xtt8YKt5cR7nHjDbUxYnFhaM9SfqSSVRT4pSvUjYIGwsVguuAe5xcAUA78Ml8L3JKTlcC96rqgxkO2QGc4Hk+3dlmVDDPbt1PT1/S8gRG5Mk2f9pLpmHyfsyn7swQ4nJzJeNGJuhNKkd7+gd5LSJw2rSxx7xuqBTaYuJF4E9EZIzz/C0RuRF4KdtrnIqgHwKbVPXbWQ57GPiCiPyMVJL4oOUHKh8rIzUqkWLyGfB2iOr1vYcGRKKjq4eW7R30DzH73XEkc0GmKnztV/738hpK+eh2Vc0asBeR9wBPA+t5e83IV4AmAFVd5ojFd4ELSZWP/pWq5qwNtfLRysDKSI1qxU16dx7pZfXW/dQlYgOexBsHuti0a2i9ggT40p/P4Yb3nlzc63xYWZzNnqw4+YR8xyhwwxBsMCKKlZEa1Uo+z2JF83buX7N9kEBA4SIR5jyCTFjS1siKlZEaRmYWL2pi8aLMwZRsIuEyqaGOy+dP9/2mKmdoSEQ6yXzBF2Ckqg5FSErCQkOVw4rm7fx6w5tcdNrUrB98wzDKQ8mhIVVtCMYkY7jT0trO0kc20tOXZM22AzaoxjAiTKHD6w2jKDKVkBqGEU1MCIxAcHMEMbASUsOIOCYERiC4c4xjMRmYY9zS2h62WYZhZMCEwAiMTCWkhmFEDxMCIzDc8FBcgql9NgzDH8pe/mlUD9aJ1DAqA/MIjMB5cF0b9z23nY//4FnLExhGBDEhMALFykgNI/qYEBiBYmWkhhF9TAiMQLEyUsOIPiYERuBYGalhRBsTAiNwrIzUMKKNlY8agWNlpIYRbQLzCETkRyKyR0Q2ZNl/vogcFJEXnK/bgrLFiAZWRmoY0STI0NCPSY2gzMXTqjrP+VoaoC1GyFgZqWFEl8CEQFWfAg4EdX6jsrA8gWFEl7BzBOeIyIvATuBLqrox00EisgRYAtDUZJOuKhHLExhGdAmzamgdMENV/wT4d+ChbAeq6nJVXaiqCydNmlQu+4wAsDyBYUSP0IRAVd9S1UPO40eBGhGZGJY9RvBYnsAwokloQiAiU0REnMdnObbYlWEYY+0mDCOaBFk+eh+wGpgjIm0i8hkRuU5ErnMO+SiwwckR/BtwlapqUPYY4WPtJgwjmgSWLFbVq/Ps/y7w3aDe34gmmdpNLJjRGLZZhlHVWIsJo6xYGalhRI+wy0eNKsPKSA0jephHYISClZEaRnQwITDKzrNb99Pdmyoj7em1MlLDCBsTAqPsNNbX4paHJZ3nhmGEhwmBUXbau3qIOQmCmKSeG4YRHiYERtnxLiyL2cIywwgdEwKj7NjCMsOIFiYERih4F5Z19yZZua4tbJMMo2oxITBC4exZE0g4iQIFHmhpM6/AMELChMAIhQUzGvnYwhMGFpX191sZqWGEhQmBERqXz59OXY11IzWMsDEhMELDksaGEQ1MCIxQydSN1DCM8mJCYISKDasxjPAJcjDNj0Rkj4hsyLJfROTfRGSLiLwkIvODssWILhYeMozwCdIj+DFwYY79FwGzna8lwPcCtMWIMBYeMoxwCUwIVPUp4ECOQy4B7tEUzwLjRGRqUPYY0cXCQ4YRLmHmCI4H3vA8b3O2GVWGhYcMI1wqIlksIktEZK2IrN27d2/Y5hgBYC0nDCM8whSCHcAJnufTnW3HoKrLVXWhqi6cNGlSWYwzyou1nDCM8AhTCB4GrnGqh84GDqrqmyHaY4SI23LCpc+SxoZRNgIbXi8i9wHnAxNFpA34KlADoKrLgEeBi4EtQBfwV0HZYlQGp04bO/DYJpcZRvkITAhU9eo8+xW4Iaj3NyoPd3JZUkGADTsPhm2SYVQFFZEsNqqD9DzB/WveYEXz9nCNMowqwITAiAzpeYL+pHLbLzdY0tgwAsaEwIgUl8+fPuAVQEoMrJTUMILFhMCIFAtmNLL0ktOIO1pgISLDCB4TAiNyLF7UxFVnNQ08txCRYQSLCYERSSxEZBjlw4TAiCQWIjKM8mFCYESWTCGiWx9ab2JgGD5jQmBEmvQQUVLhllUmBobhJyYERqRxQ0Ti2abALeYZGIZvmBAYkWfxoiYuOGXyoG2qWJjIMHzChMCoCK79s5NIxGXQtqSJgWH4ggmBUREsmNHI/UvO4YJTJiMePTAxMIyhY0JgVAwLZjTy/WsWcvulpxNLE4NbVq1nyT1rbdGZYZSACYFRcSxe1MTX08RAgcde3s3Hlj1j3oFhFElg8wgMI0gWL0qtL7j1ofUk9e3tSYWvrFrPQ8+3MXtyA5fPn86CGY0hWWkYlUGgHoGIXCgim0Vki4jcnGH/p0Rkr4i84Hx9Nkh7jOGF6xlIhn3PbWvn3ubtXHH3avMQDCMPQY6qjAN3ARcAbcAaEXlYVV9OO/R+Vf1CUHYYw5tsnoFLf1K5ZdV6nnt9P7MnN3D2rAnmIRhGGkGGhs4CtqjqVgAR+RlwCZAuBIYxJBYvamLOlAaWPfkav9u0+xhBUOChF3YCkIgJSy85bUBADMMIVgiOB97wPG8DFmU47iMich7wCvC3qvpG+gEisgRYAtDUZP/AxrG4FUUtre2sXNfGlt2dPLft2AqiPsdDeOj5NsbV1zKpoc7yCEbVI6kZ8gGcWOSjwIWq+lnn+SeARd4wkIhMAA6pareIXAtcqarvy3XehQsX6tq1awOx2Rhe3PHoJpY9tTXvcTGBr196unkJxrBGRFpUdWGmfUF6BDuAEzzPpzvbBlDV/Z6nPwC+FaA9RpVx88VzAVj+9NaM+QMXqzQyqp0gPYIEqXDP+0kJwBpgsapu9BwzVVXfdB5fBtykqmfnOq95BEaxtLS28+zW/XQe6eX7T2+lP89HPiawcEajiYIxrMjlEQQmBM4bXwx8B4gDP1LV20VkKbBWVR8WkW8AHwb6gAPA9ar6x1znNCEwhoI3h7BmWzv5Pv0CXHDKZK79s5NMEIyKJjQhCAITAsMvWlrbs1YapWNeglHpmBAYRg68XsLa1va8ogAwd0oDJ4yvt6ojo2IwITCMAnG9hN++vDtv2MjFvAWjEjAhMIwiKcVLcDFvwYgiJgSGMQRcUXi+tZ1NuzqLeq3rLdjiNSNsTAgMwydWNG/n/jXbqUuk+jUW6y0I8K6ZJgxG+TEhMIyAGIq3AClhmNY4kuPHjmBcfS2ACYQRCCYEhlEGhuoteIkJzJncQG9/kvGjagc8iFOnjaW9q8e6qBpFY0JgGCFQ7OK1YogBUzN4EiYURjbC6jVkGFXNghmNAxdjVxT2dXYD0NHVMyRxSAI72o+wo/1Ixv0xYKGTi3Dp6OrhwOEexo+qZfbkBhMNYwDzCAwjJNw1C6/vPTQQ/uno6hlSSKkU4gLvSAtDefEKSPo+y2dUDhYaMowKwhtSci/AAC3bO+gvp0IUiLcSyiWXeGTbl237pIY6GuoSbHzzLS46bSpzpjTw7Nb9WT0Zt8mgeTqDMSEwjGGAe4FrrK/lic17QvckwiImoJr67l7o0wUzmVSEY/MoLn4KVSn7wsjnmBAYRhWQnodIx70g1cRjvLLnUCS9i2okXz4n3TsqNRRnQmAYxiC83sWGnQcHhaEKuaMdarLbKJ3aRIz7Pnd20WJgVUOGYQzCW9FUKrk8EL9CLx1dPZHNjYRFb1+SZ7fu9zWcZEJgGEZJ+CEmhZDuvQhw6rSxbNh5kH2d3ccIiBt/T8+jeAkzRzDUfE5NIsbZsyaU9uIsBCoEInIhcCepCWU/UNU70vbXAfcAC4D9pIbXbwvSJsMwKotSBWfxoqYArPGHQvM5fuYIchGYEIhIHLgLuABoA9aIyMOq+rLnsM8A7ap6sohcBXwTuDIomwzDMKJAubypQokFeO6zgC2qulVVe4CfAZekHXMJ8BPn8QPA+0VEArTJMAzDSCNIITgeeMPzvM3ZlvEYVe0DDgLHBL9EZImIrBWRtXv37g3IXMMwjOokSCHwDVVdrqoLVXXhpEmTwjbHMAxjWBGkEOwATvA8n+5sy3iMiCSAsaSSxoZhGEaZCFII1gCzReREEakFrgIeTjvmYeCTzuOPAv+tlbbCzTAMo8IJrGpIVftE5AvAb0iVj/5IVTeKyFJgrao+DPwQ+KmIbAEOkBILwzAMo4xUXIsJEdkLtJb48onAPh/NCZpKsreSbIXKsreSbIXKsreSbIWh2TtDVTMmWStOCIaCiKzN1msjilSSvZVkK1SWvZVkK1SWvZVkKwRnb0VUDRmGYRjBYUJgGIZR5VSbECwP24AiqSR7K8lWqCx7K8lWqCx7K8lWCMjeqsoRGIZhGMdSbR6BYRiGkYYJgWEYRpVTNUIgIheKyGYR2SIiN4dkw49EZI+IbPBsGy8ij4vIq873Rme7iMi/Ofa+JCLzPa/5pHP8qyLyyUzv5ZO9J4jIEyLysohsFJG/iarNIjJCRJ4TkRcdW//B2X6iiDQ7Nt3vrHJHROqc51uc/TM95/qys32ziPy537Z63icuIs+LyCMVYOs2EVkvIi+IyFpnW+Q+B573GSciD4jIH0Vkk4icE0V7RWSO8zt1v94SkRvLbquqDvsvUiubXwNmAbXAi8ApIdhxHjAf2ODZ9i3gZufxzcA3nccXA78GBDgbaHa2jwe2Ot8bnceNAdk7FZjvPG4AXgFOiaLNznuOdh7XAM2ODT8HrnK2LwOudx5/HljmPL4KuN95fIrz+agDTnQ+N/GAfr9fBFYAjzjPo2zrNmBi2rbIfQ48tv0E+KzzuBYYF2V7nfeLA7uAGeW2NZAfKGpfwDnAbzzPvwx8OSRbZjJYCDYDU53HU4HNzuO7gavTjwOuBu72bB90XMC2/5LUoKFI2wzUA+uARaRWYSbSPwekWp+c4zxOOMdJ+mfDe5zPNk4Hfge8D3jEee9I2uqcexvHCkEkPwekmle+jlMME3V7Pef/IPCHMGytltBQIbMRwmKyqr7pPN4FTHYeZ7M5lJ/FCUecSepOO5I2O6GWF4A9wOOk7pA7NDXrIv19s83CKNfv9zvA3wNJ5/mECNsKoMBjItIiIkucbZH8HJDyjvYC/+GE3n4gIqMibK/LVcB9zuOy2lotQlARaErKI1fPKyKjgZXAjar6lndflGxW1X5VnUfqbvss4J3hWpQZEfkQsEdVW8K2pQjeo6rzgYuAG0TkPO/OKH0OSHlN84HvqeqZwGFS4ZUBImYvTj7ow8Av0veVw9ZqEYJCZiOExW4RmQrgfN/jbM9mc1l/FhGpISUC96rqg5Vgs6p2AE+QCq+Mk9Ssi/T3zTYLoxy2vhv4sIhsIzXC9X3AnRG1FQBV3eF83wOsIiW0Uf0ctAFtqtrsPH+AlDBE1V5ICew6Vd3tPC+rrdUiBIXMRggL70yGT5KKw7vbr3GqBM4GDjqu4m+AD4pIo1NJ8EFnm++IiJBqFb5JVb8dZZtFZJKIjHMejySVy9hEShA+msXWTLMwHgaucip1TgRmA8/5aauqfllVp6vqTFKfxf9W1Y9H0VYAERklIg3uY1J/vw1E8HMAoKq7gDdEZI6z6f3Ay1G11+Fq3g4LuTaVz9agEh9R+yKVbX+FVNz4lpBsuA94E+glddfyGVKx3t8BrwK/BcY7xwpwl2PvemCh5zyfBrY4X38VoL3vIeWSvgS84HxdHEWbgTOA5x1bNwC3Odtnkbo4biHldtc520c4z7c4+2d5znWL8zNsBi4K+DNxPm9XDUXSVseuF52vje7/TxQ/B573mQesdT4PD5GqpImkvcAoUh7eWM+2stpqLSYMwzCqnGoJDRmGYRhZMCEwDMOockwIDMMwqhwTAsMwjCrHhMAwDKPKMSEwjCyIyC2S6mT6ktMZcpHTGbI+bNsMw0+sfNQwMiAi5wDfBs5X1W4RmUiqi+UzpGq394VqoGH4iHkEhpGZqcA+Ve0GcC78HwWmAU+IyBMAIvJBEVktIutE5BdOXya3f/+3JNXD/zkROdnZ/jER2SCpuQlPhfOjGcZgzCMwjAw4F/Tfk2pp/VtSMwCedPoDLVTVfY6X8CCpFb2HReQmUquBlzrHfV9VbxeRa4ArVPVDIrIeuFBVd4jIOE31RTKMUDGPwDAyoKqHgAXAElItje8XkU+lHXY2qeEwf3DaX3+S1FARl/s8389xHv8B+LGIfI7UIBLDCJ1E/kMMozpR1X7gf4D/ce7kP5l2iACPq+rV2U6R/lhVrxORRcBfAC0iskBV9/truWEUh3kEhpEBSc2Sne3ZNA9oBTpJje0EeBZ4tyf+P0pE3uF5zZWe76udY05S1WZVvY2Up+FtHWwYoWAegWFkZjTw705r6z5SHR2XkGoX/F8islNV3+uEi+4TkTrndbeS6nIL0CgiLwHdzusA/tkRGCHVXfLFcvwwhpELSxYbRgB4k8ph22IY+bDQkGEYRpVjHoFhGEaVYx6BYRhGlWNCYBiGUeWYEBiGYVQ5JgSGYRhVjgmBYRhGlfP/AV/2kGjJBFS8AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "plt.plot(loss_monitor.loss, '.')\n", "plt.xlabel('Steps')\n", "plt.ylabel('Loss')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "通过如下方法打印量子嵌入层的量子线路中的参数：" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "array([-1.06950325e-03, -1.62345007e-01, 6.51378045e-03, 3.30513604e-02,\n", " 1.43976521e-03, -8.73360550e-05, 1.58920437e-02, 4.88108210e-02,\n", " -1.38961999e-02, -8.95568263e-03, -9.16828722e-05, 6.78092847e-03,\n", " 9.64443013e-03, 6.65064156e-02, -2.27977871e-03, -2.90895114e-04,\n", " 6.87254360e-03, -3.33692250e-03, -5.43189228e-01, -1.90237209e-01,\n", " -3.96547168e-02, -1.54710874e-01, 3.94615083e-04, -3.17311606e-05,\n", " -5.17031252e-01, 9.45210159e-01, 6.53367564e-02, -4.39741276e-02,\n", " -6.84748637e-03, -9.54589061e-03, -5.17159104e-01, 7.45301664e-01,\n", " -3.10309901e-04, -3.35418060e-02, 2.80578714e-03, -1.21473498e-03,\n", " 2.32869145e-02, -2.02556834e-01, -9.99295652e-01, -2.33947067e-05,\n", " 6.91292621e-03, -1.37111245e-04, 1.10169267e-02, -2.61709969e-02,\n", " -5.76490164e-01, 6.42279327e-01, -1.17960293e-02, -3.99340130e-03,\n", " 9.62817296e-03, -2.04294510e-02, 9.17679537e-03, 6.43585920e-01,\n", " 7.80070573e-03, 1.40992356e-02, -1.67036298e-04, -7.76478276e-03,\n", " -3.02837696e-02, -2.40557283e-01, 2.06130613e-02, 7.22330203e-03,\n", " 4.16821009e-03, 2.04327740e-02, 1.80713329e-02, -1.01204574e-01,\n", " 1.14764208e-02, 2.05871137e-03, -5.73002594e-03, 2.16162428e-01,\n", " -1.32567063e-02, -1.02419645e-01, 4.16066934e-04, -9.28813033e-03],\n", " dtype=float32)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net.embedding.weight.asnumpy()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 经典版词向量嵌入层\n", "\n", "这里我们利用经典的词向量嵌入层来搭建一个经典的CBOW神经网络，并与量子版本进行对比。\n", "\n", "首先，搭建经典的CBOW神经网络，其中的参数跟量子版本的类似。" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "class CBOWClassical(nn.Cell):\n", " def __init__(self, num_embedding, embedding_dim, window, hidden_dim):\n", " super(CBOWClassical, self).__init__()\n", " self.dim = 2 * window * embedding_dim\n", " self.embedding = nn.Embedding(num_embedding, embedding_dim, True)\n", " self.dense1 = nn.Dense(self.dim, hidden_dim)\n", " self.dense2 = nn.Dense(hidden_dim, num_embedding)\n", " self.relu = ops.ReLU()\n", " self.reshape = ops.Reshape()\n", "\n", " def construct(self, x):\n", " embed = self.embedding(x)\n", " embed = self.reshape(embed, (-1, self.dim))\n", " out = self.dense1(embed)\n", " out = self.relu(out)\n", " out = self.dense2(out)\n", " return out" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "生成适用于经典CBOW神经网络的数据集。" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "train_x shape: (58, 4)\n", "train_y shape: (58,)\n" ] } ], "source": [ "train_x = []\n", "train_y = []\n", "for i in sample:\n", " around, center = i\n", " train_y.append(word_dict[center])\n", " train_x.append([])\n", " for j in around:\n", " train_x[-1].append(word_dict[j])\n", "train_x = np.array(train_x).astype(np.int32)\n", "train_y = np.array(train_y).astype(np.int32)\n", "print(\"train_x shape: \", train_x.shape)\n", "print(\"train_y shape: \", train_y.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "我们对经典CBOW网络进行训练。" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch: 25 step: 20 time: 0.023, loss is 3.155\n", "epoch: 50 step: 20 time: 0.014, loss is 3.027\n", "epoch: 75 step: 20 time: 0.022, loss is 3.010\n", "epoch: 100 step: 20 time: 0.021, loss is 2.955\n", "epoch: 125 step: 20 time: 0.021, loss is 0.630\n", "epoch: 150 step: 20 time: 0.022, loss is 0.059\n", "epoch: 175 step: 20 time: 0.023, loss is 0.008\n", "epoch: 200 step: 20 time: 0.022, loss is 0.003\n", "epoch: 225 step: 20 time: 0.023, loss is 0.001\n", "epoch: 250 step: 20 time: 0.021, loss is 0.001\n", "epoch: 275 step: 20 time: 0.021, loss is 0.000\n", "epoch: 300 step: 20 time: 0.018, loss is 0.000\n", "epoch: 325 step: 20 time: 0.022, loss is 0.000\n", "epoch: 350 step: 20 time: 0.019, loss is 0.000\n", "Total time used: 8.10720443725586\n" ] } ], "source": [ "ms.set_context(mode=ms.GRAPH_MODE, device_target=\"CPU\")\n", "\n", "train_loader = ds.NumpySlicesDataset({\n", " \"around\": train_x,\n", " \"center\": train_y\n", "}, shuffle=False).batch(3)\n", "net = CBOWClassical(len(word_dict), embedding_dim, window_size, hidden_dim)\n", "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n", "net_opt = nn.Momentum(net.trainable_params(), 0.01, 0.9)\n", "loss_monitor = LossMonitorWithCollection(500)\n", "model = ms.Model(net, net_loss, net_opt)\n", "model.train(350, train_loader, callbacks=[loss_monitor], dataset_sink_mode=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "打印收敛过程中的损失函数值：" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA6k0lEQVR4nO29e3iU9bnv/fnNTBJOAcZwCBiSGLUpELo0QRLUonYVV3W5q0JtNX1X27dVpKvv9S6vrvVurbbubmy7XWvv9upa++0WD3V3dW2gqAi2bl31hOKJAEEwQIxiICEcEsAJBAI5zPPbfzzPM3lm5pnJJGSSOdyf68rlzHO8E8bv85v7d/++t9JaIwiCIGQenrEOQBAEQUgOIvCCIAgZigi8IAhChiICLwiCkKGIwAuCIGQovrEOwMm0adN0aWnpWIchCIKQNtTX15/QWk9325dSAl9aWsqOHTvGOgxBEIS0QSnVEmufpGgEQRAyFBF4QRCEDEUEXhAEIUMRgRcEQchQROAFQRAyFBF4QRCEDCWlyiQzhbV1rTz9TjOd5/sAONcb5HxvEKVAA1qDx3ptGOY5Xs/I7DO0+eNVgBr56wFMyPGSl+sFIM/n5eIp4wD47GwvF03MZeqE3LC/x/T8PJZVFlFV4h+xv7EgCIMjAj8EHn2pkd+/f5Dz/UZIGCNFMq75smOnEXFgvzGy+4J64H4jfb2uniBdPcHQvsOBcwMHHj+LG2vrWrl3SRkP3DzXdb8gCCOPCLwL9gj8+NkeunuCBA2N1uHibcQRayEaDaze0syx0+f59Z1XjnU4gpAViMBbrK1r5TebP+HY6fMEjcGPF4bHpl1H2HWok2sumyZpG0FIMlkt8I++1MiauhbO9gZHbRTutdI56ZiD14zMt5WDJ7s5eLKVddta+fLcmdx73aUi9IKQBJIu8EopL7ADOKy1viXZ90uEtXWtrPrTXs73D3+oHk9A83weZk4eR45XhSYdM2Wisb4lwIadbexv7+Jw5zl6XL7unOsNctaRo4+FoeGVfe281tjOz25bQG11cTJCFoSsZTRG8H8HNAKTR+FeMbmQFIzCHNnm+rxUzJ7M/TfNTXuhHi5VJf6Efvf6lgCr3/qU1xvbBx31Gxoe3NjApg/auHxmfkY8CAUhFVDJbLqtlCoC/hX4OfDDwUbwCxcu1CPtJrm2rpVfvdrEiTO9CZ/jVZCXI2I+EthC/1pjO4l+1LwexSO3VsiIXhASQClVr7Ve6LovyQL/HPBfgHzgH9wEXim1AlgBUFxcXNXSEtP5ckjUtwT4wf+q51hXT0LH53oVM/Lz+NsbLhdhSQJr61p5cGNDwscrYOk8yc8LwmDEE/ikpWiUUrcAHVrreqXU9bGO01o/ATwB5gj+Qu871BF7acEEfvn1K0REkkxtdTFvNnXwyr72hI7XmPn5N5o6WL9isfz7CMIwSGYO/hrgq0qpm4FxwGSl1P/SWv9fI32j+pYAj77cyIeHOukJJvaMmDcrn0duW+AqHPZE4glr9J8pE6Rjzb3XXcrrH3UQdEnKK+Dzhfl81N4VlsrpD2o27GyTv70gDIOkpmhCNzFH8K4pGifDycHXtwRY/th7ww8ujfF5wD8hlyuL/WOeynA+FDu7e0O2BZGTpmvrWvnxpgbXidfa6mIqZk/hoY0NYYvKbpw3kye+5foNVBCynjHLwTsCuJ4kCfwlD/zv+PYAQkqQn+flRzfPo7a6OKbI25OrQNh+mXQVhNjEE/hRcZPUWr+ZrBp4Eff0oKsnyIMbG/jWb+uorS7mZ7ctMNcSOAgamodf2EN5YT53LSqO2l7fEhjlqAUhvUl7u+C0/wWyjC2fnAiJ/J2LokfkQUOztfkkyyqL8HlU2PYNO9tGM1RBSHvSXh+bH/3rsQ5BGCJbPjnBF376Z+bPnhIm4mBOttaUFVBV4mfVrRWhUb4Gnqtvk1G8IAyBjPCiuXHezITL72zmFuaTP87HZ2d7KZs+acwnKWOxtq6V9dtbyfN56DrfT1ugG62hJ2jQl2DFUCpy+nw/D25s4KIJOQS6+0KpNgNoOtZFVYmf2upi9h45xdq6VjTQ129IRY0gDIFRmWRNlOGuZB2JShoFXFXqDzWrcFaCOBtY2NtT+aGQKjz6UiOrtzQP+bzLpk/ktb+/HjD/be964n16rYdZrs/Duntq5O8uCBZjstBpNKkq8VOYn5fwqlU3NLDtoMvX/xgNLPYfP8ur+9oTeijE2zecc0brXhda///AzXNZOr+Q/3fdTg53nk/4vP3Hz7K2rpXa6mKqSvzcsXCOjOIFYRhkxAge4KGNDaypa417jM8zYH8rJIZHQfnMfPqCRtSDIdEHQH1LgG88/j79Q/jDf/Hyafzb96pD58soXhDcGfMyydFgWWXRoMf0G5a3uce0KFhU6ufiqeNQg56ZvRgaGo91sf/4WbYdDPDKvvbQz5q6VpY/9h5fX/1e3MlPe8LUM4Q/9O5DnaFr2qN4+/Rg0GBr88kL+K0EITvIiBQNmCKwqNTvnmaJIGiYTSdaTnZz75Iyls4vDLMmgPhpjsOd5zjSeV5q8C22HQzwtcfei2sOZi9S+smmBhKZGz59vp87Vr/HsyuvpqrEz7LKIjbsbKO3z0AphT8ixSQIQjQZk6KBkbMt8CooL4ztVWPfK9GHQrx9qZqD7+zuZUdLYMjpLI8ibpemRFJpTpw2BWvrWnn4hT0YWpPr87DmbknTCELGT7LaDGUUH4+ghn1Hu4b/sIgxMRt333DOGe17JYDdpen1jzpc7QWWVRaxfvuhhPPxrzW2U98SoKrET6C7F0NrDG1Otm5tPikCLwhxyJgcvM39N80d6xAEzJWnP97UwNqI0XpViZ+7r70k4esYmtAK1pqyAnJ9HjwgaRpBSICME/iqEj8rl5SNdRgCpjj/ZFND1ARs/vic0ISrvf4gHvvbuwDz3/bhW+bj8SgMrVn14l5Z2SoIccg4gQez/rpo6riY+xXmStaJud7RCypLCWr44fpdYdtqygpCFgUa2N12iiWXT4t5jR0tgZCQu6VpBEFwJyMFHuBvb7g85j6NWfp3vt/gF7cv4OCjf82G71/N0nkzmTYpl5xIm0Phgmj5rJulv3wz9D6y7LGv36DoogmUXDTB9XxDw+NvfQoMpGm8CnJ8HmrKCpIcvSCkLxlVRRPJt35bx5ZPTsQ9RgE/v32BeI3HwW6cve/IKcsDx+DM+X6CxtCus+Tyafw+zuKln/6H+TH7tnoUoZLJtXWtvLznKDdVzJJ/NyHrGfOGH4ky0gL/m837+a9/bkro2EWl/qjuQ0J87FaJ24dQtbRySRkP3GxOhD+0sSFkQeBV8MMby9l9qDOmcdyNVp39N5/aSm+/IaWSgkCWrGR1o6aswPUXzHVJwWw7GGBNXStfe+w9vvyrt6KqP4Roqkr8PLvyajZ8/2rmFuYndM7qLc2hfPqyyiLycsKrYu697tKY577+UQfP72yjt9+QHLwgJEBGC3xViZ8VLhU1vUEd055AA/s7zvDgxgZu+vUWVvx+Bw9tjK4EEQaoKvHz8n1LmDcrMZF/9OXG0HmRVTEAhZPzXM8zDI0GKZUUhATJqIVObuSPz3HdrjH7hJ7pCca0HGg81kXjMbNEb9221ijTrQt1W8w0HrltAXesfm/Q1a/bDwbiLl667YqLXW2GNTA5z8fDt8wPrWhd9eJeygvz5d9AEFzIeIGvKSsg16tCk3lOunqCgJl/B1N4YmmTbboFhK32XLetlYWWuAxmD5DpOf6qEj93LSpOyIrAtvy1q2KcHjM/uOEyfvfeQc73R8/iPvnOAe68ao6saBWEBMh4gbdL8uKJzraDAX5x+wLuv2nukCcNDR3hIx/HHmDbwUDogWD7vYyl3/twrjfYtxbTiqAVF20Ow/bwsdM0kSPy/HE+zp/pjTov6EjT9PUbUiopCHHIeIGHxPxPHtzYwC9uX8CzK68OMxIbrulWLKIeCJA6XjMJXs8tXWUzPT+PL31+8BaKnd0D4u2Wprmy2B/zGpPzfKy5u4YNO9vE6lkQ4pAVAm/7kQ9mVfvgxgY2fdDG/TfN5Re3LwhttwV/f3tXaEQLjKjwpxOx0lU2yvqJ96dx5uHtNI1zRO6fkBtT4J965wDFBRNDFTUbdrZJuaQguJAVAg+EGjgPlh/edtC0HJ4+KZcri/0h21s38Ygc6cdLc+R4PTS1d2XFAyGRX1EDq9/6lCe/tZCqEn/UiDzQHZ2esQkampf3HI0qlxSBF4RwskbgYWhWtcfP9IY6F5UWTOCay6ZF5Z5jCX8sIj3k0y0HP9LpqgPHz4S9d47IH75lPh7l3l5RAwUTcyUPLwiDkFUCn2iqJpKDJ7s5eLKVNXWtTB3vQyswDNOw7P6b5iYs8kN9IKQibukqZ9PxeJVIkTSfOBtK02xtPhk2Ig909/Kz2xbEtC548cOjrLq1gj1HTkkeXhBikFUCD+6pGgUUXzSBls+6Bz2/81x/6LWdzpk63ofP56EvaHC+1ywfGZ/jwecbWEfWFzQwDCiaOp45EaZakSPlVK6vH+whtbaulR9vakholG/ogTSNWx4eQClwc9MwtGbPkVOShxeEOGSdwEN0qkYDbYFufnH7AjY3dbDtwElOOYR8MDpdju2JUSfoXDwVhWPCck1dK5NyvfRZMbo9MOI9TIa6ry9o0NevmZjr5ZJpE4dds19bXUx5YT6r3/qU1xvbBxV6O03jloff2nzSVdwBfF4PCiQPLwhxyEqBt1M1D21sCKUTgho2N3XwpKP/569ebeKESy32aHGmNxh6HeuBMdL7unuDHD/TG/LmsVNStvg7J55jUVXi58lvLaS+JcD9z+1mf5zSzJbPukNpGojOw3sVrum0/qBBfp5P8vCCEIeMdpMcjBW/3xFWiqcU/Py2cOtgZ875cOc5eiyr3PN9Q/TKzTDmFuZTWeIfdJRf3xLga4+9FzMvr4B/+KtyfnDDZfxm835++UoThh5wl/RPyI2Zh/d5FKturSDQ3UtNWYGM3oWsJGuabg+Ve6+7lNc/6iBop2o0/HiTKSa2yMfKOa+ta+Xpd5rpPN8X2hYvNZJpDwU71bS2rpWllo2v29+pqsTP0nmxFz5poOuc+TeMlYf3enD1nje0Dom77SopIi8IA2T1CB7cJwUVxBWtC7nX+u2t5Pk8cUsVAQ53nuNMb/+I5tnj7TvXG+Rsz0BKaKh4PfDMvVfHXC8QbxT/xcun8W+ORiBbm0+GRuTxPP3tJiGrXtwr/vBC1iIj+DjYI3WnyGvglX3tvP5RB4/cWjFiXYNqq4tTugNR5MItOyWViPgHDfjxxgZevm9J1L6qEj9fKJrC7rZTrucWTMwNO9Yp0PHsgP+iaAqB7l6ZaBWEGGS9wIO7yIO5YvKhjeEpm0wmXgmkLf4ftARiVgE1Huvi0ZcaQx2bnCwuK4gp8C9+eJS/WVxKVYk/agQf6O6NueBpZ2snt19ZJBOtghADEXgLW8AjF0FpTI+aN5s6Rjxlk044xd/u0fqqS1798bebWTq/MOrvFMuXH8wHqZ1Dj2zHZ+fl3eYvDMPMwa+5uybsoSAIgklGd3QaKrXVxTyz8moumzEpat8r+9q5Y/V7fH31e1nf4ckug9zw/auZPik8haK16fUeSU1ZAb4YnzaNmYqJXM1qp1u+s7jU/URFSNTtidZs/ncRhEhE4COoKvHzj8u/gM+lb6tt9bumrpXlj5lin82CUlXiZ/XfLMQT8adav7016u9SVeLnG1fFTnPtOXIqNFr3KsLSLXuPnnY9R2t4de8x6lsCfPOprfzylSa++dTWrP43EQQnIvAuVJX4Wb9iMUvnzYwSLyfbDprVIV9f/V7W9m61uzg5CRqmBUEkyyqLYn7gnt1xCIA1d9fwwxvLw6ph5s+aHPP+T7zdLI24BSEGkoOPgXM1Zrxl95rwBh5r61qZ7R/PxVPGRZVCprLHzIWwrLKIP2xrDZu7eL2xPWyFKph/07IZk9jfcSbqGv1BMw//gxsuc83fx/KX1xrp8CQIMUiawCulxgFbgDzrPs9prf9Tsu6XLJxCP1gVCZhiczhwjsOBc67719a1clWpPyk2v2P1AKkq8fNIhPOjoQf6rjr57jWXuK5M9XlVSJgjK2lqygrIy3GfaPV5Fcsri1heWSQTrYIQQTJH8D3Al7TWZ5RSOcA7SqmXtdZbk3jPpOGsIrFXsX56/GzC1rg2kSP+MEag1V68dnoQ+4FxoQ+H2upi3mzqCFuxavveRx739LsHokbxQUPTZD04Iytp7InW1Vuao67XHzTPs6ugZEWrIAyQNIHX5hJZ+//iHOsndZbNXgD2gqXIhUFD8UJPFoO10wvhss+ZXhqOm+S9113KG00d9Fu5mjeaOqLSNOA+ijc0PPzCHr5x1RzXSpqYE63AT17YAyArWgUhgqTm4JVSXqAeuAz4jda6zuWYFcAKgOLi9FpMFLkwyM7XHzh+5oIaYYwVzvTStoMB1m1rZWGJP+H0T1WJny+VzwiN4vuD2jVNU1tdzL/vOcqWT06EbQ8aOmY+ff6sybwdcbyNIS38BMGVpAq81joIXKGUmgpsVEpVaK33RBzzBPAEmF40yYwn2dj5ejci2/U5GYkcPIx8E3C7LNRm3bZWfhbhthnJ9Py8sPduv299S4B390eLtVLEzKfHm2jN8Xm4qWIW2w9+JhOtguBgVKpotNadSqnNwFeAPYMdn4mMRru+eO30bNweGIl+wzA0PLQpvnXDssoi1u84FDdNs7X5pKvHu+175/a3ijXRetHEXP7hxvJQoxGZaBWEAZJZRTMd6LPEfTywFPjHZN1PuLCHiDO9lOP10NTe5V4W6mKpHBnDYGmamrICcr2K3giV15gNP2LZDj98y/ywJi0An53t5SdWPOWF+UP8rQUhs0nmCH4W8K9WHt4DPKO1fjGJ9xMugMj0kvPbQOTo3hhE5CPTNJFrxapK/KxbsZhHX25ke0RF0cftXaH7R47G32zqcP2WEbTi8Xk99AdlklUQbJJZRfMhcGWyri8kFzdzsdca20NplHgi70zT+LyKZZVFrte/vnxGlMDXtwRYW9fqWhHTfvp8zHjtyVWNTLIKgo1YFQiDYo/uf37bgjDrBlvk19a1Rp3jUeaBWhOqb4+kpqwganRvaNPLxs16IJ6Xjc+ryHHxsRGEbEasCoSEcfPNt+vXywvzQyPmrc0n6bMaegcNHbXfycJSf9Qofu/R0/i8HoLB8IqY2upiWk+ejVrwlD/Ox+KyAq4vnyH9WQXBgQi8MCTcRD5ohE+k1pQV4PUo+q0DDK2jUia2A6Sb/YA2NF9bNIeLp46PEmu3csmu8/28sq+dNz8+zrp7JPcuCDaSohGGTG11MT+7bQG2o7IGnqtvCzlpVpX4WXVrBV5lTrD6PCoqZWJ7v7uhgYrZU1yNx/wTcmOWc/b1G2zY2cZvNu/POldPQXBDBF4YFrXVxdzpsAnuj7DpLS/MR3kUGnCTcdv73Q1Dm7YDbiId6O6NytvbKGU+aMQXXhBMROCFYTN/9pTQa4PwBtkbdraFFjvZtfBOqkr8rLm7htrqYlfB7ukzXDtDxRrBK+DLc2fSHxRfeEGwkRy8MGycDbE9ynxvEynabiJul2JWzJ4SZT6mgWd2HEJBmAeO8x5OPl+Yz7T8PNfJWUHIVmQELwwbO83iwSyLdI7gl1UWkevzoDDNw9xq4W02N3W4bu8PatbUtYalW2rKCnDppkjjsS7W1bWC1ty5qFgWOgkCIvDCBWDbB3g8CkPrsLx5VYmfdffU8OV5M5lbmB+zFh7gwIk4tsZAryPdUlXi5y/nznQ9TgP9hmb21PEi7oKACLxwgQS6ezG0ds17Nx3r4tV97exuO8WDG90XRNW3BAZcxmLgUeFVOPdedym5bsN4l2MFIZsRgRcuCGeaRkWkaV7eczTs2Mj3di38/ojmI0sun8a4HDO941Fw97WXRPV2XbdiMZdNnxh2nlKw6tYKACmVFARE4IULJF6a5qaKWWHHRr6PVQv/9v4TfGV+IV7LF+F37x+MEuumY118GvFguKrEz54jp7jrya1SKikIiMALI0CsNE1tdTG/uH0Bl02fyGUzJkWd5xz9O9EaXth9JGbqp74lwE82NUSVS+5oCbCuzt3HRhCyESmTFC4YW6hjdVOyUzB2KaRtd2DXwm9tPsmLu48M9JLFFHmvR4HWUamfWA1DnP71CjEdEwQZwQsXjC3UP7yxPKo8cbA8fFWJnx/ccBlFF00I227n3pXlafPTP+0NK5WMNckK4FXmQ0RKJYVsRwReGBGqSvzUlBWwtflkWN57sDy8zYyIJiFfnjuTrp5+gtawvLff4HlrZas9ybp03syoBVQeBY/ctoCf375AxF3IeiRFI4wIdkVMZJMOOx3z9DvNZplLDJZVFvFsfVto0rXTZcWqMytTVeLnijlTedVqDejcHujuZW1dq1gHC1mPCLwwItgVMc7JTaewxsrDw0B7vpsrCtm06wgA2w4G8HrA64GgAT4PLI9YDeuPaChuX2uH1WLQo5D2fUJWIwIvjAjxJlrd8vC2wDtH/pHrnYKGOdGq0Hg80dlEN18a50RrrIeNIGQLIvDCiOCsiIlMi9xUMYu3PzkR9t7GOfKPRAFa61CfVWdTkfqWALsOdcaNySPt+4QsRwReGDGcjbqdxMvDO0f+QFj548JSP7sPddIb1KGmInaa5ptPbaXHpRuUzdJ5M7lizlTJwQtZjQi8MGq45eGdI3//hFx++sc99Foqf/HU8XxuZj5rLA8bZ1OR3n4jZmcnn0ex8rpLRdiFrEfKJIURpb4l4OoDE68e3q6Fr60u5uYFA+mbTbuOcOiz7tB7u6lIrBWwYH6gv/T5GSPxqwhC2iMCL4wY9oSpmw9MovXwkXn1Dw+fwrKkCTUVsUf911w+LbTPxgBe3dcuPjSCgAi8MIK4lUra2L40X7x8Gr+4fUFYmaST4ogVraUXTSDX58FrlTzaE6ZVJX7u+/Ln8EUqPGa9fK804BYEycELI8dgnjS2qNvpGbda+FPn+sLOyR+fw5q7a1j91qd0nD5P07GuUG69qsTPHQvnhHL0kTxX30Z/0JBaeCFrEYEXRox4pZIAa+taQxOsdtlkbXVxWC185IB8/qzJocYhALvbwhdKRa6AtZk9ZRxHTp2XWnghq5EUjTCi2BOmbmIaa6LVmdrRGq4qHTj3qXcPsH57q+t59S0BNuxsw3DpCDUux4vPa6Z2pBZeyFZkBC+MGrEWPEWmdpyD+P6gjhqd31QxKzTq7+lzL5dsPnEWn0dx56JillUWyehdyEpE4IVRw5mDv6lilqsvfE1ZAav+tDfsvFyfh1/cviBsoZQ96o9VC29oCEoDbiHLEYEXRhx7wtQtDx9rotW5CnZxWQG7206FzllspVecC6VWLikLjfqVMn1rIsXe65XUjJDdJCTwSqmJwDmttaGU+hzweeBlrXXfIKcKWUYs22CbWBOtTk739Ie9//TEWfYePR22be/R02ErYH+yqSGqy9N1n5sOmA24xbJAyEYSnWTdAoxTSl0MvAL8DfC7ZAUlpC/xauEh/opWexXsia6esGPe+KiD+bMmh227qWJWaEI30N3r2sLvwPEz0oBbyGoSTdEorXW3Uup7wP/QWv+TUmpXEuMS0pTBauFjTbQ6R/4+j8KjBqx/g4bmdE8/OV5FX1CT41WUF+aH3dN5vM2nx8+G0jZSKilkIwkLvFJqMfBN4HvWNm9yQhLSmcFq4WNNtDpH/kFDUz4zP6wJ9/72rlD7vv6gDrMOrirxs+KLZaze0hw6XjGQk5cG3EK2kqjA3wf8CNiotd6rlCoDNictKiGtiWUbbFNemM+eI6fYe+QU9S2BUD9X58i/L2I4/tnZXnweFbIOfmbHoZB18NbmkyydXwjApl2HQUP7mR6Uhhyv4o6Fc6RUUshKlHZZJBL3BKU8wCSt9elBDx4iCxcu1Dt27BjpywopRH1LgLueeD9kCZzr87DuHnMi1ll9s/qtT8P6rdoNtl+J2Pb2J8dDaR0Dc3TvZOWSMh64ee5o/GqCMCYopeq11gvd9iU0yaqUWquUmmxV0+wB9iml/r+RDFLIDrY2n6TPIcLOiVjnKtiV112Kz/p0ehTcUD6D6fl5YdfqOH1+YEI3qKPEHYiqvhGEbCLRKpp51oj9NuBl4BLMShpBGBI1ZQXkeAfWqsbKjVeV+Ln72jI8yrQvWPXiXubPnkKupfpKwSXTJoacJnO8KsrHBmB8jleqZ4SsJdEcfI5SKgdT4P9/rXWfUipubkcpNQf4PTATc77rCa31P19IsEL6U1XiZ92KxWzY2YaCmLnx+pYAT71zIFQZ09NnEOju5btXl7J6SzNamw1BVi4pI398DjVlBby691jYRKtHwWuN7bzZ1CF5eCErSVTgHwcOAruBLUqpEmCw7779wN9rrXcqpfKBeqXUq1rrfcOOVsgIYk3COnPwW5tP0u+YaNWY3Zwi6+r3Hj3Nv32vGiBqn316b1Cztq6VDTvbxDZYyCoSEnit9b8A/+LY1KKUumGQc44CR63XXUqpRuBiQAQ+C4hnVwDmilZnqWTkCtjvLC6NOmfPkVNRdfTOBVD+Cbkx49FILbyQfSRqVTAF+E/AEmvTW8Aq4FTMk8LPLwWuBOpc9q0AVgAUF7t3+RHSi+HYFQS6e8NWwLpNjp7o6qG2upjWk2d5/G0zTfP0ewdZOr+QqhI/ge5e13i8ViJfauGFbCPRSdangS7g69bPaeB/JnKiUmoSsAG4z620Umv9hNZ6odZ64fTp0xMMR0hlhmNXYNfB2/7tN1XMItcbPmv65sfHqW8J0NXTj13d29tv8Phbn/KbzfvxT8hlXE70R7pi9mR+eGO5pGeErCPRHPylWuvljvf/ORGrAmtidgOwRmv9/DDiE9KQ4dgVuK2ALS/MZ9Wf9oacJYNB82ERObv/WmM7rzW2h1I7z+1s48SZgdF8nhWDiLuQbSQq8OeUUtdqrd8BUEpdA5yLd4JSSgG/BRq11r+6sDCFdGK4dgWRk69VJX4WlxXQcPgUWoeXVD6zvRW7D0hoMrXP4Mm3m6OMx7YfNFNGMoIXso1EBX4l8HsrFw8QAL49yDnXYNbKNzhG+w9qrV8acpRC2jGYXUFtdXGUTTCET842HesKK3v8zuLS0DW/cVVxWLNtBXg8KqzyxkYmWIVsJdEqmt3AXyilJlvvTyul7gM+jHPOO4DL0hNBMImstImcnC3yTwg7/n1HLn/+7Clh+64q9XPblUWuvvAKc6L1SOe5kPeNIGQDQ2q6rbU+7Zgo/WES4hGyBNuT5r/9uYm7nng/JPbOyVkifJKcvVkjK2Z2tnZSXpjPI7ctIGJuFg0ENazb1iq+8EJWMSSBj0BG58Kw2bCzLeQM2WvZ/0ZW0nz32rKwD2jjsS7WWmmZmrICfA5vAkNrtjafpLa6mGdWXs1FE3PC7hc0dMyqHkHIVC5E4IdmQykIDiJHB4qByVm7pLG2upgFReGpGLvEsqrEz6pbK/B5lJl/VypsoVPgbHg3SfMY8YUXsou4OXilVBfuQq6A8UmJSMgKllUW8Wx9W6iUcpnl7R45ORvZgNu5crW8MJ8bPj+DNz7qIGhoHn5hD2CueHX70HqU4uFb5ksOXsga4gq81jo/3n5BGC5VJX7W3eNeSumcfI1swG2/tydke/qMkJj3WyJfWTw16n4a0FrHXO0qCJlIomWSgjDi2KLu9IOPrKRZcnn46ma7Ibc9IRs5Uu83tOskqrTtE7IREXghaQxmOGZX0tiNtNetWBxVSTMtPw+fV4Waedh2Bc7VskBYaaRLKTwoJD0jZB0XMskqCDGxR+K/fKUpZmliIpU0FbOnhOXd+4MDC5bsCdk7FxWHJm3tydRItDYnaKVEUsgmZAQvJAU3w7HI0XO8SpqtzSfxT8hl1Yt7Od83UP9u6AFbYPt6G3a24fUqgtY3gevLZ4T1brV5+5MT1B34LNQDVhAyHRnBC0khciTulvteVllErs+Dwmy+7ayk+cENl4UshCPZ3NQBDHxLWFfXSr/1TQBlCnykE6VNb7/B8zvbRurXFISURkbwQlIYzHDMPmbdPTWh9n1O6lsCHO48h8/riRL5A8fPAO4TrX39BnuOnOKOhXPCvGqcyAIOIVsQgReSxmCGYzbP72yjt98ItdQDQpU0Po9iUamfbQcHcuctn3WHTbT29hnYjwANPLPjEKu+WoHPA5FfAHK9iuXWNwVByHQkRSOMKW65eue2oKG5rnwGN86bGTrHMHTYROvf/1U5i0oHHiT9Qc3GD9qiqmm8HsVPv1oh+Xcha5ARvDCm2J4yfUGN16NCufrIhiE1ZQW8+fFx+voNvN6BnL79LeFI57mwUf6OlkCkVxlBQxY6CdmFCLww9igFaOu/7vn7+pYAQcPMtweN8LxLfUsADeR4zQeFUlFGlCG6zvW57xCEDERSNMKYsrX5JP1BI9SUY4NV4WJX0tjifv+GDwlauh404PG3PgUGKmn+sK0VrbXVYNs8ThFdivnE280hR0pByHRE4IUxxWn7q4Hn6ttCi5HqWwI8uLGBu57cyv6OM2Hn7T1iGpCF5+vN/LzG/GB/oWgKnohPuKHh4Rf2yIInISsQgRfGlKoSP3csnBMaaduNtZ017m618Ic7z7O2rjW83t6ryPF58GC278vzeUKjfie2d7wgZDoi8MKYs6yyiLyc8EVRkTXubumWl/ccDbMsWLdiMd+9uhSUOaEay3QsV0zHhCxBBF4Yc6pK/Dx8y3wqLp4Sco90jsxzvYra6mLuXVIWdp7tUWPn6wGeeucAhjbTPYZ2aTsmpmNCFiFVNELSGMxN0nncT/+4h96gBk7x5sfHWXdPjetK2GOnz7Np1xEAnn73AEvnF4YmYn/92scEI4rfp03K5fgZR2mkju7nKgiZigi8kBQifd3X3B3b4Gtr80n6HH6/9oIne1TuzJf/0RJ3MB0oV7/1KSuvuzSq+QeYo/gwccfMzTtb+wlCJiMCLySFRNwkbWrKCsjxKmsEP5CHdz4kfB5F8UUTiJwzfeOjDmbk54Xy9R6guGACLSe7XT1nnK39aquLR+z3FYRURHLwQlJIxE3SpqrEz7oVi6mtLubGeTO5o8r0inE+JHqDmv3Hz0ada5dFhvL1OR5WLLmUnBhukjDQ2k9KJYVMR0bwQlJIxE0y8ngYMBnbsLONh2+ZT67PE5V6caKByXk+13v9ZFNDWKcnJ0GHn40gZCoygheShnM1aiI8v7ONnr6BtE6gu5c1d9dwV3VxaITu80Dh5Lyw85565wBA2L0C3b1xbYGdvjeCkKnICF5ICepbAjy749BA3bslwM6Hw4muHt78+DjHTveEndtv6JDFge0tP3/2lJg9W5WCVbeKq6SQ+YjACylBZCWNbRbmnGj1KIURw0XsD9taWb+9NbRyNdfn4af/YT57jpxiW/PJsPz9pdMnUV6Yn7TfRRBSBUnRCClBTVmBaRRmYVij8khv+FgukbYXjY3d2em5HYeiJmf3d5zhrifdG4ELQiYhAi+kBFUlflbdWoFd/GIbj/kn5JJr+ctoEm+3l+PzcKKrJ1R6GYn0ZhWyARF4IWWorS7mzkXFIXsBexS+5u4aFhRNSfg6l82YxLp7apienxf3uI6unrj7BSHdEYEXUor5s6fgiRjFNx3rovHo6ZjnRBqRHThxlqZjXcyfHf+h8NbHxyVNI2Q0IvBCylDfEmDVi3vDKl76+g3Wb28NTcAq4KpSP851TDk+D0sdPVuDhuahjQ1s+qANT+z1TiFrYkHIVETghZTBnlB1ooE9R06Fcu8aCHT3hTXU/ouiKdx73aWhxiH2cdsOBvB6VMiR0rm6VTH4CltBSHdE4IWkUt8S4Deb9yeUCnHaGzhH6JFNO/Z3nAmbbN1+MEDTsS7uvvaSqGuOy/Hwl3Nncn35DK6cMzUs/fMVy4lSEDIVqYMXksZQHCVhwN5gw842TnT18EZTB/2xvAYiWL+9lRvnF0Zt7zof5JV97a7nbNp1hEWXFIjpmJCxyAheSBpujpKJ8PzONl5rbEcbiRZFmmkc/4RccuOYjLmxfrs04BYyFxF4IWkMxVHSxvlQ0JqYk6QKmDp+4Ato0IDNTR2sW7GYpfNmxp1cdfJh2ynW1onIC5mJCLyQNJz9UgdLz9iEterL8bDii2XRbfcwc+id5/rDtr3xUQcAK6+7lDsXFbOo1O96buR1HtrUICIvZCRJy8ErpZ4GbgE6tNYVybqPkNpUlfiHNJFp92d9ec9RbqqYRW11Mc0nzsbMozsJGmaHp7c/OU5Pn4HXo7h3SRn1rQF2HAzEthzW8PALeygvzJdJVyGjSOYI/nfAV5J4fSEDsWvh391/glUv7qW+JWCWQCaYW3+9sT3kH99vaJ565wCfm5mPGuT0oMORUhAyhaQJvNZ6C/BZsq4vZCaxWv2tt3Lrg8m8oQkTc0ObHZ98gyTl7VWzsrJVyCTGPAevlFqhlNqhlNpx/PjxsQ5HGGPsHLwHUGqgQXZViZ8nv7WQ575/NYWDeMxce9k0fB6FR5m2wcsri7i+fMag9+6zOkkJQqYw5gKvtX5Ca71Qa71w+vTpYx2OMMbYOXiPx/R+t9M0zv1/c3Vp3Gu8++lJVt1awd9bk7sAb3wUO4fvtf4vkFG8kGmMucALmc9QVrOC2W7P0BpDQ09f9Ki6pqwgbr170NCs396Kf0IuW5tP8vzOtqjVsGE4Zl/Fn0bIJGQlq5BUhrqaFUwB93kUvUEdGlUvrywKnVdV4uenX63gwY0NMa+xu+0Uu9sa8Cgz/+71qtCqWI8izMsmqE1rBENLr1Yhs0jaCF4ptQ54HyhXSrUppb6XrHsJqctwVrNWlfi5Y+GcMF/4yFH83iOnErq/3Qnq6wvnUFtdzI3zZnLRpNzoA+2Z2cHKbQQhjUhmFc1dWutZWuscrXWR1vq3ybqXkLoMZzUrwLLKopD7o1tuPHETA/O+yyuLWF5ZxBtNHZzo6o06xjDMbwuSohEyCcnBC0llOKtZ7fPijeKXVxbhS+DTq4DvLC6lqsTP42996mpe5rEePpGVO4KQ7ojAC0mnqsTPD264bMirRCNH8eu3HwqzFPB4Bv/4amD1lmbu+8MHvN4Yu5Lm5orCmJU7gpCuiMALKYs9ircJGpqHX9hDfUuArc0n6euPVxoTzqZdR4jlPGxoeGH3kVDlzlCcLwUhlRGBF0aFoZZK2iyrLApbhWpbCtSUFSTsGJkIpnOlGvJcgSCkMiLwQtKxSyV/+UoT33xq65BEvqrEz6pbK0IdnjTwzI5DAPzl3JmxTwRKLpowpDgrZk8e8lyBIKQyIvBC0hlu4w+b2uriMDHvD5qukYOZkB0+dW5IIr+r7RRd5/rY2nxScvBCRiACLySdWP4yQ2F6hP/Ma43tNB3rYtVXK0KpmsgS9v6gpvWzbrwK8vO8gxqVgTkhO5xvGoKQiojAC0lnMH+ZRFhWWYTXkXTXGh7a2MD/2PxJaFWqdplE1ZgrVbt6gqHaeQWUFkxgbmG+671kolXIFETghVFhMH+Zwagq8fPIrRVhE6saaOs8P6TrKCAvx8MVc6bSeKwr5nFer0y0CumPeNEIo0Kkv8z67YeomD2F2urihK9hH/vjTQ0MoR83CjN94/Mo7lg4h/w8H6u3NMc9p3iIE7SCkIrICF4YFdxq2h/aOPReqLXVxfzstgXEa/DkbMYN5khfAdeXz2D+7Cn8+95jg95nf8cZ7npS8vBCeiMCL4wayyqLwoR5uA2va6uLeWbl1SydN9O1Fr7HZQFUUMMr+9p5cGMDB092J3QfycML6Y4IvDBqVJX4o2rXtTZTLkMVebvD07Mrr+aiieFVOef6jEErZhTE9ZQH8HnFOlhIb0TghVHFrXbd0IQsCIZKVYmff7ixPGp7ZIreOdJXQI5X0R8jka8cP4KQzojAC6NKWANth4LaFgTDoba6mCWXT4t7jFPLPQqmTx4Xc6JWWz9BQ0uKRkhrROCFUcdOr/zcMVl6of1Q/+7Ln0v42KCGw4Fzgx4n1sFCuiMCL4wZtdXF3LmoOG7npkQZ6ZG2R0G/ofnpn8Q6WEhfROCFMWWwzk2JYtfZJ8pgR9rpm95+g+eH+dARhLFGBF4YUxLpv5rode6+9pKEj585OW/wgyyG0h5QEFIJEXhhzBmpUXz++JyEjlPA8TM9CR2X61UsrywaciyCkAqIwAtjTuQofriNr2vKCgatbQe7Qmbw69VWF7NuxWLxhhfSFhF4ISVYVllEXs6FWQpXlfi5vnzGiMV0vKtHvOGFtEYEXkgJRsJSGGBafuK59cF4ZV+7eMMLaY0IvJAyBLp7CRqmpXBv3/DSNMsri0Z0BaqhzUoaWfAkpCMi8ELK4J+QG6pYMaz3wyGys9OF4lHiSSOkJyLwQsoQ6O4daL8H7DlyasjX2Np80rWz04VwyxdmyUSrkJaIwAspg3Ox0nDLJWvKCkIllyPF9oOfjej1BGG0EIEXUoaRKJeMbCwyEhzuPM99f/hgRK8pCKOBCLyQUoxEueSyyqKE6uGHwqZdR4bsWS8IY40IvJBSOMslg4bm4Rf2DKsZyEiP4gGefid+H1dBSDVE4IWUwy6X1JiOjsPp+BTZHnAk2H/8rIzihbRCBF5IOWrKCvA6nCGNYbT1qyrxU3HxlBGP7cGNDbLoSUgbROCFlKOqxM+qWyvC2uwZGh7a2MCK3+9IWGC/cVVxUuK764n3ReSFtEAEXkhJaquL+dltC8JEXmPaB9yx+r2ERvO11cXcdsXsEY+tN6hZ/th7UlkjpDwi8ELK4ibyYI3mE0zZ/PrOK5lbmJ+U+DbtOsLlD70keXkhZVF6pJf9XQALFy7UO3bsGOswhBRjbV0rD21siGq8oYCl82Zy73WXxl1pWt8SYPlj7yU1RoDSggn88utXyKpXYVRRStVrrRe67hOBF9KBtXWt/HhTQ6iVnhMFXDpjEt+95hJqq93z7o++1MjqLaNX5uj1QPnMfB65bYEIvpBUROCFjKC+JcDqtz7ltcb2mH4zcwvzqSzxs6yyKEpY7/vDB2zadWQUIo2NCL8w0ojACxnF2rpWfrKpgWCcj65HwUJLQD8720vZ9Ence92l/Nv7B8dc5IdCrleR6/WgPIq5hfncf9NceTAIYYyZwCulvgL8M+AFntJaPxrveBF4IVFCo/l97Qk3xVbAVaV+9nec4bPuvmSGJ8RBYT6AUTAhx4vHq+jr1+R6FT6fhzyfl4unjGPqhFym5+e5fhsTBhgTgVdKeYGPgaVAG7AduEtrvS/WOSLwwlCpbwnw6MuNbD8odelCeuMBmh/96yGfF0/gk1kmuQjYr7Vu1lr3An8Abk3i/YQspKrEz7Mrr2bD96+mtrqYRaX+qLJKQUgHDKDsgf89otf0jejVwrkYOOR43wZURx6klFoBrAAoLk7OykMh86kq8Ye+xte3BNiws40TXT10dvdyuPMcRzrPJ5zKEYSxwhjh6yVT4BNCa/0E8ASYKZoxDkfIAJxib+MUfZvO7l4+O9vLRRNzmeqwJe7s7qX5xFlOneujL95MriCMMCOdUkmmwB8GnJ6tRdY2QRh13EQ/ESIfDLEeCrH2dXb3svfIac72BlGYZZLK0TTWbjJu79OAYSDfNrKQ4ebg45FMgd8OXK6UugRT2O8EapN4P0EYcYb7YBCEVCBpAq+17ldK/T/AnzHLJJ/WWu9N1v0EQRCEcJKag9davwS8lMx7CIIgCO6Im6QgCEKGIgIvCIKQoYjAC4IgZCgi8IIgCBlKSrlJKqWOAy3DPH0acGIEw0km6RQrpFe86RQrpFe86RQrpFe8FxJridZ6utuOlBL4C0EptSOW4U6qkU6xQnrFm06xQnrFm06xQnrFm6xYJUUjCIKQoYjAC4IgZCiZJPBPjHUAQyCdYoX0ijedYoX0ijedYoX0ijcpsWZMDl4QBEEIJ5NG8IIgCIIDEXhBEIQMJe0FXin1FaVUk1Jqv1LqgTGM42mlVIdSao9j20VKqVeVUp9Y//Vb25VS6l+smD9USlU6zvm2dfwnSqlvJynWOUqpzUqpfUqpvUqpv0vVeJVS45RS25RSu61Y/7O1/RKlVJ0V03qlVK61Pc96v9/aX+q41o+s7U1Kqb8a6Vgj4vYqpT5QSr2YyvEqpQ4qpRqUUruUUjusbSn3OXDcZ6pS6jml1EdKqUal1OJUjFcpVW79Te2f00qp+0Y9Vq112v5g2hB/CpQBucBuYN4YxbIEqAT2OLb9E/CA9foB4B+t1zcDL2P2eagB6qztFwHN1n/91mt/EmKdBVRar/Mxm6PPS8V4rXtOsl7nAHVWDM8Ad1rbVwPft17/LbDaen0nsN56Pc/6fOQBl1ifG28SPw8/BNYCL1rvUzJe4CAwLWJbyn0OHLH9K3C39ToXmJrK8Vr38wLHgJLRjjUpv9Bo/QCLgT873v8I+NEYxlNKuMA3AbOs17OAJuv148BdkccBdwGPO7aHHZfEuF8AlqZ6vMAEYCdmb98TgC/yc4DZf2Cx9dpnHaciPxvO45IQZxHwOvAl4EXr/ikZL+4Cn5KfA2AKcACrOCTV43Vc/0bg3bGINd1TNG6NvS8eo1jcmKm1Pmq9PgbMtF7HinvUfx8rJXAl5sg4JeO10h27gA7gVczRbKfWut/lvqGYrP2ngILRitXi18B/ZKCHckEKx6uBV5RS9UqpFda2lPwcYH6TOQ78Tyv99ZRSamIKx2tzJ7DOej2qsaa7wKcN2nz8plRNqlJqErABuE9rfdq5L5Xi1VoHtdZXYI6MFwGfH9uIYqOUugXo0FrXj3UsCXKt1roSuAn4gVJqiXNnKn0OML/hVAKPaa2vBM5ipjlCpFi8WHMtXwWejdw3GrGmu8CnemPvdqXULADrvx3W9lhxj9rvo5TKwRT3NVrr51M9XgCtdSewGTPFMVUpZXckc943FJO1fwpwchRjvQb4qlLqIPAHzDTNP6dqvFrrw9Z/O4CNmA/QVP0ctAFtWus66/1zmIKfqvGC+eDcqbVut96PaqzpLvChxt7Wk/JO4I9jHJOTPwL2rPe3MXPd9vZvWTPnNcAp62vbn4EblVJ+a3b9RmvbiKKUUsBvgUat9a9SOV6l1HSl1FTr9XjMuYJGTKH/WoxY7d/ha8Ab1kjpj8CdVtXKJcDlwLaRjBVAa/0jrXWR1roU8/P4htb6m6kYr1JqolIq336N+e+3hxT8HABorY8Bh5RS5damvwT2pWq8FncxkJ6xYxq9WJM1sTBaP5izzx9j5mUfGsM41gFHgT7Mkcb3MHOprwOfAK8BF1nHKuA3VswNwELHdb4L7Ld+/u8kxXot5lfDD4Fd1s/NqRgv8AXgAyvWPcDD1vYyTMHbj/n1N8/aPs56v9/aX+a41kPW79AE3DQKn4nrGaiiSbl4rZh2Wz977f9/UvFz4LjPFcAO6/OwCbOyJCXjBSZifhub4tg2qrGKVYEgCEKGku4pGkEQBCEGIvCCIAgZigi8IAhChiICLwiCkKGIwAuCIGQoIvBCVqKUekiZ7pQfWm5/1Zbb34Sxjk0QRgopkxSyDqXUYuBXwPVa6x6l1DRMZ8L3MOuPT4xpgIIwQsgIXshGZgEntNY9AJagfw2YDWxWSm0GUErdqJR6Xym1Uyn1rOXdY3uo/5MyfdS3KaUus7bfoZTao0zv+i1j86sJwgAygheyDkuo38G0H34N04P9Lcs/ZqHW+oQ1qn8ecwXpWaXU/ZirT1dZxz2ptf65UupbwNe11rcopRqAr2itDyulpmrTO0cQxgwZwQtZh9b6DFAFrMC0n12vlPpOxGE1mE033rWsir+N2bDBZp3jv4ut1+8Cv1NK3YPZ5EEQxhTf4IcIQuahtQ4CbwJvWiPvb0ccooBXtdZ3xbpE5Gut9UqlVDXw10C9UqpKa31yZCMXhMSREbyQdSizX+bljk1XAC1AF2YLQ4CtwDWO/PpEpdTnHOd8w/Hf961jLtVa12mtH8b8ZuC0eRWEUUdG8EI2Mgn475YNcT+mS98KTGvXf1dKHdFa32ClbdYppfKs836M6VwK4FdKfQj0WOcB/FfrwaEwHQN3j8YvIwixkElWQRgizsnYsY5FEOIhKRpBEIQMRUbwgiAIGYqM4AVBEDIUEXhBEIQMRQReEAQhQxGBFwRByFBE4AVBEDKU/wOAnbvBbUa6TgAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "plt.plot(loss_monitor.loss, '.')\n", "plt.xlabel('Steps')\n", "plt.ylabel('Loss')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "由上可知，通过量子模拟得到的量子版词嵌入模型也能很好的完成嵌入任务。当数据集大到经典计算机算力难以承受时，量子计算机将能够轻松处理这类问题。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 参考文献\n", "\n", "[1] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. [Efficient Estimation of Word Representations in\n", "Vector Space](https://arxiv.org/pdf/1301.3781.pdf)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.8" } }, "nbformat": 4, "nbformat_minor": 2 }