{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# 使用字符级RNN生成名称\n",
    "\n",
    "`Ascend` `GPU` `进阶` `自然语言处理` `全流程`\n",
    "\n",
    "[![](https://gitee.com/mindspore/docs/raw/r1.5/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9taW5kc3BvcmUtd2Vic2l0ZS5vYnMuY24tbm9ydGgtNC5teWh1YXdlaWNsb3VkLmNvbS9ub3RlYm9vay9yMS41L3R1dG9yaWFscy96aF9jbi9taW5kc3BvcmVfcm5uX2dlbmVyYXRpb24uaXB5bmI=&imageid=59a6e9f5-93c0-44dd-85b0-82f390c5d53b) [![](https://gitee.com/mindspore/docs/raw/r1.5/resource/_static/logo_notebook.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.5/tutorials/zh_cn/mindspore_rnn_generation.ipynb) [![](https://gitee.com/mindspore/docs/raw/r1.5/resource/_static/logo_download_code.png)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/r1.5/tutorials/zh_cn/mindspore_rnn_generation.py) [![](https://gitee.com/mindspore/docs/raw/r1.5/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/r1.5/tutorials/source_zh_cn/intermediate/text/rnn_generation.ipynb)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 概述\n",
    "\n",
    "本教程中，我们将通过反向操作来生成不同语言的名称。这里仍通过编写由线性层结构构建出的小型RNN网络模型来实现目标。此次与[《使用字符级RNN分类名称》](https://www.mindspore.cn/tutorials/zh-CN/r1.5/intermediate/text/rnn_classification.html)这篇教程最大的区别在于，不是通过输入名称中的所有字母来预测分类，而是输入一个分类类别，然后一次输出一个字母，这种用于预测字符来形成一个单词的方法通常称为“语言模型”。\n",
    "\n",
    "> 本篇基于GPU/Ascend环境运行。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 准备环节"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 环境配置"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "本教程我们在Ascend环境下，使用PyNative模式运行实验。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "from mindspore import context\n",
    "\n",
    "context.set_context(mode=context.PYNATIVE_MODE, device_target=\"Ascend\")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 准备数据\n",
    "\n",
    "数据集是来自18种语言的数千种姓氏，点击[这里](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/intermediate/data.zip)下载数据，并将其提取到当前目录。\n",
    "\n",
    "数据集目录结构为`data/names`，目录中包含 18 个文本文件，名称为`[Language].txt`。 每个文件包含一系列名称，每行一个名称。数据大多数是罗马化的，需要将其从Unicode转换为ASCII。\n",
    "\n",
    "可在Jupyter Notebook中执行以下代码完成数据集的下载，并将数据集解压完成。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "!wget -NP ./ https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/intermediate/data.zip\n",
    "!unzip ./data.zip"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 数据处理"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 导入模块。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "import os\n",
    "import glob\n",
    "import string\n",
    "import unicodedata\n",
    "from io import open"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`find_files`函数，查找符合通配符要求的文件。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "def find_files(path): \n",
    "    return glob.glob(path)\n",
    "\n",
    "print(find_files('data/names/*.txt'))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "['data/names/German.txt', 'data/names/Dutch.txt', 'data/names/English.txt', 'data/names/Italian.txt', 'data/names/Vietnamese.txt', 'data/names/Portuguese.txt', 'data/names/Korean.txt', 'data/names/Spanish.txt', 'data/names/French.txt', 'data/names/Russian.txt', 'data/names/Greek.txt', 'data/names/Arabic.txt', 'data/names/Irish.txt', 'data/names/Chinese.txt', 'data/names/Czech.txt', 'data/names/Polish.txt', 'data/names/Japanese.txt', 'data/names/Scottish.txt']\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`unicode_to_ascii`函数，将Unicode转换为ASCII。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "all_letters = string.ascii_letters + \" .,;'-\"\n",
    "n_letters = len(all_letters) + 1\n",
    "\n",
    "def unicode_to_ascii(s):\n",
    "    return ''.join(\n",
    "        c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn'\n",
    "        and c in all_letters\n",
    "    )\n",
    "\n",
    "print(unicode_to_ascii('Estéves'))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Esteves\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`read_lines`函数，读取文件，并将文件每一行内容的编码转换为ASCII。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "def read_lines(filename):\n",
    "    lines = open(filename, encoding='utf-8').read().strip().split('\\n')\n",
    "    return [unicode_to_ascii(line) for line in lines]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "定义`category_lines`字典和`all_categories`列表。\n",
    "\n",
    "- `category_lines`：key为语言的类别，value为名称的列表。\n",
    "- `all_categories`：所有语言的种类。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "category_lines = {}\n",
    "all_categories = []\n",
    "\n",
    "for filename in find_files('data/names/*.txt'):\n",
    "    category = os.path.splitext(os.path.basename(filename))[0]\n",
    "    all_categories.append(category)\n",
    "    lines = read_lines(filename)\n",
    "    category_lines[category] = lines\n",
    "\n",
    "n_categories = len(all_categories)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 将所有语言的数量和种类进行打印显示。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "print('# categories:', n_categories, all_categories)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "# categories: 18 ['German', 'Dutch', 'English', 'Italian', 'Vietnamese', 'Portuguese', 'Korean', 'Spanish', 'French', 'Russian', 'Greek', 'Arabic', 'Irish', 'Chinese', 'Czech', 'Polish', 'Japanese', 'Scottish']\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 创建网络\n",
    "\n",
    "该网络基于[《使用字符级RNN分类名称》](https://www.mindspore.cn/tutorials/zh-CN/r1.5/intermediate/text/rnn_classification.html)教程中的RNN网络进行了扩展，附加了一个与输入`input`和隐藏状态`hidden`连接在一起的类别`category`张量。该张量与字母输入一样采用one-hot编码。\n",
    "\n",
    "该网络的输出为下一个字母出现的概率，将最有可能出现的字母作为下一次迭代的输入`input`。\n",
    "\n",
    "与上一个网络结构略有不同，为了有更好的效果，在`output combined`层之后我们又添加了一个线性层`o2o`。与此同时也新添加了一个`dropout`层，该层以一定的概率（此处为0.1）将输入的部分随机归零。这一步骤通常用来防止过拟合。\n",
    "\n",
    "![rnn2](https://gitee.com/mindspore/docs/raw/r1.5/tutorials/source_zh_cn/intermediate/text/images/run2.png)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "import numpy as np\n",
    "\n",
    "from mindspore import nn, ops, Tensor\n",
    "from mindspore import dtype as mstype\n",
    "\n",
    "class RNN(nn.Cell):\n",
    "    \"\"\"定义RNN网络\"\"\"\n",
    "    def __init__(self, input_size, hidden_size, output_size):\n",
    "        super(RNN, self).__init__()\n",
    "        self.hidden_size = hidden_size\n",
    "        self.i2h = nn.Dense(n_categories + input_size + hidden_size, hidden_size)\n",
    "        self.i2o = nn.Dense(n_categories + input_size + hidden_size, output_size)\n",
    "        self.o2o = nn.Dense(hidden_size + output_size, output_size)\n",
    "        self.dropout = nn.Dropout(0.1)\n",
    "        self.softmax = nn.LogSoftmax(axis=1)\n",
    "   \n",
    "    # 构建RNN网络结构\n",
    "    def construct(self, category, input, hidden):\n",
    "        op = ops.Concat(axis=1)\n",
    "        input_combined = op((category, input, hidden))\n",
    "        hidden = self.i2h(input_combined)\n",
    "        output = self.i2o(input_combined)\n",
    "        output_combined = op((hidden, output))\n",
    "        output = self.o2o(output_combined)\n",
    "        output = self.dropout(output)\n",
    "        output = self.softmax(output)\n",
    "        return output, hidden\n",
    "    \n",
    "    # 初始化隐层状态\n",
    "    def initHidden(self):\n",
    "        return Tensor(np.zeros((1, self.hidden_size)),mstype.float32)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 训练\n",
    "\n",
    "### 准备训练"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 通过`random_training_pair`函数随机选择一种语言和其中一个名称作为训练数据。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "import random\n",
    "\n",
    "# 随机选择\n",
    "def random_choice(l):\n",
    "    return l[random.randint(0, len(l) - 1)]\n",
    "\n",
    "# 随机选择一种语言和一个名称\n",
    "def random_training_pair():\n",
    "    category = random_choice(all_categories)\n",
    "    line = random_choice(category_lines[category])\n",
    "    return category, line"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "对于训练集中的每个名称，该网络的输入为：`(category, current letter, hidden state)`，输出为：`(next letter, next hidden state)`。因此对于每个训练集，我们都需要`categoryTensor`（代表种类的one-hot张量），用于输入的`inputTensor`（由首字母到尾字母（不包括EOS）组成的one-hot矩阵）和用于输出的`targetTensor`（由第二个字母到尾字母（包括EOS）组成的张量）。\n",
    "\n",
    "因为我们需要预测当前字母所对应的下一个字母，所以需要拆分连续字母来组成字母对。例如：对于`\"ABCD<EOS>\"`，我们将创建出`('A', 'B'), ('B', 'C'), ('C', 'D'), ('D', 'EOS')`字母对。\n",
    "\n",
    "![pair](https://gitee.com/mindspore/docs/raw/r1.5/tutorials/source_zh_cn/intermediate/text/images/pair.png)\n",
    "\n",
    "我们在训练时会持续将`category`张量传输至网络中，该张量维度为`<1 x n_categories>`的[one-hot张量](https://en.wikipedia.org/wiki/One-hot)。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`category_to_tensor`函数将类别转换成维度为`<1 x n_categories>`的one-hot张量。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "source": [
    "def category_to_tensor(category):\n",
    "    li = all_categories.index(category)\n",
    "    tensor = Tensor(np.zeros((1, n_categories)),mstype.float32)\n",
    "    tensor[0,li] = 1.0\n",
    "    return tensor"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`input_to_tensor`函数，将输入转换成一个由首字母到尾字母（不包括EOS）组成的one-hot矩阵。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "def input_to_tensor(line):\n",
    "    tensor = Tensor(np.zeros((len(line), 1, n_letters)), mstype.float32)\n",
    "    for li in range(len(line)):\n",
    "        letter = line[li]\n",
    "        tensor[li,0,all_letters.find(letter)] = 1.0\n",
    "    return tensor"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`target_to_tensor`函数，将目标值准换成一个由第二个字母到尾字母(包括EOS)组成的张量。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "source": [
    "def target_to_tensor(line):\n",
    "    letter_indexes = [all_letters.find(line[li]) for li in range(1, len(line))]\n",
    "    \n",
    "    # 添加EOS\n",
    "    letter_indexes.append(n_letters - 1)\n",
    "    \n",
    "    return Tensor(np.array(letter_indexes), mstype.int64)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "为了方便训练，我们将使用`random_training`函数来获取随机对`(category，line)`，并将其转换为所需格式的`(category, input, target)`张量。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "source": [
    "def random_training():\n",
    "    category, line = random_training_pair()\n",
    "    category_tensor = category_to_tensor(category)\n",
    "    input_line_tensor = input_to_tensor(line)\n",
    "    target_line_tensor = target_to_tensor(line)\n",
    "    return category_tensor, input_line_tensor, target_line_tensor"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 训练网络\n",
    "\n",
    "与分类模型依赖最后输出作为结果不同，这里我们在每一步都进行了预测，因此每一步都需要计算损失。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 定义`NLLLoss`损失函数。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "source": [
    "import mindspore.ops as ops\n",
    "\n",
    "class NLLLoss(nn.LossBase):\n",
    "    def __init__(self, reduction='mean'):\n",
    "        super(NLLLoss, self).__init__(reduction)\n",
    "        self.one_hot = ops.OneHot()\n",
    "        self.reduce_sum = ops.ReduceSum()\n",
    "        \n",
    "    def construct(self, logits, label):\n",
    "        label_one_hot = self.one_hot(label, ops.shape(logits)[-1], ops.scalar_to_array(1.0), ops.scalar_to_array(0.0))\n",
    "        loss = self.reduce_sum(-1.0 * logits * label_one_hot, (1,))\n",
    "        return self.get_loss(loss)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "source": [
    "criterion = NLLLoss()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- MindSpore将损失函数，优化器等操作都封装到了Cell中，但是本教程rnn网络循环的每一步都需要计算损失，所以我们需要自定义`WithLossCell`类，将网络和Loss连接起来。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "source": [
    "class WithLossCellRnn(nn.Cell):\n",
    "    \"\"\"构建有损失计算的RNN网络\"\"\"\n",
    "    \n",
    "    def __init__(self, backbone,loss_fn):\n",
    "        super(WithLossCellRnn, self).__init__(auto_prefix=True)\n",
    "        self._backbone = backbone\n",
    "        self._loss_fn = loss_fn\n",
    "\n",
    "    def construct(self, category_tensor, input_line_tensor, hidden, target_line_tensor):\n",
    "        loss = 0\n",
    "        for i in range(input_line_tensor.shape[0]):\n",
    "            output, hidden = self._backbone(category_tensor, input_line_tensor[i], hidden)\n",
    "            l = self._loss_fn(output, target_line_tensor[i])\n",
    "            loss += l\n",
    "        return loss"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 创建优化器、`WithLossCellRnn`实例和`TrainOneStepCell`训练网络。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "source": [
    "rnn_cf = RNN(n_letters, 128, n_letters)\n",
    "optimizer = nn.Momentum(filter(lambda x:x.requires_grad,rnn_cf.get_parameters()),0.0001,0.9)\n",
    "net_with_criterion = WithLossCellRnn(rnn_cf, criterion)\n",
    "net = nn.TrainOneStepCell(net_with_criterion, optimizer)\n",
    "net.set_train()\n",
    "\n",
    "# 训练网络\n",
    "def train(category_tensor,input_line_tensor, target_line_tensor):\n",
    "    new_shape = list(target_line_tensor.shape)\n",
    "    new_shape.append(1)\n",
    "    target_line_tensor = target_line_tensor.reshape(new_shape)\n",
    "    hidden = rnn_cf.initHidden()\n",
    "    loss = net(category_tensor, input_line_tensor, hidden,target_line_tensor)\n",
    "    \n",
    "    # 返回一个序列最后一个\n",
    "    for i in range(input_line_tensor.shape[0]):\n",
    "        output, hidden = rnn_cf(category_tensor, input_line_tensor[i], hidden)\n",
    "\n",
    "    return  output, loss / input_line_tensor.shape[0]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 为了跟踪网络模型训练过程中的耗时，定义`time_since`函数，用来计算训练运行的时间，方便我们持续看到训练的整个过程。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "source": [
    "import time\n",
    "import math\n",
    "\n",
    "# 定义可读时间回调字符串\n",
    "def time_since(since):\n",
    "    now = time.time()\n",
    "    s = now - since\n",
    "    m = math.floor(s / 60)\n",
    "    s -= m * 60\n",
    "    return '%dm %ds' % (m, s)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 在训练过程中，每经过`print_every`（500）次迭代就打印一次，分别打印迭代所用时间、迭代次数、迭代进度和损失值。同时，根据`plot_every`的值计算平均损失，将其添加进`all_losses`列表，以便于后面绘制训练过程种损失函数的图像。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "source": [
    "n_iters = 7500\n",
    "print_every = 500\n",
    "plot_every = 100\n",
    "all_losses = []\n",
    "\n",
    "# 每经过100次迭代，就重置为0\n",
    "total_loss = 0 \n",
    "\n",
    "start = time.time()\n",
    "\n",
    "for iter in range(1, n_iters + 1):\n",
    "    output, loss = train(*random_training())\n",
    "    total_loss += loss\n",
    "    \n",
    "    # 分别打印迭代所用时间、迭代次数、迭代进度和损失值\n",
    "    if iter % print_every == 0:\n",
    "        print('%s (%d %d%%) %.4f'% (time_since(start), iter, iter / n_iters * 100, loss.asnumpy())) \n",
    "    \n",
    "    if iter % plot_every == 0:\n",
    "        all_losses.append((total_loss / plot_every).asnumpy())\n",
    "        total_loss = 0"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "5m 56s (500 6%) 3.7287\n",
      "11m 37s (1000 13%) 4.1077\n",
      "17m 8s (1500 20%) 4.0800\n",
      "22m 51s (2000 26%) 4.1025\n",
      "28m 28s (2500 33%) 4.0878\n",
      "34m 3s (3000 40%) 4.0277\n",
      "39m 38s (3500 46%) 4.0859\n",
      "45m 22s (4000 53%) 3.9899\n",
      "51m 7s (4500 60%) 3.5648\n",
      "56m 52s (5000 66%) 4.0283\n",
      "63m 34s (5500 73%) 4.0877\n",
      "71m 50s (6000 80%) 4.0858\n",
      "79m 42s (6500 86%) 4.1082\n",
      "88m 11s (7000 93%) 3.7557\n",
      "97m 32s (7500 100%) 4.0461\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "- 使用`matplotlib.pyplot`绘制训练过程中损失函数的图像。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(all_losses)"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f7efddbead0>]"
      ]
     },
     "metadata": {},
     "execution_count": 22
    },
    {
     "output_type": "display_data",
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABIPklEQVR4nO29eXxcd3nv/35ml2ZG+2pJ3vfYjhOcBUJI4kAaQpqQFChLIVBoaAstvb2UJuWWQlouF36UpRfoJYVAS9jalCWEncRhTZzYcWzHdryvkq19H2lGM/P9/XHOGY1GM9JI1jKSn/frpZdG53xn5jsa6XOeeVYxxqAoiqIsXlzzvQFFURRldlGhVxRFWeSo0CuKoixyVOgVRVEWOSr0iqIoixzPfG8gk6qqKrN8+fL53oaiKMqCYvfu3R3GmOps5wpO6JcvX86uXbvmexuKoigLChE5neucum4URVEWOSr0iqIoixwVekVRlEWOCr2iKMoiR4VeURRlkaNCryiKsshRoVcURVnk5C30IuIWkT0i8liWc34R+baIHBORnSKyPO3cFhF5SkQOiMh+EQnM0N7H0Dc8wqd/foTnz/bMxsMriqIsWKZi0b8POJTj3DuBbmPMauDTwMcBRMQDPAz8qTHmMuBGYGTau50Ak4TPPn6UXae6ZuPhFUVRFix5Cb2INAKvAb6UY8mdwL/btx8BbhYRAW4B9hlj9gIYYzqNMYmL23J2Soo8eFxC12BsNh5eURRlwZKvRf8Z4ANAMsf5BuAsgDEmDvQClcBawIjIT0XkORH5wMVtNzciQkXQR+eACr2iKEo6kwq9iNwOtBljdk/j8T3Ay4G32N/vEpGbszzHvSKyS0R2tbe3T+NpLCpDfjrVolcURRlDPhb9dcAdInIK+BawXUQezljTDDRByi9fCnQC54BfGWM6jDER4EfAlZlPYIx50BizzRizrbo6a/O1vKgM+ugcjE77/oqiKIuRSYXeGHO/MabRGLMceCPwhDHmjzKWPQrcY99+nb3GAD8FNotIsX0BuAE4OGO7z6Ay5FMfvaIoSgbTblMsIg8Au4wxjwJfBr4mIseALqwLAsaYbhH5FPAsYIAfGWN+ePHbzo766BVFUcYzJaE3xjwJPGnf/lDa8WHg9Tnu8zBWiuWsUxXyMxCNE40n8Hvcc/GUiqIoBc+iqoytCPoA1H2jKIqSxqIS+kpb6NV9oyiKMsriEvqQLfRq0SuKoqRYXEIf9APQOaAploqiKA6LSugrQuqjVxRFyWRRCX3Y78HndtGhPnpFUZQUi0ronX43XVodqyiKkmJRCT1YAVnNulEURRll0Ql9RdCnWTeKoihpLDqhrwr5tbGZoihKGotO6CuCPrrUdaMoipJiUQr9YCzB8MisDLJSFEVZcCw6oa/S6lhFUZQxLDqhr9DqWEVRlDEsOqHXfjeKoihjWXxCrx0sFUVRxpC30IuIW0T2iMhjWc75ReTbInJMRHaKyPKM80tFZEBE3j8De56QypDlutHqWEVRFIupWPTvAw7lOPdOoNsYsxr4NPDxjPOfAn489e1NnaDPjc/jUoteURTFJi+hF5FG4DXAl3IsuRP4d/v2I8DNIiL2fV8LnAQOXNRO80REqNLqWEVRlBT5WvSfAT4AJHOcbwDOAhhj4kAvUCkiIeBvgY9M9OAicq+I7BKRXe3t7XluKTcVIZ+2KlYURbGZVOhF5HagzRizexqP/2Hg08aYgYkWGWMeNMZsM8Zsq66unsbTjKUy6Nf0SkVRFBtPHmuuA+4QkduAAFAiIg8bY/4obU0z0AScExEPUAp0AtcArxORTwBlQFJEho0xn5vJF5FJZdDH8fYJry2KoiiXDJMKvTHmfuB+ABG5EXh/hsgDPArcAzwFvA54whhjgOudBSLyYWBgtkUetFWxoihKOtPOoxeRB0TkDvvHL2P55I8Bfw3cNxObmy4VQT9DIwkisfh8bkNRFKUgyMd1k8IY8yTwpH37Q2nHh4HXT3LfD095d9MkVR07EKO4YkovUVEUZdGx6CpjYbQ6VjNvFEVRFqvQ29WxOoBEURRlsQq99rtRFEVJsTiFXjtYKoqipFiUQl/s8xDwutRHryiKwiIVerCqYzu0OlZRFGURC732u1EURQEWsdBXBLU6VlEUBRax0FcG/WrRK4qisJiFPuSjYyCK1XJHURTl0mXxCn3QRzSeJBJLpI71RkYYHklMcC9FUZTFx6IV+oqMoqlTHYNs/+cn+cfHDs7nthRFUeacRdvxqyqtDUIo4OHtX3mGzsEYZ7oi87wzRVGUuWXRCr1j0bf0DPNPPzxES+8wK6qCmomjKMolx6J13ThtEP7h0QPsPt3Np9+wlauXV2ijM0VRLjnyFnoRcYvIHhF5LMs5v4h8W0SOichOEVluH3+ViOwWkf329+0zuPcJqQxarpuOgSgfvG0Dr9lSn5o8pZk4iqJcSkzFdfM+4BBQkuXcO4FuY8xqEXkj8HHgD4EO4PeNMS0isgn4KdBwkXvOiyKfm3W1Ya5fU8W7rl8BWO2L40lD31Cc0mLvXGxDURRl3slL6EWkEXgN8FGsUYGZ3Al82L79CPA5ERFjzJ60NQeAIhHxG2PmxH/yk7+6HhFJ/ey0L+4YjKrQK4pyyZCv6+YzwAeAZI7zDcBZAGNMHOgFKjPW/AHwXDaRF5F7RWSXiOxqb2/Pc0uTky7yMHbEoKIoyqXCpEIvIrcDbcaY3dN9EhG5DMud8+5s540xDxpjthljtlVXV0/3aSbF8dt3aUBWUZRLiHws+uuAO0TkFPAtYLuIPJyxphloAhARD1AKdNo/NwLfBd5mjDk+Q/ueFlW2Rd+Rw6I/0xnhmZNdc7klRVGUWWdSoTfG3G+MaTTGLAfeCDxhjPmjjGWPAvfYt19nrzEiUgb8ELjPGPPbmdv29CifZMTgP//8MG/+t6fZc6Z7LrelKIoyq0w7j15EHhCRO+wfvwxUisgxrGDtffbx9wKrgQ+JyPP2V81F7fgi8LpdlBV7c+bSt/QMEU8a3vuNPfRGRuZ4d4qiKLPDlCpjjTFPAk/atz+UdnwYeH2W9f8E/NNF7XCGmahP/fneYdbXhTnWNsAH/nsv/++PXjIuoKsoirLQWLSVsbmoyjFiMJk0tPVFuWFdNfe9ej0/PdDKfzx1eh52qCiKMrNcckKfa8RgVyRGLJGkriTAO1++gpvX1/DRHx7ihebeediloijKzHFJCn1nFqG/0DsMQH1pABHhk6+/nMqQj/d9a4+2TFAUZUFz6Ql90E93JEY8Mbb2q7XPEvrakgBgZei847rlHG8fpG84Puf7VBRFmSkuOaGvCvkwBrozsmrOpyz6otSxxvJiAJq7h+Zug4qiKDPMJSf0lWkDSdJp7RvGJaNFVQBLyizRb+lRoVcUZeFyyQl95ohBh/O9w1SH/Xjco7+SBkfoe7ML/amOQe78/G85161TqxRFKVwuOaF3LPbMgGxr3zB1aW4bsLpd+jyunK6bnSc72Xu2h4d+c2pW9qooijITXHJC7zQ268zIpb/QO0xdiX/MMZdLWFIaoDmH6+Z0p2XJ/9euswxENWCrKEphcskJfWmRF7dLxrluLKEPjFvfUF6UU+jPdEUIeF30R+M8suvsrOxXURTlYrnkhN7lEqsNQlowdiAapz8aH+e6AVhSWpQzGHu2K8JVyyvY2lTGvz91mmRS8+0VRSk8LjmhB8v3nt6q2CmWqiv1j1vbUF5EW3+UWHz8zJXTXRGWVhTzjuuWc7JjkCePtM3ephVFUabJpSn0Id8YH71TLFVXksWiLyvCmNGLgUPv0Ag9kRGWVhRz2+Z6akv8fOW3p2Z134qiKNPh0hT6oH9Mv5tRiz6Lj95Oscz005/tsgKxSyuK8bpdvPXaZfz6aAdHW/tna9uKoijT4tIU+tDYVsUXUhZ9/kJ/xhH6Sqt69k1XL8XncfGV352ajS0riqJMm7yFXkTcIrJHRB7Lcs4vIt8WkWMislNElqedu98+flhEfm+G9n1RVIX89EfjDI8kAMuiLy3yUuRzj1vrWPmZAVlH6JsqLKGvDPl57dYlfOe5c/REdPi4oiiFw1Qs+vcBh3KceyfQbYxZDXwaaxA4IrIRa/zgZcCtwBdEZLyazjGVdnWs4745nyO1EiDgdVMd9o8T+tOdEcqLvZQEvKljb75mGcMjSX55pH2Wdq4oijJ18hJ6e8D3a4Av5VhyJ/Dv9u1HgJvFGs10J/AtY0zUGHMSOAZcfXFbvnhS/W5s941VFZtd6MEKyGbz0S+tDI45tr4uDMCpDm2JoChK4ZCvRf8Z4APA+BxDiwbgLIAxJg70ApXpx23O2cfmlUq7DUKHnUt/oS+3RQ/QUDa+OvaMnVqZTsDrpr40wOmuwRnesaIoyvSZVOhF5HagzRize7Y2ISL3isguEdnV3j77bo+U62YgxkgiScdAlNoJLPqGMqtoyhlAMpJI0twzxNKK8emYyyqLU60RFEVRCoF8LPrrgDtE5BTwLWC7iDycsaYZaAIQEQ9QCnSmH7dptI+NwRjzoDFmmzFmW3V19ZRfxFRJb1Xc1h/FGGuyVC6WlBUxPJIc9en3DJNIGpZVBMetXVYRVKFXFKWgmFTojTH3G2MajTHLsQKrTxhj/ihj2aPAPfbt19lrjH38jXZWzgpgDfDMjO1+mgR9bvweF50DsdEc+glcN6N96a21mRk36SyrKqZjIKpNzhRFKRimnUcvIg+IyB32j18GKkXkGPDXwH0AxpgDwH8CB4GfAO8xxiQubssXj4hQFfLTkSb0tRP66J1cekvgHR/8ssosQm9b+ac71U+vKEph4JnKYmPMk8CT9u0PpR0fBl6f4z4fBT467R3OEtaQ8GiqWGoi182o0I9a9D63K+vFwRH/M50RLltSOtPbVhRFmTKXZGUsWAFZy3UzhM/joqzYm3NtWbGXYp87lUt/titCY3kRbpeMW+sI/Sn1088q3YMx2vqGJ1+oKMolLPQhq9/Nhb4o9aUBrLT/7IiIlUtvT5o63RlJtT7IJBzwUhn0qetmlvmbR/by519/br63oSgLgktX6IM+OgaiXOgdmtA/77CkrIiWXivF8kzn+Bz6dDTFcnaJJ5I8dbyT1n616BUlHy5doQ/5iMaTHG8fnDDjxqHBtuh7IiP0R+OTCH1QLfpZ5MUL/QzGEvQPa2aTouTDpSv09uzYrsHYhIFYh4ayAJ2DMY7YbYgns+jP9w2nmqYpM8uuU10ADAzHU0VsiqLk5tIVersNAkycWunQUG5l3uw8aYlMLh89wPLKIMbAuW5138wGz57uBiCeNESzTP5SFGUsl6zQV4VGxwbmY9EvsefJPnW8E4Cm8txC71wEtLnZzGOMYdepLpyEp77hkfndkKIsAC5ZoR9j0ecj9HYu/e4z3VSF/AT9uUsQlttdLU93qdDPNOe6h2jti3LF0nLAct8oijIxl6zQVwRHhT6fYGxdaQCXQCyezNrMLJ3yYi/hgEcDsrPArtOW6+ymdVZPJA3IKsrkXLJC7/e4Cfs9uASqw/5J13vTKmGXVY5vZpaOiEw7xXLH4TZ+duDClO93qfDsqW7CAQ9XLrMteu0ppCiTMqUWCIuNypCPopgbrzu/611DWRHne4ezNjPLZFllkAPNvVPe02d+cZRYPMktl9VN+b6XArtOdfGSZeWUFlmVzGrRK8rkXLIWPViWfH3ZxG6YdBw//USplQ7LKoo51z1EPDG1rJAznYN02QNRlLH0RGIcaR3gquUVhP2O0GswVlEm45K26O+/bQNTScN2hD5b18pMllcGiScNLT3DE6ZiptM3PEJ3ZASvWzDGTNiW4VJkt51WuW1ZOeGA9aerrhtFmZxLWuivtDM38mVtbQif28WKqol99JDe3Gwwb6E/Y/v0RxKGvuF4yj2hWDx7qhuvW7i8qSzVUE5dN4oyOZe00E+VO7c2cPWKijE5+LlYNo0Uy7NpazsHoir0Gew61cXmhlICXjcAAa9LLfopcrJjkIMtfbxmS/18b0WZQy5pH/1UcbuExgkKpdKpCfsJeF2c7sg/xfJMmtA7YwsVi+GRBPvO9XLV8orUsZDfqxb9FHnwVyf4i28+x6BeIC8p8hkOHhCRZ0Rkr4gcEJGPZFmzTEQeF5F9IvKkiDSmnfuEfb9DIvIvcok4nl0uYVlFcEp96dOt/46B6Qv9Yuz/sr+5l1giybY0oQ8HPBqMnSLH2wdIGnhhGhlhysIlH4s+Cmw3xlwObAVuFZFrM9Z8EvgPY8wW4AHgYwAi8jKs4eJbgE3AVcANM7P1wmdpZTFnusZa9Od7h3I2OzvbFaG2ZHRw+XTYc6abzR/+GSfaB7KeTyYNH/vRIQ6d75vW488Xu05ZgdiXLBuNq4QDHnXdTJET7dbf495zPfO7kTSGYokF3xcqnkjy2V8cpTdSmIZHPsPBjTHGUQ2v/ZVpMm4EnrBv7wDudO4OBAAf4Lfv23qRe14wLLeLppJJ69f1tadO8fKP7+BfHj+adf3pzghbm8oA6JqmRf+NnWcYiMZTzdcyOdk5yBd/dYKP/ODAtB5/vth1qotV1cExFc0hv0ddN1Ogd2iEjgHLgNh7rnAs+i/+6jiv/uyvGZliKnIhcfB8H5/+xRF+uP/8fG8lK3n56EXELSLPA23Az40xOzOW7AXutm/fBYRFpNIY8xSW8J+3v35qjDmU5fHvFZFdIrKrvb19mi+l8FhaGSQaT9LcM8T/+t5+/v77B0gaw3NnusetjSesdatrQoQDHjpz+OjPdkX4z11ns54biiX4kf2HdrAlu8V+wD7+9Ikudp/OfjEoRF5o6eVy+yLoEA54FlWvmzOdEbY+8DNevDA7n7acT3lhv4e9Z3tm5Tmmw9G2AfqH4wu6ZYgTUzue45P0fJOX0BtjEsaYrUAjcLWIbMpY8n7gBhHZg+WaaQYSIrIa2GDfrwHYLiLXZ3n8B40x24wx26qrq6f/agqM5XZa5R99eScPP32GP71hFX+4rYmDLX3j/Ojne4dJJA1LK4qpCvlzCv3DO0/zgUf2pXLK0/nZwQsMxhKUBDwcaMlusR1o6cXndlER9PG5J45d5CucG3ojI7T2RVlXGx5zPOT3LirXzbOnuuiJjPDsqfHvbS6+8tuT/PpofsaR47a5bXM957qH6BwojMK8c/aIzhcv9M/zTqZPj+2yOda2gIXewRjTg2Wh35pxvMUYc7cx5grgg2lr7wKeNsYM2O6fHwMvnYF9LwicLpbne4b51Bsu575Xr+eyhlL6huO09I4dg+f0xWmqKKYi6Mv5T3jBvt+/Pnl83LnvPNdMQ1kRd13RwIsX+kkkxwdlDzT3sbYuxB9ft5wdh9sXRFDuSJslAGszhD4c8CyqNsXO68wVX8nEGMMnf3qYf/v1ybzWn+gYwOMSfv/yJQDsKxD3TbPtnz+8gIV+wVv0IlItImX27SLgVcCLGWuqRMR5rPuBh+zbZ7AsfY+IeLGs/XGum8VKY3kR9796Pd9+97XcfaWViLSx3hKrQxmuFSe1clllkMqgj84cPvrzttD/4lDrmH+Mtr5hfn20nbuuaOCyhlIisQSnMj4KG2M40NLLpiWlvO1lywkHPHx+R+Fb9c5UrzW1oTHHnWDsYskyOtZqicTx9vxcGL1DIwzGEhxo7s3rd3C8bZClFcVsXVqGSGEEZIdiiVSG2UK26Lsj1mto7hliKFZ4k+XysejrgR0isg94FstH/5iIPCAid9hrbgQOi8gRoBb4qH38EeA4sB/Lj7/XGPODmXwBhYyI8O4bVqV6pwOsqytBhHFZL6e7BvG6hbqSAJUTuG5a+4a5fk0VRV43X/zlqFX//edbSBq468oGLltSAoz64x3O9w7THRnhsiUllAS83PPS5fzkwAWOtRX2P9jR1gGCPjcNGX2JQn4PxsDgLPxjDcUSvPtru/jVkbmLGU3VondcHp2DsZQBMBEnOgZYWR0k5PewpiZUEH765h7LwPF5XIvCojemMK36fLJu9hljrjDGbDHGbDLGPGAf/5Ax5lH79iPGmDXGmLXGmHcZY6L28YQx5t3GmA3GmI3GmL+e3ZdT+IT8HpZVFHMwQ+jPdkVoKi/G7RIqgz66I7FUto6DMYYLvcNsqC/hTVcv5ft7W1LVtN/Z08zWpjJWVYdYUxPG65ZxAVnHTXNZQykAf/zyFQQ8br6wY7wbqJA4fKGfNbXhcb1/Qk6/m1kIyH78Jy/y0wOtfO3p0zP+2NmIxOKc7Roi6HPT3JM7BTcdR+jBqjOYiETScKozwqpq61PRlsYy9p3L75PAbHLWfg3XrarkTFdkwRZy9dg9qmCBCr0y82yoLxln0Z/piqTaH1eGfCSSht6hsf7nnsgI0XiS2pIAf/KKFbgEvvTrExxs6ePQ+T7uvrIBsKyjNTXhcQHZAy19uAQ21FkWf0XQx5uvsS4YZ6bRO3+uONrWz9oMtw1AOGC1iBiIzqyf/jdHO/jq704R9nv4zdGOORny7gTxblxfgzFWq4LJSM89n6wldnP3ELF4kpXVVtzo8qYyOgdjYy4W84Hz/Ns31AKjbrqFRtdgjI31JbgEjhdgQFaFfh7YWF/C6a5IKmPEGMPpzkiq/bGTK55ZNHWhz/p4XlcSoL60iNdubeBbz57lS78+gdct3L5lyehzLCkZl91zoKWXldUhinzu1LF7X7EStwhve2gnP9p/ft4tvEy6BmN0DMTGBWLBShME6JtBi753aIS/eWQvK6uDfOJ1WxgaSeSsSZhJjtr++VvtOQT5WIXnuq1PAOtqw5Na9M7jrbQt+q2NZcD8B2TPdUfweVxcv7oKWLgB2e5IjJqSAMsqgxxTi14By6I3Bg7b+dK9QyP0D8dTHS+dpmmZbRBSQl9qnf/TG1cRSyT5zp5mblpXM6aY6LIlJXQOxmjrH71YHGjpY5Ptv3eoLQnw0Nuvwudx8edff467vvA7dp7onOFXPH1GA7FZhH4WXDcfefQAbf1RPvWGrdy0voaA18UTh2a/xu9IWz8+t4sb7RGJJ/IIyDb3DNFQXsSmhlJeyFE34ZAServz6rq6MD63a94Dsue6hmgsK2JpRTHFPveCDch2R2JUFPtYVR3keFvh1QOo0M8DG2yxPXje+qNOT62E0cHlmY3NWnsdobeCkquqQykL0MnqcbhsieWHd9w3nQNRzvcOp46n8/I1Vfz4fa/gE3+whQu9w/zhg08XTDaOI/SZOfQw6qOfqerYn7xwnu/saeY9N61ma1MZAa+b61ZV8cThtln/pHO01QqUhgNeGsqK8rboG8uL2dRQQnt/lNa+3AHZEx2DlBZ5U8aAz+Ni45KSeQ/InuuO0FBehMslrKkNL0jXjTGG7sERyoJeVtWEONkxOOWBQ7ONCv08sKQ0QGmRN+Wnd1Irx7luMnLpz/cOI2J1xnT421vX847rlrN9fc2YtRvsNE4nIOtk4FzWMNaid3C7hDdc1cSO99/IS5aV8+jzLRf1GmeKI639hAOeVA+gdEJ+Z/jIxfvoB6Nx/u67L7CpoYS/2L46dXz7hhrOdg3NeiHM0bZ+VtdYbpWV1cH8LPruCA1lRWy2g+v7J3DDnGgfYFV1cExA+/LGUvY392att5grnIsVwPra8IJ03QzGEsQSSSqKfayuDhFLJFNB5kJBhX4eEBE21IdTIjxO6IsdH32GRd83TGXQP2bG7fKqIP/w+5fh84x9K8MBL8sqi1MCnxL6+vEWfTpFPjfXrarkaFt/QeQDH2kdYG2WjBsYDcbOhEV/uLWfrsEYf7l9zZjf703rrAvoEy+2XfRz5MLJuHHiEKuqQ5xoH5jwU0Tf8Ah9w3Eay4vYuMRK2X0hRzU0WLn5jn/e4fKmMiKxxLxliURicToHYzSWW59Q19WF6RyM0d5fGBW7+dJt/5+WB32ssi/WhRaQVaGfJzbUl3DYrl490xmhKuQjaFuoHreLsmLvuKKpC33DKf98PmysL0mlcR5o6aWpoojS4smHmWxqKCVp4OD5+Q3UGWM42po94wZGLfqZEPpTdpaL84/qsKSsiA31JTw+i0LvfFpwXueq6iCDsQStfbkFr9m2GBvLiyn2eVhVHcpZ5dw/PEJ7fzSVceOwxQ7IPj9P7pvR12AJ/fo660K30Kx6p1iqotiX+lRWaAFZFfp5YmN9CUMjCU53DnKmKzJu4Hhl0Dc+66Z3mLqS/IeZX7akhNOdEfqGRzjQ0jepNe+wuXFyV8Bc0D4QpTsywpqa8f55sNxNQZ97RvrdnOoYxCXQlGWwzPb11ew+3T1rLWiP2Bk3a9Isepg488ZJS2ywRXJzQykvNGcPyDpuoJVVYy9iK6uC89rg7FzaxQosix6YtaZus0VXyqL3UhLwUhP2F1zPGxX6eWJDvROQ7csu9CH/xVv0dtB316kuTnYMpipmJ6OuJEBVyM/+HMJxsfQPj/AX39wzaQWok3LoCEA2QjM0fORkZ4TG8uJxLjCA7etrSCQNv8yzedhUOWpn3Cyz/wYcF8tEvx8nh96xhi9bUsKFvuGsbo8THdbjrK4Za9G7XMLmxtJ5S7F0XkOT/RoqQ36qQr4Fa9GX2y7XVdWhgiuaUqGfJ9bUhvC4hH3nemnpHcph0Y8K/fBIgp7ICHUlgbyfw8mweWT3OcByyeSDiLC5oYT9zT15P9dU+NrTp/nB3ha+9Wz2dssOuXrcpBPyz8zwkVMdgyzPMfR9a1M55cVedsyS+8bJuPHYsYHaEj9Bn3vCnjfN3UMEvC4q7cC9E5DN5r453jaI2yUsrRj/+i5vKuPQ+T6i8bmPx5zrHsLncY2ZwbyuLszhBZZ50zVoGRpOEsXqmhDH2iaOscw1KvTzhN/jZnVNiF8cbMUYq3d9OpWhsR0sW/vGplbmQ03YspB+ftDKA8/XogdLOI61DRCJzWxJ+vBIgod+Y3Vb/MXBifPTj7QOUFbspXqCYezhwMXPjTXGcKpjkBWV2ecBu13CjetqePJw26xkqBxp7R9TJyAirJzEKjzXPURDWVEqSH3ZBEJ/omOApvKirJ9WVlYFiScNbTniARd6h3npxx6fFXfKuW4rh97lGg20r6st4Uhr9s6rhUpPJIZLoMRODlhdE6J/OF5QQWUV+nlkQ30JJ+wg4HiL3k/P0EgqH9dpWjUVi97K7ilhJGGoDvupmcJ9NzeWWQHZSQpxpsp/7T5Hx0CM2zbXcaJjcEL3xNHWftbWZM+4cbDmxl6c0HcOxuiPxnNa9GC5b7ojIzx/Nv9e8fkQicU51z3E2owg8KpJUiybe4bGDKoP+T2srApmrZA9kSXjxqHaTtVt68+eg3/oQh/ne4f53bGZL6JzcujTWV8XZngkmcpEWwh0DcYoK/alLlhOjKWQArIq9POIk+sOWYQ+5MMY6LYDgK0ZVbH54rhvpmLNw6grYLLS+qkQTyR58FfHuWJpGX932wYAHj+U3R1ijOFwa/+EbhuYmbmxTsbNREL/irXVuF3Cjhdz++kjsfi4/kST4QTtMl/nyurQhC1vs4nkpobScR1Lk0nDyY5BVlVnf22O0OeyPtttS//oLHQ4Tc+hd1iXJfPmYEsf939nX8HOHuiOxChPy2ZbXYAplir088hGOwvG73GNKYICy6KH0Yi+M3CkdgpWOYwGZDdlqYidiNoSvx2QnVzoj7X1842dZyb1ST627zxnu4b4sxtW0VhezPq6ML/I0V6gtS9K/3B8wkAs2D76i7TonQZiyytzC31pkZdV1cEJ/cd//70DvP0rz0zpuTMzbhwcq9AJpKYzGI3THRlJBWIdNjWU0NwzNKaiurlniGg8OalFn0voHUt/pgOkmTn0DlbNxOjzPXemmzc++BTffOYs39vTPKN7mCm6B0fGtB+pLfET8nvyniswF6jQzyOORd9UUTzGTwnjq2Mv9A0T8ntSRUL58pJl5fg9Ll62qnJK9xMRtjSWTppiufNEJ3d94Xf83Xf388Vfnci5zhjDvz55nDU1IV5pdyp85YZadp3upicyvvd+KhCbI7XSIeT3XnTWzalOK1iZKTqZNJUXp9pCZ+PQ+T4OtvSNay89EUdbx2bcODg579ncN809dmpl2XiLHsZ+CsvscZNJZdCPSyaw6O3jR1tnNriYmUPvUORzs6yimMOtfTx1vJO3fmkn5UEfyyuLC1foI5brxkFEWFUdLKgUy3wmTAVE5BkR2SsiB0TkI1nWLBORx0Vkn4g8KSKNaeeWisjPROSQiBwUkeUz/BoWLJUhP3UlgdRs2XSq7H43HWkWfbY2AJPRUFbEvg/fwsvs7oBTYVNDKcfbcwdkf3rgAm996Bmqw35euaGGT/zkxZzzS594sY3Drf382Y2rUhe1mzdYaYtPHh5/H0focxVLOYQDHgZjiYsK3p3qiNBUXjSmIjYbTRXFnOseyip4xhjOdkWIxpOcn6DnTCZH28Zm3DisqAoikj2XfjS1cuzfjeOmSw/IpnLoc1j0bpdQGfLTnmN0pdMUrz8aTzXVmwkyc+jTWVcX5nfHO3n7V55hSVkR//Xul/KGq5p47kxPQbbT7hqMparZHVbZmTeFQj4WfRTYboy5HNgK3Coi12as+STwH8aYLcADwMfSzv0H8P8ZYzYAVwOzV2K4APmXN13B3966ftzxSjvTpCvNoq8rnZrbxsHvcU++KAubnQrZLAHZbz1zhj97eDcb60t45E9fxmffeAVrasL8xTf3jLN6jTF84cnjNJQVpeaVAlzeWEZVyJ/VfXOktZ/KoC/1e8hFqoPlRfjpT06QWplOY3kRA9F4ahB0Oj2REfrtPZzKo5e8Q2bGjUPAa03UymrR57CGS4usthf/75fHufUzv+K1n/8tX/r1CUoCnpThkI3qkD9n1k1bf5SA15KJydw3bf3DfO6Jo9z0ySf5468+m/rkkY3MHPp01tWG6YmMsLomxLfuvZaakgB32H8333++sKx6Y4zlow+O/f2urglxoW+4YIbX5zNhytiDvQG89lemSbMReMK+vQO4E0BENgIeY8zP7ccaMMYU3iV5Hrl6RUXWf/SyIi8uGe1309o7PGX//MWyxa6QzSyo+drTp7nvO/t5xdpqvvEn11ARtNo3fPGtLyGZNLz7a7sZsq3sxw+18sdffZbdp7u59xUrx1jNLpewfX01vzzSTiw+ttvfkdaBSQOxcPFCb80CGJzQP+/gdBc92z3+Tzg9SySfoSFg+dqzZdw45Cq8Odc9hM/typp2+oHfW88rN9SyrLKYkiIvDeVF3POy5RNmLlWHc1v07f1RrlpeAYwWsGWy71wP7/3Gc7zsY0/wyZ8doTrsZ+eJTm751C/5+s7TWT8BZcuhd3j9tibe9fIVfONPrk1d6BvLi7l6RQXfe765oPLTB2MJRhJmTDAW0qqbC8Sq9+SzSETcwG5gNfB5Y8zOjCV7gbuBzwJ3AWERqQTWAj0i8h1gBfAL4D5jzJhUAhG5F7gXYOnSpdN/NYsIl0uosIumEklDa3+U+mla9NOltiRAddg/xhVwunOQj/7wIDeuq+bf3rZtXIO1z77xCv7435/lbQ/tpKVnmOaeIarDft538xrefM349/bmDbX8565zPHuqi+ts99IP9raw71wPf/KKlZPuMeS3p0xNMyDbPhBlMJZgRR4WvdMe4WzXUKpPjEO60Odr0Tsinu1CD5af/pmTXSSTZkwM55zdhz4zrgPwmi31vGZLfV7P71Ad9mdtD2yMoa1/mFs21vLihf6sa4ZiCd744NN4XMI9L1vOW65ZysrqEGe7Itz3nX188Lsv8MN95/nE67aMcdNky6F3aKoo5n/dvnHc8ddubeDvvrvfmquQZ/HfdInE4vQPxyc1rtIbmqXjZN4cbRvg8qayWdnjVMgrGGvPft0KNAJXi8imjCXvB24QkT3ADUAzkMC6kFxvn78KWAm8PcvjP2iM2WaM2VZdXT3Nl7L4qAz66RyI0jkQJZE0U8qhnyk2N5SyzxZ6Ywz3f2c/XpeL/3P3lqw+7ZvW1/D+W9bx7KluVlQF+de3XMnv7tvO/3jV2qzrr19Thc/jSrlvfvLCBf7q28+zbVkF77t5zaT7G+1JP72A7KkOS6Dzcd00VVhuhmwWvXOsqaKIU535Cb1j+edKfVxVHWJoJDHON+4US80UNWE/HQPRcUHk/mic4ZEkNSV+1taGsgr9c2e6icQSfPZNV/D3t29MxQKaKop5+J3X8L/v2sy+c7289cvPjKm+zZYeOhm3ba7D65ZZDcoOxRI8+KvjvPzjO7jl07+a9NODk+GU6aNfVlFMebE3Z8xqrplS1o0xpgfLNXNrxvEWY8zdxpgrgA+mrT0HPG+MOWGMiQPfA668+G1fGlQEfXQOxFL/6HPtugFL6I+3DzAYjfNfu8/xu+Od3Hfb+gnjBe+5aTX7PnwLD7/rGl69uX7CIGexz8N1qyp5/FAbjx9q5S+++RxbGkt56B1XUeyb/AOn47rpn6brxrG+V+ThugkHvJQVe7MW85ztsjqQXlZfmrfrxsloqQln/106mTeZ7pvm7sikGUJToTrsZyRh6MmoAXD2Vx32s6YmzNG2gXEXg6dPdOJ2Scq9k46I8OZrlvKFt1zJyY5BvvTrk6lz2XLoJ6Os2MeN62p4dG/LjFfODo8k+PJvTnL9J3bwv3/0Ii4ReodGiEzSqjvV5ybDove4Xdy6qZ5fHGwtiHbf+WTdVItImX27CHgV8GLGmioRcR7rfuAh+/azQJmIOGb6duDgDOz7kqAy5KNrMJbKoZ9uMPZi2NxQijHwyyPtfPSHh7h6RQVvumpy91rJFNJAb95Qy5muCO/+2m7W15Xw1XdcnWpBPBnhi2xVfLJzEK9bWFKW3+82V4qlM9x9eVWQM12RvISoczCG1y2UFGV/retqw7hdMqaobHgkQcdAbEYt+ly59E6AtiYcYG1tmEgsMS7AuvNEF5saSid8v16xtppXb6rj/z5xlHPdkZw59Pnw2q0NtPVHeXqGx11+5AcH+cfHDrKuLsQjf/pS/vpVa4HJYz+jDc3G/73//pZ6BmMJdhye//yTfCz6emCHiOzDEu6fG2MeE5EHROQOe82NwGEROQLUAh8Fy+WD5bZ5XET2AwL82wy/hkVLZdBHx0A0bVbsPAi9HZD92//ex9BIgo/dvTmrX/ViuHlDDS6xfNVfe+fVlBblf5Fw6gqm66M/1TFIU3nxuPTGXDRVFKVSA9M50xWhqbyYFVXFjCRMKjNmIjr6o1QG/TkDpZUhP3df0cA3njnD+V7r8VJpiRUzKPShHEJvF0vVhP2sq7NcMunum+GRBM+f7eHaFeOt+Uz+1+0bEYR/euxQzqyhfLh5Qw0hv2dG3TexeJLH9rVw1xUNfP1d17JtecXoJ8VJXIKZDc3SuWZlJVUhPz/YO//T2vLJutlnjLnCGLPFGLPJGPOAffxDxphH7duPGGPWGGPWGmPeZYyJpt3/5/Z9Nxtj3m6MGV8do2SlMuSnb9jKzPC4hKrg1PPoL5bakgA1YT/9w3Hed/OaVDbBTFJfWsT33/Nyvv3ua8cUnuRDKJV1Mz0ffb6plQ5NFcU0dw+NcWHEE0laeoZZWlGcyt45mYefvnMwlpoPnIu/vHkNyaRJzfB1LOqpuj0mImXRD4yNBaS7llbbhWtH0jJvnjvTTSyR5NqVkxfjNZQV8d7tq/nJgQt8fecZYHqvIeB1c+umOn7ywgWGR2bGJfLb4x30D8e5PS2I7fxd9U1iQHQPjm1olo7bJbxmcx1PvNg272mWWhlbwDgicLClj5qwf8Yt6Xy5bnUVmxtKuTePLJjpsrmxdEruHodirxuR6blurNTKSF6plQ5N5cXEEkla05qAne8dJpE0LK0oTmXv5JN50zEQzZpeOOb5Kop5w1VNfPvZs5zrjqTyz2c0GGvHfjIt+vb+KD6Pi5IiD6VFXupKAmMs+qdPdOES2La8PK/nedf1K1hRFeSrvzsFZM+hz4c7ty6hPxrnqeMz4775yf4LhPweXr5mtKiwxDEgJhP6yNiGZpn8/uVLiMaTk3ZqnW1U6AsYp9f4gZZeaufBbePwz6+/nP/+s5dNWjk6H7hcQsg/vQ6WrX1RhkYSrKjK37JM5dJ3jbpmnOBsU0Ux1WGrl3w+AdnOgckteoD33rQaQfjcE8dotj/dzWRgPuhzU+R1jyuaauuPUh0adS2trQtnCH0nmxpK827L4fe4+fAdlwHkzKHPh/V1Vv+mbNlPUyWeSPKzgxe4eUPNmMJCJ213sr+rzIZmmVy5tJz60gCP7RvvvkkkzZzVBBTef66SwikW6Y6MzHkOfToul2TtZV4ohKcp9Cfz6FqZiWOFpgdkU8PdK4sREZZVBidNsTTG0D4QnbDXvsOSsiLefM3SVNZTfVkA9wx+uhORrEVT7f1RatLabqy1y/oTSZPyz1+Th38+nRvWVnPn1iVsWlIy7U+olUEfXrekWndfDDtPdtEdGeHVm+rGHA/n6RLsGoxl9c87uFzC7Vvq+eWR9jGjKNv7o/zeZ37FnZ//7aST1maCwv3vVVIWPcxPauVCIRzwjvuH3H26i7/61p4Jg2mOGE/FddNQXoTIWGvyTFcEr1tSdQ4rqoKTum4GonFi8WReFj3An9+4Co9LeP5sD41lM+efd6gO+7MGY9MvRGtrw0TjVq/4PWd6iMXz889n8qk3bOU/3/3Sae/VZX+iuTADQv/jF85T5HVzw9qaMcdH6zMmNiB6IiOpEYK5uH3LEkYShp8evABAb2SEt355J83dQ5zpinD7//0N/7Xr7Kxa9yr0BUxlWvB1PoqlFgqhLD3pv7enhe8938J7v7EnNbwlk1Mdg/jcLpZMwd/t97ipDQfGuG7OdkVoKCtKWdnLq4o52z3ESI7nBeiw5wHn676oKQnwtpcuA5hyoVE+VIeyCX2GRV/nBGT72XmyExHYliV/fjLcLsk7yykX9aWBVCbSdEkkDT95oZWb1ldT5BvbDyrky0/ouwZjkwr9lsZSllYU84O9LQxG47zjq89won2QB9/2En78vuvZ0ljK3zyyj7/81vOz1nNfhb6AKSny4LHFYz5SKxcK2Xz0B1p6KQl4+OWRdv7++weyWkunOgdZWlk8ZTdIU0XRGNfNWTuH3mFFVYhE0mRNw3Rw2k9PxU/9pzesoqzYO+UhMvlQUzLWdRONWzOK04u51jhl/a39PH2ik8uWlEwpFXYmqSstumjXze7T3XQMRLl10/iWEfnEfnI1NMtExHLf/O54J+/46rM8f7aHf3nTVq5fU019aRFff9e1/M3vreNH+8/zh198ekptrvNFhb6AEZGU/08t+tyEA2OHjySShhcv9HP3lY38+Y2r+OYzZ7L2yj/VMbWMG4em8uJxrpulY4S+2H783O6bDltU83XdWGv9/O6+7bz9ZcunuOPJqQ756YmMpNoUOJ84qtMG4gT9HhrKitjf3MueMz1cs2LqbpuZYklpgPO9wxfl7vjR/vP4PC62r6/Jet6aXpbbwh6IxhlJGCqCk1/sfv/yJSSShmdOdvGJ110+5uLidgnvuWk1//nul/I/XrlmVrLr8is/VOaNypCftv6oWvQTEA54xuQ7n+ocJBJLcNmSEv7gykbOdEX4Pz9+kaby4lTDr2TScKpzkOvTUurypbGimAvPNxONJ4jFk3RHRsZY9Klc+o5BbsrxGCkhnWLmST5tIaaDI+hO1W1b32ixVDrr6sI88WIbIwkzLf/8TFFXGkj97icKhuYimTT89MAFXrGmOmdV72QWvdOuejLXDVizcN967TI2N5byupc0Zl3zkmX5palOBxX6AsfpI67B2NxkBmOduakb7cyOT77+ci70DvPebz7HP/88yPq6MI3lxUTjySll3Dg0lRdhDLT0DKf6mKRb9BVBH+GAZ8IUS8ein+xj/1yR3gahoawoZx+eNbUhnnixDRG4ehr++ZnCyUI73zs0LaHfe66H873DvP+WdTnXTDZ43mlolo/Qiwj/+NrMXpBzhwp9gVMV8lNe7CXgnd7wkEuBkN/D8EiSkUQSr9vFwZY+vG5JjSEMeN18+Z6r+MrvTnKwxRr39+MXrAyI6fi7R3PpI6mmV+lCLyJW5s0EKZadA1b+daHUJjiC7gi8M1mqJmOq2Vr7d7qhroTSCfLHZ5u6UisgfaF3ODVZayr8+IULeN2SGmuZjVDAO+Gw964cDc0KERX6AufdN6zkts1T6y9+qeF89B4YjlMe9HGgpZe1teExuf+lxV7+6pVrUz9HYnE6B2JjXC75sjRtAEkkagl95uMsrwzy3JnunI/RMRCddHrWXOJY9E5/m7b+KCJjU3yB1LD2a1bOnzUPoxZ9yzQDsr852sFVyysmvFiFA55UJXI2nFnH0/lEMdcUhjmh5GR9XQmv2pjb6lDGTpkyxnCwpY+N9RNb6sU+z7REHiw3mtctnO2y8qBLi7zjsk+WVwVp6Rka04M9Hav9QeEIhBMUdiz69v4olUHfuDTItbVh7rqigTdsa5rzPaZTFfLjcQkXcqRYPn2iM+fA8+GRBEda+9k6yUCQkowgfyZOQ7OJKmMLBRV6ZcETTjWgGqGtP0rnYGxWUhAd3C6hoayIs90RznZHUgNJ0llRVUzSkLWlMTjtDwrHove6XVQEfWlCP0x1lj75Po+LT//hVjZMciGdbdx20VS2FMtYPMnbvvwMn/nFkaz3ffFCP/GkYfMkU6omC8ZO1NCs0FChVxY86a2KD7RY07Aum+VRc00VVl/6zNRKh9HMm+xCn2/7g7kkvWiqrT86JrWyEKkrzV4de7pzkFgiye7T2V1n++2JaZONIwwHvAyNJHIW3HVFrGKp+Wo2OBVU6JUFT8pHH41zoNnKuFlfl30O60zRWF7Mma4I57qGsrqAJupiGY0n6B+Oj/N/zzfpRVPt/dFxqZWFRi6hP2oP5D7c2p+1BcYL53opK/ZO2g8//e8qGz15FEsVCvlMmAqIyDMisldEDojIR7KsWSYij4vIPhF5UkQaM86XiMg5EfncTG5eUWBsX5KD5/tYXlmcd0fF6dJUUURPZIRYIpnVoi8r9lFe7M3al77TaX9QYEJaHfLT1mfNjl0IQl9fEqCld2hc0dQxW+iNgb1ne8fdb39zL5sbSnMOfHEIT9Lvxmp/UPhuG8jPoo8C240xlwNbgVtF5NqMNZ8E/sMYswV4APhYxvl/BH51kXtVlKykz4090NI3rXS7qdKUNjQjm9CDFZDNZtE7Ql9oFr3TwbI7EiOeNAXvuqkvK2J4JDkuBfJY2wAVQR8ijMt8cgKxk/nnYXKh7x6cvKFZoZDPhCljjHH6aHrtr8y6443AE/btHcCdzgkReQnWeMGfXfRuFSULYbt3eEuPlQWzcRYDsQ7p7pqmHJOSVlQGsxZNOcVSBWfRh/3E4kmOt1t7zjW0vFAYLZoa67451jbA5oZSVleHxgl9voFYGI395OqA2h2ZuEVxIZGXj15E3CLyPNCGNTN2Z8aSvcDd9u27gLCIVNoDw/8Za26soswKAa8Lj0t49mQXwNwIve3fdQk5u1+uqglxvnd4nI83JfTzMBpyIhwL3gloZxZLFRpOW5B0P30iaTjePsDqmhBXLi1nz5meMU3C8g3EwsQ+eqeh2VRHX84XeQm9MSZhjNkKNAJXi0hmLe/7gRtEZA9wA9AMJIA/B35kjDk30eOLyL0isktEdrW3t0/1NSiXOCJCKOBh77keYHrVrlOlIuij2OemvrQo51CWtbVWQPho2lQmSGtRHC4skRgVeiugXWhZQZlks+ibu4eIxpOsqQlx5bIyeodGOJH2qSrfQCxM7LqZSkOzQmBKlbHGmB4R2QHcCryQdrwF26IXkRDwB/balwLXi8ifAyHAJyIDxpj7Mh73QeBBgG3bts3NbC1lURHye+iJjFAV8s+Jy0FEWF4ZpHyCf/S1tVZb3yOt/VyxdLRhVedAlCKve9YalE2XmgyhL3SLviZsTdpK70t/rN26qK6uCaWK2J47081qu8VyvoFYSAvyZ7Houwfzb2hWCEz6lyYi1cCILdxFwKuAj2esqQK6jDFJ4H7gIQBjzFvS1rwd2JYp8ooyE1j+1KE5seYdPvn6yyccsdhUXkzA6+JI69hRcR0D0YKz5gGqQ9YF8mhrPyG/p+AuRJm4XUJN2D/GoncyblbXhCgJeCkJeNhzpps3bGtKBWLvXZffkPuSCXz03Quo/QHk57qpB3aIyD7gWSwf/WMi8oCI3GGvuRE4LCJHsAKvH52V3SpKDsK2P3UuhX7jkpKUpZgNl8tqrHYkw3XTORgbMz2sUCgp8uBzu4gnTcGnVjpk5tIfaxugKuSjzC5k2rq0nOdO9wBweAqBWAC/x4r9ZGuD4HSuXCg++kkv2caYfcAVWY5/KO32I8AjkzzOV4GvTnmHipIHzsfsuQjEToW1tWF+fXRs3Km9P0pjjkyd+cQZEt7cM1RwGUG5qC8NcPjC6IX0WNsAq6pHL75XLi3js48fpW94ZEqBWLB+H7laFTuFZYUex3DQylhlUeAEzuYih34qrK0N0dYfTXU6BMuiL6SGZuk4AdkFY9GXFKUmTRljONY2MOZT1pVLy+3CqR72TyEQ65BtHjGkp8gW5vuYiQq9siioCvkpLfKybJodKWeL0YHalu84mTR0DcamNCt2LhkV+sLOoXdYUhYgEkvQNxynvT9K33B8jNBvXVpmFU6d7plSINYh7Pdm9dF39Mco9hVeQD0XKvTKouC9N63mv//spQXXYMpJsXT89D1DIySSZkqzYucSR+gLvSrWIT2XPj0Q61AS8LKmJsTTJzrzrohNJ5frxmozvTB+R6BCrywSyoM+VtfMbiOz6bCkNEDI70kJfeojf4GKhONzXiium/SRgsfaLaFfk/F3cOXScp460TmlQKzDREK/UC6GoEKvKLOKiLCmNjRO6AvVondy5ws9h94hfaTgsbYBQn4PtRl7vzKthiHfQKxDOOClP5rFdVNgg2MmQ4VeUWaZdbXhlI/eqYot1GyN9XUl+NwuVlbnThstJGrCfkSskYLH2gZYVRMa54O/clkZwJQDsWAV4mVLr2zvV9eNoihprKkN0zUYo2MgSmfKoi9MkXjJsnL2f+QWGnL07yk0vG4XNWE/F3qHONo2wOosF6iVVVaV7FQDsTDquklvhTySSNJtV2EvFBZGyFhRFjDrnIDshX46BqK4XUJZUeH2SPF73PO9hSlRV1rE4dYB2vujWQvYXC7hM2/cOq1PUaGAh3jSEI0nCXit34tTLLVQag1AhV5RZh2n583h1n46B6zWtoWWHbSQqS8J8LODFwByVirftK5mWo/ttCruGx5JCb0zbrFQ3W/ZUNeNoswy1WE/ZcVejrQOLLi0vIVAXWkApxPxRC0ppoPTWiPdT+8E1KsXSLEUqEWvKLOOiLDW7nmTSJoFla2xEHBSLH0eV2pOwEyRrVWxY9EvpAu2WvSKMgesrbNSLNWin3nq7cDxyqogHvfMSlq24SOpeQIL6H1UoVeUOWBdbZj+4TjnuocKblbsQsex6FfNsNsGso8T7BiIUuxzE/QvHIeICr2izAFrakerNRdStsZCoK7EEvpsqZUXi+O66cvw0S8kax7UR68oc8LaNKFXi35maSwv4v23rOXOrQ0z/tiO0GcGYxdanEUtekWZAyqCvpQVqBb9zCIivHf7GppmoXOp46PPDMYuNIt+UqEXkYCIPCMie0XkgIh8JMuaZSLyuIjsE5EnRaTRPr5VRJ6y77dPRP5wNl6EoiwE1tVZroWqApwupWTH43ZR5HUzEE330ccW3MU6H4s+Cmw3xlwObAVuFZFrM9Z8EvgPY8wW4AHgY/bxCPA2Y8xlWAPFPyMiZTOxcUVZaDhdFRfKsArFIr2DZTyRpDsSW1DFUpDfKEEDONONvfaXyVi2Efhr+/YO4Hv2fY+kPU6LiLQB1UDPxWxaURYit22u51z30IITiUudUMBDv51e2TUYw5iF537Ly0cvIm4ReR5owxoOvjNjyV7gbvv2XUBYRCozHuNqwAccz/L494rILhHZ1d7ennlaURYFV6+o4Ev3bJvxXG9ldgkHvCmLfnRW7ML6VJbXX5wxJmGM2Qo0AleLyKaMJe8HbhCRPcANQDOQcE6KSD3wNeAdxphklsd/0BizzRizrbq6enqvRFEUZRYI+z0M2Hn0C7FYCqaYXmmM6RGRHVj+9hfSjrdgW/QiEgL+wBjTY/9cAvwQ+KAx5ukZ2reiKMqcEA54aO0bBhZm+wPIL+um2gmgikgR8CrgxYw1VSLiPNb9wEP2cR/wXaxA7SMzuG9FUZQ5IeQfDcaONjRbZEIP1AM7RGQf8CyWj/4xEXlARO6w19wIHBaRI0At8FH7+BuAVwBvF5Hn7a+tM/oKFEVRZpFwwJvqddPRH6XIu7DaH0B+WTf7gCuyHP9Q2u1HgHEWuzHmYeDhi9yjoijKvBEKeBiIxkkkjVUVuwDTYzX8ryiKMgEldhuEwVjcKpZaYP55UKFXFEWZkPQ2CAux/QGo0CuKokyI06p4YDhOx0B0wQViQYVeURRlQpwOlj2RGF0Rdd0oiqIsOkK20J/uimDMwquKBRV6RVGUCXGCsSc7BoGFVywFKvSKoigTEvJbPvqT7bbQq49eURRlcRHOsOgXYvdRFXpFUZQJKPa5cQmc6lSLXlEUZVEiIoT8HqLxJAGvi6DPPd9bmjIq9IqiKJPg5NJXhfyIyDzvZuqo0CuKokyC46dfiBk3oEKvKIoyKU4bhIVYFQsq9IqiKJOiFr2iKMoiJ2T76BdiVSzkN2EqICLPiMheETkgIh/JsmaZiDwuIvtE5EkRaUw7d4+IHLW/7pnpF6AoijLbpCz6Rey6iQLbjTGXA1uBW0Xk2ow1n8QaF7gFeAD4GICIVAD/AFwDXA38g4iUz9DeFUVR5oRF77oxFgP2j177y2Qs2wg8Yd/eAdxp3/49rNGDXcaYbuDnWIPFFUVRFgzhSyEYKyJuEXkeaMMS7p0ZS/YCd9u37wLCIlIJNABn09ads49lPv69IrJLRHa1t7dP8SUoiqLMLul59AuRvITeGJMwxmwFGoGrRWRTxpL3AzeIyB7gBqAZSOS7CWPMg8aYbcaYbdXV1fneTVEUZU541cZa/mL7apZVFM/3VqbFlLJujDE9WK6ZWzOOtxhj7jbGXAF8MG1tM9CUtrTRPqYoirJgWFJWxP+8ZR0u18KrioX8sm6qRaTMvl0EvAp4MWNNlYg4j3U/8JB9+6fALSJSbgdhb7GPKYqiKHNEPhZ9PbBDRPYBz2L56B8TkQdE5A57zY3AYRE5AtQCHwUwxnQB/2jf71ngAfuYoiiKMkeIMZkJNPPLtm3bzK5du+Z7G4qiKAsKEdltjNmW7ZxWxiqKoixyVOgVRVEWOSr0iqIoixwVekVRlEWOCr2iKMoip+CybkSkHTh9EQ9RBXTM0HZmC93jzKB7nBl0jzPHfO5zmTEma2uBghP6i0VEduVKMSoUdI8zg+5xZtA9zhyFuk913SiKoixyVOgVRVEWOYtR6B+c7w3kge5xZtA9zgy6x5mjIPe56Hz0iqIoylgWo0WvKIqipKFCryiKsshZNEIvIreKyGEROSYi9833fhxE5CERaRORF9KOVYjIz0XkqP193gami0iTiOwQkYMickBE3ldoe7T3ExCRZ0Rkr73Pj9jHV4jITvt9/7aI+OZ5n24R2SMijxXi/uw9nRKR/SLyvIjsso8V2vtdJiKPiMiLInJIRF5aSHsUkXX278/56hORvyqkPaazKIReRNzA54FXYw0qf5OIbJzfXaX4KuMHot8HPG6MWQM8bv88X8SB/2mM2QhcC7zH/t0V0h4BosB2Y8zlwFbgVhG5Fvg48GljzGqgG3jn/G0RgPcBh9J+LrT9OdxkjNmalvNdaO/3Z4GfGGPWA5dj/U4LZo/GmMP2728r8BIgAny3kPY4BmPMgv8CXgr8NO3n+4H753tfaftZDryQ9vNhoN6+XQ8cnu89pu3t+1hTxAp5j8XAc8A1WFWInmx/B/Owr0asf+7twGOAFNL+0vZ5CqjKOFYw7zdQCpzEThYpxD1m7OsW4LeFvMdFYdEDDcDZtJ/P2ccKlVpjzHn79gWsqVzzjogsB64AdlKAe7TdIs8DbcDPgeNAjzEmbi+Z7/f9M8AHgKT9cyWFtT8HA/xMRHaLyL32sUJ6v1cA7cBXbDfYl0QkSGHtMZ03At+0bxfkHheL0C9YjHXpn/ccVxEJAf8N/JUxpi/9XKHs0RiTMNZH5UbgamD9/O5oFBG5HWgzxuye773kwcuNMVdiuTrfIyKvSD9ZAO+3B7gS+FdjzBXAIBkukALYIwB2zOUO4L8yzxXKHmHxCH0z0JT2c6N9rFBpFZF6APt723xuRkS8WCL/dWPMd+zDBbXHdIwxPcAOLFdImYh47FPz+b5fB9whIqeAb2G5bz5L4ewvhTGm2f7ehuVXvprCer/PAeeMMTvtnx/BEv5C2qPDq4HnjDGt9s+FuMdFI/TPAmvsDAcf1kepR+d5TxPxKHCPffseLL/4vCAiAnwZOGSM+VTaqYLZI4CIVItImX27CCuOcAhL8F9nL5u3fRpj7jfGNBpjlmP9/T1hjHlLoezPQUSCIhJ2bmP5l1+ggN5vY8wF4KyIrLMP3QwcpID2mMabGHXbQGHucXEEY+3Ax23AESy/7Qfnez9p+/omcB4YwbJU3onlu30cOAr8AqiYx/29HOvj5T7gefvrtkLao73PLcAee58vAB+yj68EngGOYX189hfAe34j8Fgh7s/ez17764Dzv1KA7/dWYJf9fn8PKC/APQaBTqA07VhB7dH50hYIiqIoi5zF4rpRFEVRcqBCryiKsshRoVcURVnkqNAriqIsclToFUVRFjkq9IqiKIscFXpFUZRFzv8PNwB0L7vyPCkAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 验证模型\n",
    "\n",
    "在训练结束后，对获得的模型进行验证。这里，我们向网络中输入一个字母并推理得出下一个字母。将输出的字母作为下一步的输入，重复直到EOS标记处。"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "source": [
    "max_length = 20\n",
    "\n",
    "# 根据类别、起始字母、隐藏状态开始推理\n",
    "def sample(category, start_letter='A'):\n",
    "    category_tensor = category_to_tensor(category)\n",
    "    input = input_to_tensor(start_letter)\n",
    "    hidden = rnn_cf.initHidden()\n",
    "    output_name = start_letter\n",
    "    \n",
    "    for i in range(max_length):\n",
    "        output, hidden = rnn_cf(category_tensor, input[0], hidden)\n",
    "        topk = ops.TopK(sorted=True)\n",
    "        topv, topi = topk(output,1)\n",
    "        topi = topi[0,0]\n",
    "        if topi == n_letters - 1:\n",
    "            break\n",
    "        else:\n",
    "            letter = all_letters[topi]\n",
    "            output_name += letter\n",
    "        input = input_to_tensor(letter)\n",
    "        \n",
    "    return output_name\n",
    "\n",
    "# 遍历提供的字母，得到输出名称\n",
    "def samples(category, start_letters='ABC'):\n",
    "    for start_letter in start_letters:\n",
    "        print('语言类型：%s 首字母：%s 输出结果：%s' %(category, start_letter, sample(category, start_letter)))\n",
    "\n",
    "samples('Russian', 'RUS')\n",
    "samples('German', 'GER')\n",
    "samples('Spanish', 'SPA')\n",
    "samples('Chinese', 'CHI')"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "语言类型：Russian 首字母：R 输出结果：Rrnarao\n",
      "语言类型：Russian 首字母：U 输出结果：Uilehidaauritai\n",
      "语言类型：Russian 首字母：S 输出结果：Sa\n",
      "语言类型：German 首字母：G 输出结果：Gallh\n",
      "语言类型：German 首字母：E 输出结果：Ehuhkiakena\n",
      "语言类型：German 首字母：R 输出结果：Rcsahonuianah\n",
      "语言类型：Spanish 首字母：S 输出结果：Stadlalugtnaaa\n",
      "语言类型：Spanish 首字母：P 输出结果：Perahaiarsaorrol\n",
      "语言类型：Spanish 首字母：A 输出结果：Aaan\n",
      "语言类型：Chinese 首字母：C 输出结果：Cadco\n",
      "语言类型：Chinese 首字母：H 输出结果：Hn\n",
      "语言类型：Chinese 首字母：I 输出结果：I\n"
     ]
    }
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore-python3.7-aarch64",
   "language": "python",
   "name": "mindspore-python3.7-aarch64"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}