next_path_prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next web page prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is comparison of different classifiers for a problem of predicting next web page which user will visit based on the current one. Goal is to use this findings to build a WP plugin for link prefetching using the `rel=\"prefetch\"` attribute."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`PagePaths` class is responsible for managing connection to the Google Analytics API and obtaining data from it. First argument is user's profile ID (ga:XXXXXXX), second is path to the client credentials JSON file and third start date. End date is 'today'. Check this [tutorial](https://developers.google.com/analytics/devguides/config/mgmt/v3/quickstart/installed-py) for more about GA setup. \n",
    "\n",
    "`get()` method returns the data directly from the API which can be written to CSV file with `write(data, file_name)`.\n",
    "\n",
    "`paths` is a list of all web page paths (string) in the dataset. \n",
    "`connections` is a list of path indices (from, to) and number of transitions between them for certain date."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prev</th>\n",
       "      <th>next</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>17403</th>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17404</th>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17405</th>\n",
       "      <td>1</td>\n",
       "      <td>477</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17406</th>\n",
       "      <td>515</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17407</th>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       prev  next  count\n",
       "17403     3     6      1\n",
       "17404     0     6      1\n",
       "17405     1   477      1\n",
       "17406   515     1      1\n",
       "17407     6     3      1"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import configparser\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from page_paths import PagePaths\n",
    "\n",
    "config = configparser.ConfigParser()\n",
    "config.read('config.ini')\n",
    "\n",
    "pp = PagePaths(config['GA']['viewID'], config['GA']['clientsecret'], '2010-01-01')\n",
    "\n",
    "# paths, connections = pp.get()\n",
    "# pp.write(paths, 'paths.csv')\n",
    "# pp.write(connections, 'connections.csv')\n",
    "\n",
    "paths = pd.read_csv('paths.csv', names=['path'], engine='c')\n",
    "connections = pd.read_csv('connections.csv', names=['prev', 'next', 'count'], dtype=np.int32, engine='c')\n",
    "connections.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split connections to train and test sets (70:30)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "paths(classes): (516, 1)\n",
      "train samples: (12185, 3)\n",
      "test samples: (5223, 3)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "connections_train, connections_test = train_test_split(connections, test_size=0.3, shuffle=True, random_state=22)\n",
    "\n",
    "print('paths(classes): ' + str(paths.shape))\n",
    "print('train samples: ' + str(connections_train.shape))\n",
    "print('test samples: ' + str(connections_test.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before further usage, we need to split connections to feature (X) and target (y), and one-hot-encode first. X, as being quite sparse, is transformed to compressed sparse row matrix, and y to numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "from scipy.sparse import csr_matrix\n",
    "\n",
    "def encode(data):\n",
    "    X = np.zeros(shape=(data.shape[0], paths.shape[0]))\n",
    "    y = []\n",
    "    counter = 0\n",
    "    for index, row in data.iterrows():\n",
    "        X[counter, row['prev']] = row['count']\n",
    "        y.append(row['next'])\n",
    "        counter += 1\n",
    "    \n",
    "    return csr_matrix(X), np.array(y)\n",
    "\n",
    "X_train, y_train = encode(connections_train)\n",
    "X_test, y_test = encode(connections_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we are interested in n >= 1 recommendations, probabilites are predicted for each path in the dataset and then we are extracting top `PREDICTIONS_NUM` for single path. Also, accuracy is calculated for each set of predictions at step `i` and then summed up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import metrics\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "PREDICTIONS_NUM = 10\n",
    "\n",
    "def get_predictions(classifier, X):\n",
    "    probabilities = classifier.predict_proba(X)\n",
    "    predictions = [p.argpartition(-PREDICTIONS_NUM)[-PREDICTIONS_NUM::][::-1] for p in probabilities]\n",
    "    \n",
    "    return np.array(predictions)\n",
    "\n",
    "def get_accuracy(y_true, y_pred):\n",
    "    accuracy = 0\n",
    "    accuracy_cumulative = []\n",
    "    \n",
    "    for i in range(0, PREDICTIONS_NUM):\n",
    "        accuracy += metrics.accuracy_score(y_true, y_pred[:,i])\n",
    "        accuracy_cumulative.append(accuracy)\n",
    "    \n",
    "    return accuracy, accuracy_cumulative\n",
    "\n",
    "def plot_accuracy(data, title):\n",
    "    legend_names = []\n",
    "    for row in data:\n",
    "        legend_names.append(row[0])\n",
    "        plt.plot(row[1:-1])\n",
    "    \n",
    "    plt.legend(legend_names, loc='lower right')\n",
    "    plt.ylabel('accuracy')\n",
    "    plt.xlabel('number of predictions')\n",
    "    plt.xticks(np.arange(PREDICTIONS_NUM + 1, step=2))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we test few classifiers and comparing them considering accuracy and execution time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd4FFXfxvHvyab3TkgjAQIhpFBC71Wkgw0QBekdQRSER8TyIPgoFkQUEVFfQWxIFenSpIReQoeQDum97Z73jw0QIGgoyybhfK6LazOzM7O/CTD3zpw5Z4SUEkVRFEW5nYmxC1AURVHKJxUQiqIoSqlUQCiKoiilUgGhKIqilEoFhKIoilIqFRCKoihKqVRAKIqiKKVSAaEoiqKUSgWEoiiKUipTYxdwr1xdXaWfn5+xy1AURalQDh48mCSldLuXdSpcQPj5+REREWHsMhRFUSoUIUTUva6jLjEpiqIopVIBoSiKopRKBYSiKIpSKhUQiqIoSqlUQCiKoiilUgGhKIqilEoFhKIoilKqCtcPQlEU5V5oi3QkRWeRn1OIlCClREqg+FVKidTpXynx/h3TutLXg1u3ccf2i9dz97PHP9TViL+Je6cCQlGUSqUwX0vCpXTiz6URdz6NxIsZFBXqjF0Wwa29VEAoiqI8SnnZhSRcSCeuOBCuRWWi00mEABdvW4JaeeJZ0xEbRwsAhBAIE/0ronha3DYfSlnm5vvXl9e/d/NnCSRnFxCbnktceh6xabnEpeUQm5aHla85bYz4e7ofKiAURalQstPziT9fHAjn0kiOywIJJhpBFT976nXyxTPAEY8aDlhYPdxDXKFWR3xaHjFpOcSm5hKblnvzNS2X+LQ8CrS3nq04Wpvh5WiFTsqHWsujYNCAEEJ0AT4BNMBiKeWc2973Bb4FHIuXmSalXG/ImhRFqTiklGQm5904O4g7l0b61VwATM1N8KjuQOPu/njWdKSKvz2m5poH+rzcAi2xaTnE3H7wL35NyMjjluO8lPhZ6qhhUURH00K8HfOpIvNw0eZiX5CNTW4WIj4N7alUbK3aQ6vqD1Tfo2awgBBCaIAFQCcgBjgghFgtpTxVYrH/AD9JKRcKIYKA9YCfoWpSFKV8kzpJSkL2LWcI2Wn5AFhYm1K1piN1W3rhGeCIq68tGk3Zb8SUUpKeW3jXg39sWi5ZGdnY52fjUJCFQ0E2ToU5eJvkU4d83HW5OBbmYJefjWVOJqZZ6ZCeDkVFpX+emSmZDnZo7a0ptLMiQ5tKxWqBMOwZRGPgvJTyIoAQ4kegF1AyICRgX/yzAxBnwHoURSlndFodSTFZN8Ig/nw6edmFAFg7mOMZ4IhnTUc8AxxxrmqDMBFl3nZOQRH7L6Ww71QM0QePkxOXgEV2Jg4F2TjkZ2NfkE31wmwaaXNxLMjCOjcLs8KCUrclBRTaWpJna06WrSmJjiZkVjUh3cqBVEsdyZaFJJnnk2apI8MaMq0gz1yCyAQyAXi2VmNCH/g39mgZMiC8gOgS0zFAk9uWmQVsFEKMB2yAjgasR1EUIysq1HL1cuaNS0YJF9IpzNcCYO9qiV+oC54BjlSt6YiDm9WNBuOyyC/UcuzgGc7sOUjq8WNYx5zBLy2B7pmZd3T4yrMwIdNakGkNybY6LltBhjVkWJuQaQWZ1pBhJYpfIdsSLMxMsTGzwtbcFhszG2zNbr46mdnga253c775re/bmtviaOH4EH+Tj4axG6n7A0ullB8KIZoB3wshgqWUt7TyCCFGACMAfH19jVCmoij3Kzstn1O744iOTCHxcga6Iv1FfGdPG2o39cCzpj4QbJ0syrzNotwc4o/t58zePaQcP45VVDQeV9OwzdfSENABiU4Q5SnY6W5CopcVRe7O4GCPiaM9Vtb2tx7kzW3xM7O9cTC/PQBszGww05gZ5hdUjhkyIGIBnxLT3sXzShoKdAGQUv4thLAEXIGrJReSUi4CFgGEh4dXvFsBFOUxlHAxnWPbYrhw8Co6KXH3tSO0rbf+DKGGI5a2/3zAzS3KJSYjhriok6SdPELh2XOIc9E4xqRRJakQjdRfpnAxgyh3wf66VmT4uGERWAPHunXxcq9BYztv+tp642Dh8Gh2upIxZEAcAAKEEP7og6EfMOC2Za4AHYClQog6gCVwzYA1KYpiQNoiHRcOXeXo1hiuXs7A3FJDSFtvQtp54eBmfcuyUkqS85KJzowmJjOGmNTLZJ47DecuYXU5Efe4XKpdlbjngHvxOlftNVx2tmGvnw+FftVxC6tH86at6Obpj5nJ4/cN39AMFhBSyiIhxDjgT/S3sC6RUp4UQrwNREgpVwOvAF8JISahb7AeLGUFvFlYUR5zORkFnNwZy4kdseSkF+BYxZpWz9UisJkHwlxyOPEw5yLP6YMgK4aUhMuYXoihakIBfomSalclLZPAtPjicqHGhBgXB/Z5unDR3ptElwA8whoRHlyN1jVd8XOxvqf2CeX+iIp2PA4PD5fqmdSKUj5cu5LJsa3RnI1IRFck8Q1yJrS9D1UD7YhIjODPqD/568Jmqp9Oo2acxP+aCf7XBA7pN28NzXe0J6mKN2dsvDhg4s5F+6qkOHsQXsONFjVcaV7ThToe9pjcwx1Myp2EEAellOH3so6xG6kVRalgdFodF48kcWxbNPHn0zG10BDUwpOgNh6clSdYEvUpW37ejMeFdDqc0vBJpA7zXB2YajCvXoPs8Oqctq/KPuHMpgJ7kk1tMNMI6vs60aKGK+NquhDq7Yi5qRps2thUQCiKUiZ5WYWc2h3H8e0xZKXmY+diSdOn/MmqHsuWhB+YsWMrlglpdDhlyrxIDXbXtAgrMyzbd+ZQ7aasNanK3isZ5BRoEUBwVQeerulC8xquNPJzwtpcHY7KG/U3oijKP0qOzeLYthjO7kugqFCHZy0HXDtp2We2gUWxWym6kEbbc2b897Q17he0IHTYNGtKzvAnWG5enZ9OJpN7UUt1twKeauBNi5ouNK3ugqO1ubF3TfkXKiAURbmDTieJOp7E0a0xxJ5JRWNmgkMwnPXaz7LMNWRfTqdplAX/OeeI73GBKMzDvKYX9q+8xNHaTVhyJps9p5KxME2iVz1PBjX3o66nutW0olEBoSjKDfm5RUQWX0bKSMrDzA4y61/gT6sVJOkSCI60YspFN2odzMckIxuNixUOAwZg0rkrv2ba8P3eK8Suv4KngyVTuwTSr5EPTjbqTKGiUgGhKAqpCdkc3xZD5N/xFBXoKHBLY1/QBiLt9uGdZ8mwU16EHtRiGp2AMC/ArmMH7Hv2JKZGKB/tj2Hlyljyi3Q0re7MG93r0LFOFUzvYSA9pXxSAaEojympk1yJTOHolitEn0pFmui45HaUQ+6bwTqVgYk1eH2PH+ZHzgIZWIeHYz9iDNYdO7ElJpeley6zf8PfWJqZ0LeBN4OaVyPQw/5fP1epOFRAKMpjpiCviFN7YonYcoH8ZMg1z+SE906iPY/QJ78aA4+7YbUnAZl3CPNq1bCfMB6Hnj3JdHRj2YFo/u+Lg8Sn5+HtZMX0roE8G+6jGpwrKRUQivKYSE7MYPP6g1w9VIBJoRlXba5wrvZe6rubMj5SYLckH23SHkwcHLDv0xvHXr2wDAvjZFwGc/dcZvXR4xQU6WhZ05W3ewXTPtAdjeq8VqmpgFCUSu7Q+eNs++kUFldc0AlJtGskzrUy6Ho1H9dt8RScPQtmZli1aY1Dr17YtmmDVmPKHycS+PaLvzkYlYq1uYZnw70Z1MyPgCp2xt4l5RFRAaEolZBWp2Vb1HY2rj+A58kwNNiTHXiJ5tY59Nx3nNxf94JOh0lYKFVmvoH9k09i6uTEtcx8lu6M4v/2RnE1M59qLta80T2Ipxt642ClBsN73KiAUJRKJLMgk5XnVrImYiN1jrenWlZjzDwyaZVzCJPv1qLLyaHI0xOXkSNw6NkTC39/AI5Ep/HtxiOsOxZPgVZH61puzHmqGm1ruasxkB5jKiAUpRKIzohm2ellrDq7mtqXWtA6bhBmFoL6DpHY/fIFQqPBrns3HHr1wjo8HGFiQn6RlpWHY1i6J4qj0WnYWpgyoIkvLzSrRg03W2PvklIOqIBQlApKSklEYgTfn/qe7dHb8cyqybNR0zDLsMHXLplq2z7BLC8dx2eexnXUaMyq6J+qkJiRxw/7rrBs3xWSsvKp7mrDrB5BPNXQGztLdRlJuUkFhKJUMPnafNZfXM//Rf4fZ1PP4qbxYET6TMQpZ6zNCql19muc4w9j36M7buPGYe7ri5SSg1EpLN0TxR/H49FKSbva7gxq7kermq7qMpJSKhUQilJBJOUmseLMCn468xMpeSkEOAUwvcps8rY5kpNegG/SHvwif8GxbUvcFv6OZe1apOUU8OPfl/kpIpoTsRnYWZjyYjM/XmxWDT9XG2PvklLOqYBQlHIuMjmS/4v8P9ZfWo9Wp6WNdxue832erK3WXDichG1+LA1PfEvVoCq4/7AU89Awdp67xs/LDrHpZCIFWh2BHna80zuYvvW9sLFQ/+2VslH/UhSlHNLqtGyP3s73kd9zMPEgVqZWPFPrGQYEDiD3pDm7Po2ksCAD/8t/UNsuDo9P3iChZgjzD8Xy2/qtJGTk4WhtxoAmvjzd0JtgLzWSqnLvVEAoSjly/TbVZaeXEZsVi6eNJ1PCp9AnoA+6NFO2frSP+ASJQ9oFGufupOqE59nmVoefD8YSsXoHJgLa1HJjZo8gOtRxx8JUY+xdUiowFRCKUg5cybjCstPLWHluJTlFOTRwb8Ar4a/QzqcdJtKEiO/3cfDvTIS2kDrJ27F7IozvHCexfu9VcgtPUN3NhqldAunbwIsq9pbG3h2lklABoShGIqXkQMIBvo/8nr+i/0JjoqGLXxcG1hlIXde6AMTuPM62H86QjjOumecx8dPxTo2OXLpchK3FNXrX9+Tphj408HVECHUnkvJwqYBQlEfs9ttUnSycGB46nOdqP4e7tb6vQs75S+z6ZDPnC/wwLTIDk+O86elDns6C5q4OTOjiTZe6VbEyV5eQFMMxaEAIIboAnwAaYLGUcs5t738EtCuetAbcpZSOhqxJUYzl9ttUazrW5K3mb9HVvyuWpvrLQoWJiUR+/AMRsZ7kWtegIO8yXzvbY101hNENfOjbwAsfZ2sj74nyuDBYQAghNMACoBMQAxwQQqyWUp66voyUclKJ5ccD9Q1Vj6IYS05hDnMPzGX1hdUU6Ypo492GgUEDaeLR5MZloaLUVC7O/4oThwqJ92iGziyTrVbp+LdvwGcNfWji76w6symPnCHPIBoD56WUFwGEED8CvYBTd1m+P/CmAetRlEcuOTeZsVvGEpkSybO1nuX5Os/j5+B34/3c9AwOz/uC3L9OcNG/D/lV7LhqX0BQnzC+q++lhr5QjMqQAeEFRJeYjgGalLagEKIa4A9svcv7I4ARAL6+vg+3SkUxkOiMaEZuHsm1nGt80u4T2vq0vfHeiUtXOfb5N3hv+5OEat24FjgYrCUtXgymfr0qRqtZUUoqL43U/YBfpJTa0t6UUi4CFgGEh4fLR1mYotyPk0knGbNlDDqpY/ETiwlzCyMlu4DfI65wZfnPtN23GnfHIE42eBVpakGTntWp38kXjcbE2KUryg2GDIhYwKfEtHfxvNL0A8YasBZFeWR2x+5m0vZJOFk48UWnL3A09WL8soNk/vknz5/cQJgWToaNJtPSG69ajrR9PhDHKqrhWSl/DBkQB4AAIYQ/+mDoBwy4fSEhRCDgBPxtwFoU5ZFYc2ENM3fPpIZjDRZ2XEhapiWjPv6Fgdu+oXp6HFdCn2W/S0tMLcxo91RN6rSoqvovKOWWwQJCSlkkhBgH/In+NtclUsqTQoi3gQgp5eriRfsBP0op1aUjpcKSUrLkxBI+PvQxTao24eO2H3PgQja/zZ7H64d+Jd8tgKOtXiM1U0ONMDda9auFjYOFsctWHoSUUJANeWmQmwa5qcU/p942XfxzQGdoNsbYVd8Tg7ZBSCnXA+tvmzfztulZhqxBUQxNq9Py/oH3WXZ6GU/6P8m7zd/l/7acJf9/sxkTc5i4pgM5Y90MKxNznhxZm+r13YxdslJSUb7+IP5PB/eS75X8WVd49+0KDVg5gZUjWDqC1D26fXpIyksjtaJUSPnafF7f+TqbojYxKGgQY+tNZN5na2j83Tzc8tK53PddLqY4US3IhU5DgrCwVretGoxOpz945yTf+Sc76e4hUJjzz9u1cNAf5K0c9Qd8e8+bP1s63gyBG9PFP5vbQgW/fKgCQlHuU0ZBBhO3TiQiMYIp4VPo6vMsC8e8S9ddv5Lr4snZXv8lIV5L/c6+NO1dQ3V0uxfXL9/cOMinFL8m3XbwT9Ef/HOSITfl7t/STS3B2uXmAdzZHyzr3zzwl3qgdwJLBzB5fIczUQGhKPchMTuRUZtHcTnjMnNazaF6fjBb+77AkzEnudq4K1e8+5B9rYiOg+tQu2lVY5drfNrC0r/V3zjwlzz4F88ryit9W0KjP9hf/+NWG2xcb51n7QzWJeaZq7vE7ocKCEW5RxfSLjBq8ygyCzJZ2HEheTvSSJ7zHHXys7k66A3OJHphVgS9X6mPh/9j+KCevHRIOAEJxyDhOMQfg2un73693tLh5oHc3gs8wooP8MXzbjn4O+sv+Zio/iKPggoIRbkHhxIPMX7reMw15nzTYTGX5v5KtfUrSHJ0J3ncx5yKyMHNx5quo0Owdarkz2WQEjLj9QGQcBwSjupfUy/fXMbGDTxCoWYHcPS59Vv99QO+RrXLlFcqIBSljLZEbWHqzqlUtanKR0GziHrxDfyjThMZ1har9iO5eCCFgHB32r1YB7PKNgy3TgvJ54vPCIqDIOGY/lLQdc41oGo9aPCiPhQ8QsDOw3g1Kw9MBYSilMGK0yuYvX82wS7BzCx8irTnRuJUWMjxgVMQpqFcPJxCk57VafhktYrf8a0gB66euvUSUeJJKMrVv68xB/c6ULurPgiqhkKVumBhZ9y6lYdOBYSi/AMpJfMPz+er41/Rrkorhm53Rvf7DBIdvdBNe5ecw0Xk5+Xw5KgQqtergP0bspOLg6BEGCSfu3k3kKWDPgTCX7p5VuBWW10WekyogFCUuyjUFfLO3++w8vxKXrTvRMdPzmN+aRub67QleNhkzmyIxdrenKdfq4eLl62xy/1nUkJaVIn2guLXjBLDo9l76wOgbm/9q0coOPpW+Hv5lfunAkJRSpFTmMOUv6awM3YnMzPaEjhvG7k6wS/dxtG+SSdOrYnGM8CRLiOCsbIzN3a5pSvMg4vb4fQaOPPHzfYCYQKutaFaC/3loethYO1s1HKV8kcFhKLcJiUvhXFbxnE+/gQLDwbjsm0zJ1z8OfHCKzQtcuX01hiCWnnS+rlaaEzL2e2W+ZlwbhNEroFzG6EgCyzsodYTNwPBPQjMrIxdqVIBqIBQlBJiMmMYtXkU5hdjWbTeCbPYoyyr3RHvYaMJPJRJdGIKrZ6rRUhbr/LTGJ2Toj9DiFwDF7aCNl9/e2nI0xDYA/xbg2k5PctRyjUVEIpSLDI5ktGbRtFyfzb9NxWRbprHJ23HMPCprlzdEEu2lPSYEIZPYDm4FJMRB6fXQeRquLwbpBYcfKDRUKjTA3yaPNZDRCgPhwoIRQH2xO1hxvqXGbW+iLBTuUR4BLK83WBeaRjM2ZVROLhb0XVMKI7uRhyyIfkCnF6rP1OIOaCf51oLWr6sD4Wq9VSDsvJQqYBQHntrL67lux9nMHu1xD5Dx1d1uxPfsRcv27hwem0U1YJd6DS0LhZWj/i/i5T6/gfXQyHxhH5+1XrQ/g19KLjVfrQ1KY8VFRDKY0tKybfHl3JhwQe8uVNHhr0rk1oNoFG7ZjwTp+Pc0Xjqd/KlaZ9HOBKrTgexB/WXjiLXQOolQIBvM3jiPQjsBk7VHk0tymNPBYTyWNJJHZ9ueouq835mwGXJ4RoNmF2nD5PahWD+dzKJafl0GFyHwEcxEqu2CKJ26wPh9Fr9+EYmZvrG5RYT9aFg6274OhTlNioglMdOgbaABYtG0PSrfdgWaviqydNs8mvKnGYBxG2IochcQ5/JDfCobsCRWAvz4OI2iFwLZ9bpH15jagUBHaFOT/3jKa0cDff5ilIGKiCUx0pGdiq/vfYMXbbEkuLhxMSQoUjf6syr7sXZVVG4+dgZbiTW/Ex934TINfq+CgVZ+qGra3fRtyfU6KCeW6CUKyoglMdG/PmjHBvzEk2u5HKsUTAzqwygca2q9Be2nN0UQ81wd9obYiTWqD2w5zM4v/nWPgp1eoCf6qOglF8qIJTHwrnfviPj7Tm4Ssnynj35zqQ1L9X3ps7FAi5FXX34I7FKqQ+EnR/Clb/1z0FoNKy4j0Jj1UdBqRBUQCiVmjYjg5P/mYzZxt0kepmyuO1IjuVWZ1bzGrDzGql52oc7EqtOq78DaeeH+sHw7L3hyfeh/gvq8pFS4Rg0IIQQXYBPAA2wWEo5p5RlngVmARI4KqUcYMialMdH1q7dXJg6CU1KJhvaOvG91xh00p2Pm/kT82cs1nbmPPVqPVy9H8JIrEUFcGwF7P5Y/2AdlwDo9TmEPKMuISkVlsECQgihARYAnYAY4IAQYrWU8lSJZQKA14EWUspUIYS6l095YLrsbOLfn0vGip+56gJrR4ey7upz+Fk7M6WqBxfWRVO1pgNPjgx58JFYC3Lg0Hew51P90NkeofDMt/pLSeoyklLBGfIMojFwXkp5EUAI8SPQCzhVYpnhwAIpZSqAlPKqAetRHgM5ERHETJtKUWwcaxsLLvZ6ij8jGtPMx4XndNZc2BFHUEtPWvd7wJFYc9PgwGLY+7l+GG3f5tDjU/2zl9VwF0olYciA8AKiS0zHAE1uW6YWgBBiN/rLULOklBtu35AQYgQwAsDX19cgxSoVmy4/n2sff0LK0qUkOWlYONACl5BxbNhTla7+brRMkMQmptLquQBC2nrff2N01jV9KBxYDPkZULMTtJoM1Zo/3B1SlHLA2I3UpkAA0BbwBnYIIUKklGklF5JSLgIWAYSHh8tHXaRSvuUeP0HctGkUXLjA1oZmrO3iQoDNJFbv0TCgRhVqns0lRyvpMT4Mnzr3ORJrWjTsmQ+HvoWifP1T11pOgqphD3dnFKUcMWRAxAI+Jaa9i+eVFAPsk1IWApeEEGfRB8YBA9alVBKyoICkL74g6ctF5DtY8cFzJmgb1aVa5nBW789ldHUPHI5mYuZkQfexoTh52Nz7h1w7q294PrZCPx3WD1q8DK4BD3dnFKUcMmRAHAAChBD+6IOhH3D7HUq/A/2Bb4QQrugvOV00YE1KJZF35ixxr08j/1Qk55p6899m8bQJ7EHixe5sOJ3Oqz4ecCgd95oOPDkqBCvbe2yMjjsCu+bBqdVgaqnvw9BsHDj6/Pu6ilJJGCwgpJRFQohxwJ/o2xeWSClPCiHeBiKklKuL3+sshDgFaIFXpZTJhqpJqfikVkvykiUkfTof7GxZPsiX3z3jGRU6kS17gzgalcYMV3cKTqRTq0kV2g+sg8bsHhqjo/bo+zCc36x/VGerydBkNNg+pH4SilKBCCkr1iX98PBwGRERYewyFCPIv3SJ+Nenk3vkCNo2jZnW5BJJlgVMD3+Xz9ebE5uYxURLJ/Ljc2nS05+GT/qVrTG6tF7PzcbozxosDThgn6I8QkKIg1LK8HtZx9iN1Iryr6ROR+oPy7j64YcIc3PipjzHq+ar8LCtygcN/sf0n65SlJbDBJ09Rdfy6TysLgHhVf59wzotnFqlv5Skej0ryh1UQCjlWmFsLHEz/kPO3r1Yt2rJqme8+Sr+F5p4NGFk4CzGfH8atxxJzxwrNGaSHpPr//sw3dd7Pe/6CFIugEtN1etZUUqhAkIpl6SUpP/2G4mz3wMpcXxzOm+57GJ3/C8MCBxAe/ehDF1ymNACU5qnaXDwsKTbmFDsXa3uvlHV61lR7okKCKXcKbx6lYQ3ZpL1119YN25M0eujGXH6v8QkxvBmszdx0bVm0NcH6VxkQe008AlyovPw4Ls/M7owV9+57e8FqtezotwDFRBKuZK+bh0Jb7+DzMujyvTXiWznz6s7J2OmMWNx58XExHswekUEzxRZ4ZEpCW7jRatnAzDR3OVOpbMb4Y9XIfUy1OwIrV5RvZ4VpYzKFBBCiN+Ar4E/pJQ6w5akPI6KUlNJePttMv/YgGVYKFXfe48VuTuZt20cNR1rMr/9fDYdK+CDVUcYXGCNfZ6kxbMBhLa7y7AZadGwYZr+Gc+uteDF1VC9zaPfMUWpwMp6BvE58BLwqRDiZ+AbKeUZw5WlPE4yt24lfuabaNPTcZs0CbvBA3nnwGxWXVhFR9+OvNviXb7aEcuyDecZUmCNjTCh85i6+IW43rmxogL95aS/5upvX+3wpr6Dm2p8VpR7VqaAkFJuBjYLIRzQ93zeLISIBr4C/q94qAxFuSfazEwSZ79H+sqVWAQG4vv1YrJ8XRi6ZQRHrx1ldNhoRoSM5J21kezaEc0LeZbY25vTbWxY6c9wuLwL1r0C105D7W7Q5T1wqvbod0xRKokyt0EIIVyAgcALwGHgB6AlMAj9YHuKUmbZe/YQN+M/FF29isvoUbiNHk1k5nkmrO1HRkEGH7b5kHY+HZm84ghx+6/RJ8+CKtXs6DomFBsHi1s3lpkIm97Q37rq6Av9V0DtLsbZMUWpRMraBrESqA18D/SQUsYXv7VCCKG6NStlpsvJ4eoHH5K6bBnm1avjt3wZVqGhbLi0gTd2v4GTpRPfPfkd1WwDGLH0AGZH02lXYEb1+m50fCkIM/MSt6PqtBCxBLa8A4U50PpVaDlZdXJTlIekrGcQn0opt5X2xr123VYeXzmHDhE37XUKo6NxHjQIt0kvg4U58w/PZ9GxRdR3r89HbT/CVNrz0qK9+J7OoVqRKQ2eqEbTXtURJiUao2MOwrpJEH8UqreFrh+oEVYV5SEra0AECSEOX39OgxDCCegvpfzccKUplYXUarn20Uckf70EMy8vfL9dik3jxuQU5vD6tklsjd5K34C+zGgyg7RsHcO/2EO9qEJcpIb2LwZSp7nnzY2iHfB9AAAgAElEQVTlpMCWt+HgUrCtAk8vgbp9VX8GRTGAsgbEcCnlgusTxc+PHo7+7iZFuSup1RI3dRoZa9fi+OyzuL/2GhpbG2IyY5iwbQIX0i4wrfE0BgQO4EpKDpMW7KN5vMTWwoweo0Pxqu2k35BOB0eX69sactOg6RhoOw0s7Y27g4pSiZU1IDRCCCGLh34VQmgAdd+g8o9KhoPb5Mm4jhgOwIGEA0zePhmt1LKw40KaezYnMj6DN+fvp22KwMbZkqcm1sexSnFbQuJJ/d1JV/4GnybQ7UPwCDHininK46GsAbEBfYP0l8XTI4vnKUqp7hYOP535iff2vYePvQ/z28+nmn019l9KZuFnh2iTrcGpmh19x9fD0tYM8jNh+xzYu1A/7HbPz6De82ByD893UBTlvpU1IKaiD4XRxdObgMUGqUip8EoLh0JdIXP3z2XFmRW08mrF3NZzsTO3Y8uJBFZ9dZzwfA0+Dd3o9lJdNBoBJ1fChtchMx4aDtZ3eLO+z+dJK4pyX8raUU4HLCz+oyh3VVo4ZBRkMHnbZPYl7OOlui8xscFENCYaft0TxaFl56hdpCG0azVa9qiOSL4A66fAxW360Vaf/R58Ghl7txTlsVTWfhABwHtAEGB5fb6UsrqB6lIqoNLCITk3mVGbR3E+7Tz/bflfetboCcCSdWeIXxdNFWlCm8GBBDd0gm2zYffH+mdAP/k+hA8FjRpPUlGMpaz/+74B3gQ+AtqhH5dJXQhWbigtHBKyExi+Uf/6WfvPaOHVAikln/xwHN2ua9iYaegzoR4+8gAsmAJpURDyLHR+F+zK8EQ4RVEMqqwBYSWl3FJ8J1MUMEsIcRCYacDalAqitHC4nH6ZEZtGkFmQyZedvqRBlQbodJK5CyKwPZmBtDVl0BgvnPaPuzni6qA14N/a2LujKEqxsgZEvhDCBDgnhBgHxAKljJamPG5KC4czKWcYsWkEAEueWEIdlzrkFWj539y9OMfmU+Ruzpi2h7BY3k+/kY6zoOlYNeKqopQzZb1MNBGwBiYADdEP2jfo31YSQnQRQpwRQpwXQkwr5f3BQohrQogjxX+G3UvxinGVFg5Hrh7hpT9fwszEjKVdllLHpQ7pWQV8MHMXzrH52PrmMN7tVSx2zIQa7WHsPmg5SYWDopRD/3oGUdwp7jkp5RQgC337w78qXm8B0AmIAQ4IIVZLKU/dtugKKeW4eytbMbbSwmFP3B5e3vYy7tbuLOq0CE9bT+JSc1gwex+emZK63ntoW/A/0KoRVxWlIvjXgJBSaoUQLe9j242B81LKiwBCiB+BXsDtAaFUMKWFw5aoLby641X8Hfz5stOXuFq5ciI2nYUfRxCUKWjqsIyGchW0fg1aTQYzK2PvhqIo/6KsbRCHhRCrgZ+B7OszpZS//cM6XkB0iekYoEkpyz0lhGgNnAUmSSmjS1lGKSdKC4dV51cxc89MQlxDWNBhAQ4WDmw6lcjSrw/TPNucUOs1NKhxEfr+Da41jb0LiqKUUVkDwhJIBtqXmCeBfwqIslgDLJdS5gshRgLf3vYZAAghRgAjAHx9fR/wI5X7VVo4/BD5A3P2z6Fp1aZ80u4TrEytWLzzIlt+j6Bltis1LPbQsqM5ovMG1c6gKBVMWXtSl6nd4TaxgE+Jae/ieSW3m1xicjHw/l0+fxGwCCA8PFzeRy3KA7o9HFyGD+OLo1+w4MgCOvh24P3W72OCKW+sOkHG7m20zq5NFYvTdBwSjAjrbezyFUW5D2XtSf0N+jOGW0gph/zDageAACGEP/pg6AcMuG27VUs8na4nEFmWepRHq7Rw+CDiA7479R09a/TkreZvkVsgGf/DPhqdW061rE7YWKTSdUpHTH1qGbt8RVHuU1kvMa0t8bMl0AeI+6cVpJRFxX0m/gQ0wBIp5UkhxNtAhJRyNTBBCNETKAJSgMH3WL9iYLeHg9OwIcz6exa/nfuNAYEDmNp4KnFpeby6ZCNjUj7nQsYwMDehx/QuWFZxNHb5iqI8AFH8iId7W0nfaW6XlLL5wy/pn4WHh8uICPUY7EfhlnCYNAmHYS8xbec0NkZtZFTYKMaEjeFIdBpfLl3Km0Vfsj1lGpnCm75Tm+DqbWfs8hVFKUEIcfBeHxF9vyOhBQDu97muUgHcHg42Q19g/Lbx7I7dzZTwKQyqO4j1x2I59/ObfGqyirVZs0mTvnQfV0+Fg6JUEmVtg8jk1jaIBPTPiFAqodvDweKl/ozaNIoj147wVvO36FOzD0s2HqDmzklMMDnBBjmP+Bw/Or4UhE+gemaDolQWZb2LSX0lfEzcHg7ixacY+udQzqWd4/3W79POuxOff/cDfS++gatpJrtclnDxhCPN+tSgdhMPY5evKMpDVKaxmIQQfYQQDiWmHYUQ6t7FSub2cCga2JPBGwZzKf0S89vPp6lbO3769DVGXhyPlZUVJ0PXcOyEIyFtvKjfWfVPUZTKpqyD9b0ppUy/PiGlTEP/fAilkrg9HLL6debFP14kKTeJLzt9STX8Of5RdwZmfEWiZ3uS2q5n98Zcqtdzo+VztRBCGHsXFEV5yMraSF1akKhHfVUSt4dD8jNtGLFhEFJKvn7ia+Sla4i1z9GEZKIazcQsYAibPz2Kh78DnYYEYWKiwkFRKqOynkFECCHmCSFqFP+ZBxw0ZGHKo3F7OMT0aXzLcN2Ff60nYE1fzISOpKd/x67+KNZ/cRw7F0u6jQnF1Fxj7F1QFMVAyhoQ44ECYAXwI5AHjDVUUcqjcXs4nO0RzIhNI3CycOLb9p9T+H9TCTv2Dict62M5bjf23k1Y89kRNKYm9BgfhqWtmbF3QVEUAyrrXUzZwB0P/FEqrtvD4WiXGry6ZRz+Dv58GvwyZl/2pWZ+DOuqjKDT8PeQRbDyg0PkZxfR55UG2Luq4boVpbIr611Mm4QQjiWmnYQQfxquLMWQbg+H3R2qMPmvydRxqcN8l864fvcU5Geyut4XdB39PhphwoYvj5Man02XkcG4+aq7nhXlcVDWhmbX4juXAJBSpgohVE/qCuj2cPiztQ1zdv+HJlUa8W6qwGP/a/ytq0tm9y/o0zgUqZNs/S6SmNOpdBhUB98gF2PvgqIoj0hZA0InhPCVUl4BEEL4Ucrorkr5dms4vMwvTXUs2D+H9lUa8+bpYzimneMr8TQNh8ylmZ8rAHtXXeDs/kSa9KxOYLOqRt4DRVEepbIGxAxglxDiL0AArSh+gI9SMdweDt80yOC7I9/R0zmM/xxYT26RCTNsZjJm2Ch8nK0BOL49hkN/XqFuay8aPlnNyHugKMqjVtZG6g1CiHD0oXAY+B3INWRhysNTMhxcX57I/JA4fjv1GwMsfZl6cA2HdLX43msm7wzqgr2l/s6ki4evsWPFWfxCXWndT3WEU5THUVkH6xsGTET/VLgjQFPgb0p5PKhSvkgpSXj7HTLWrsX55QnMCTzHxnMbGVFoybhLu/iqqBtXGrzKB73CMNPo71mIP5/GxiUnqeJnT+dhdVVHOEV5TJW1H8REoBEQJaVsB9QH0v55FaU8SFrwOWkrVmA/dDBv1jzKxqiNTE7P5aXYWEYWTsbkiXd5p0+9G+GQmpDNus+PYetkQbexoZipjnCK8tgqaxtEnpQyTwiBEMJCSnlaCFHboJUpDyz1xx9J+uwzbHp2Z3rQcQ7HHmXWtWTqZnvQp+gNpgx4gifq3hyBNTs9nzWfHsVEI+gxvh5WtuZGrF5RFGMra0DEFPeD+B3YJIRIBaIMV5byoDI2biTh7Xewad2azzsXcDjmKO9fSyItqyXDzF5i4dAWhHjfGKCXgrwi1n52lNzsQvpMro+Dm+oIpyiPu7I2Uvcp/nGWEGIb4ABsMFhVygPJ3r+fuCmvYhUSwp4RDVl3cj5j0zLZkD6EM25d+HlwOFUdbgaAVqtjw6ITJMdm021MKO7V7I1YvaIo5cU9j8gqpfzLEIUoD0femTPEjBmLmbc3aW+NZO6+CbTOyeXQ1RcpqtWDn/vXx9bi5l+7lJJt358m+lQK7V4IpFqw6ginKIqeGrK7EimIiSV62HBMbGyw+3QuI/a8iEdhIeZx3RDBvfn8uXqYam69L2Hf6ouc2ZtA4x7+BLXwNFLliqKURyogKomilBSihw1Dl5+P93ffMHH/aNK1+bSPCyct4AUWlBIOJ3bEcvCPKIJaehLe1c84hSvlUmFhITExMeTl5Rm7FOUeWVpa4u3tjZnZg4+2bNCAEEJ0AT4BNMBiKeWcuyz3FPAL0EhKGWHImiojXXY20SNHURgfj++Sr1kU+Rb7ClPocNWXRO8JLBpQ/8ZtrNddPHKNHcvPUC3EhTb9VUc45VYxMTHY2dnh5+en/m1UIFJKkpOTiYmJwd/f/4G3V9Z+EPdMCKEBFgBPAkFAfyFEUCnL2aHvZ7HPULVUZrKwkJiJL5N38iReH81jb+pPfJ1xigbpdlx1msmXLzTEwvTWvgwJF9PZ9PVJ3HzteGJYMCYag/0zUCqovLw8XFxcVDhUMEIIXFxcHtqZnyGPDI2B81LKi1LKAvQPGupVynLvAHPRP4RIuQdSpyNuxgyyd+3C461ZpNieYUb0WnzyTMk2fZevBjXC0uzWcEhLzGHdgmNYO1rQbWwYZhaqI5xSOhUOFdPD/HszZEB4AdElpmOK590ghGgA+Egp1xmwjkrr6v8+IGP1GtxenohFsCWTjn6ClCaYFs5gyZBW2FjcegUxJ6OANfOPgIAe48Owtlcd4ZTySwjBwIEDb0wXFRXh5uZG9+7dAVi6dCnjxo27Yz0/Pz9CQkIIDQ2lc+fOJCQkPLKaKxujXVsQQpgA84BXyrDsCCFEhBAi4tq1a4YvrgJI/noJKd98g9Pzz+PcJZS3t03mvJkZjjnD+W5IzxuD7l13vSNcTkYB3ceG4ehubaTKFaVsbGxsOHHiBLm5+nFBN23ahJeX17+spbdt2zaOHTtGeHg4s2fPNmSZlZohAyIW8Ckx7V087zo7IBjYLoS4jH4AwNXFo8beQkq5SEoZLqUMd3NzM2DJFUP6qlVc/d//sOvShSpDevDjqhdYZ2OJc057fhg8EkfrW88MtFodf351kqToTJ4YFkwVf9URTqkYunbtyrp1+gsMy5cvp3///ve0fuvWrTl//rwhSnssGPIupgNAgBDCH30w9AMGXH9TSpkOuF6fFkJsB6aou5j+WdaOHcTN+A/WTZviOW00h5Z143/OVtjm1eTH59/DxdbiluWvPxHuyslk2j5fG79Q17tsWVFK99aak5yKy3io2wzytOfNHnX/dbl+/frx9ttv0717d44dO8aQIUPYuXNnmT9n7dq1hISEPEipjzWDBYSUskgIMQ74E/1trkuklCeFEG8DEVLK1Yb67Moq9+hRYia+jEVAAN7vvcG15b14zcEMoXXgmz4L8XC49bKRlJJdP5/j7D79E+Hqtirb6bmilBehoaFcvnyZ5cuX07Vr1zKv165dOzQaDaGhobz77rsGrLByM2g/CCnlemD9bfNm3mXZtoaspaLLv3iJ6JGjMHV1xffT/5H/S39etyrgmsaKj1vPJ9Dd4451ItZf5ti2GMI6+Kgnwin3rSzf9A2pZ8+eTJkyhe3bt5OcnFymdbZt24arqzpbflCqJ3UFUJiYyJVhQ0GjwXfhfIrWj+JLXSIHrOwYFzKV9tUb3LHOsW0x7F9zicCmHrR4qqa6ZVGpsIYMGYKjoyMhISFs377d2OU8VlQPqXJOm5FB9LDh6NLS8Vm4AN3u6exJP84SRzs6ePVgZIM7G+3O7k9gZ/HjQtu9EIhQT4RTKjBvb28mTJhQ6ntLly7F29v7xp+YmJhHXF3lJqSUxq7hnoSHh8uIiMejHVuXl8eVYcPIPXoM34ULMYldwrVzv9Pb0xdvxxr83OsHLDS3NkpfPp7EHwuP41HDgR4TwjA1Ux3hlHsXGRlJnTp1jF2Gcp9K+/sTQhyUUt5xl+g/UZeYyimp1RI7ZQq5EQfx/OADTFJWwulfeaFqbcwtTFnY+ZM7wiHufBobFp3AxduWbmNCVTgoivJAVECUQ1JKEt56m6zNW6gyfTrmlsew3PUV/VxDSbVI5/O2n+Nle+sdSUkxmaxbcAw7Z0u6jwvD3Er91SqK8mBUG0Q5lPTZAtJ++gmXESOwqpmD1a45zLCtxym7NEaHjaalV8tblk+7msPqT49ibqmh58R6aggNRVEeCvU1s5xJXb6cpAULcHiqL3ZtvbBYPYIlZiGsd8+gpWdLRoaNvGX57LR8Vn9yBKmV9JhUHztnSyNVrihKZaMCohzJ2PAnCW+/g23btrgMaInm54Fsoxbf+JvhYVGFOa3mYCJunvTlZRey+tMj5GUV0mtSfZyr2hixekVRKht1iamcyN63n7hXX8UqLAy3l/vBL4M5rfPks+Ca5GgzmNd2Hg4WDjeWL8zXsvazo6RdzeHJ0SFU8VPjKymK8nCpgCgH8k6fJmbsWMx8ffF4ayLan54nTuvAFw17cD77ODOaziDI5eazlrRFOv748jhXL2fwxNBgfAKdjVi9ohiGra3tjZ/Xr19PrVq1iIqKumWZpUuXYmJiwrFjx27MCw4O5vLly/+47WHDhnHq1Kl/XGbw4MH88ssvd8zfvn37jSHHKzsVEEZWEBPDleHDMbG1xfN/b5L30/NkaTX82HgSf6WtoW9AX/oG9L2xvE4n2fzNKaJPpdB2YCDV66vRbZXKbcuWLUyYMIE//viDatXuHDLG29ub//73v/e0zcWLFxMUdMcDLh8JrVZrlM+9HyogjKgoJYXoocOQBYV4f/QeWSsHIYpyWBX+Hmsyv6eOcx2mN5l+Y3kpJTuWn+H8was071uToBaeRqxeUQxvx44dDB8+nLVr11KjRo1Sl+nevTsnT57kzJkzd7y3ceNGmjVrRoMGDXjmmWfIysoCoG3btlzvcPv1119Tq1YtGjduzPDhw295CNGOHTto3rw51atXv+VsIiMjg27dulG7dm1GjRqFTqcD9EOSh4SEEBwczNSpU28sb2tryyuvvEJYWBh///0306ZNIygoiNDQUKZMmfLgvygDUY3URqLLziZ6xEgKExPxWfgJKX+Ox6kgiZVhn7KJXxEI5rWdd0tnuH2rLnJyZxwNnvClfmdfI1avPFb+mAYJxx/uNj1C4Mk5/7hIfn4+vXv3Zvv27QQGBt51ORMTE1577TVmz57Nt99+e2N+UlIS7777Lps3b8bGxoa5c+cyb948Zs68OV5oXFwc77zzDocOHcLOzo727dsTFhZ24/34+Hh27drF6dOn6dmzJ08//TQA+/fv59SpU1SrVo0uXbrw22+/0bx5c6ZOncrBgwdxcnKic+fO/P777/Tu3Zvs7GyaNGnChx9+SHJyMkOHDuX06dMIIUhLS7vf36LBqTMII5AFBcRMmEheZCSe77/Htd1v4J57gbWBc4h0P8GZlDO81+o9vO28b6xzZPMVDm6IIqilJ017l/5NSlEqEzMzM5o3b87XX3/9r8sOGDCAvXv3cunSpRvz9u7dy6lTp2jRogX16tXj22+/vaMNY//+/bRp0wZnZ2fMzMx45plnbnm/d+/emJiYEBQURGJi4o35jRs3pnr16mg0Gvr378+uXbs4cOAAbdu2xc3NDVNTU55//nl27NgBgEaj4amnngLAwcEBS0tLhg4dym+//Ya1dfl9uqM6g3jEpE5H3PQZZO/ejce7b5MY+Rk1Mo+ysvosRAM7Vu1dxaiwUbT2bn1jndN/x7P7l/PUaOBGmwG11cisyqP1L9/0DcXExISffvqJDh06MHv2bKZPn37XZU1NTXnllVeYO3fujXlSSjp16sTy5cvvuwYLi5tn8CXHrbv9/+C//Z+0tLREo9HcqHX//v1s2bKFX375hc8++4ytW7fed42GpM4gHiEpJVfnvk/G2rW4vfwy8Vd/JyB1J2u9J1Grawfe2/8eLTxbMCp01I11Lh65xtbvT+NTx4lOL9XFRI3MqjxGrK2tWbduHT/88MO/nkkMHjyYzZs3c/259U2bNmX37t03HjmanZ3N2bNnb1mnUaNG/PXXX6SmplJUVMSvv/5aprr279/PpUuX0Ol0rFixgpYtW9K4cWP++usvkpKS0Gq1LF++nDZt2tyxblZWFunp6XTt2pWPPvqIo0ePlukzjUGdQTxCKUuWkPLttzi9MJBY06OExaxlg9sQWjw/nv7r++Fm5cacVnPQmOi/acSeSWXj4pO4V7Ojy8gQNGYqz5XHj7OzMxs2bKB169a4ubnRs2fPUpczNzdnwoQJTJw4EQA3NzeWLl1K//79yc/PB+Ddd9+lVq1aN9bx8vJi+vTpNG7cGGdnZwIDA3FwcCh1+yU1atSIcePGcf78edq1a0efPn0wMTFhzpw5tGvXDikl3bp1o1evXnesm5mZSa9evcjLy0NKybx58+7n1/JIqOG+H5G0338nftrr2Hd9koT61jS8MJ/tjn1oPvYrxm8fx4GEA3z/5PfUddU/vetqVAa/f3QYWydL+r7SAEtbMyPvgfI4eZyG+87KysLW1paioiL69OnDkCFD6NOnj7HLeiAPa7hv9ZX0EcjavZv4Gf/BullTYhtXpeGF+eyz7UCLcYtZdOJL9sTtYXqT6TfCITUhmzXzj2JpbUbPCfVUOCiKAc2aNYt69eoRHByMv78/vXv3NnZJ5Ya6xGRg+efPEzvxZSxq1CCheyPCj03nqFUj6o1fxt8Je/jy2Jf0rtmbpwL0dzhkpuSx+pMjCAE9J9bD1sniXz5BUZQH8cEHHxi7hHJLnUEYUFFKCtGjRiMsLbk2sCf1jr3BeYsgak9YybWCJKbtnEYd5zrMaDIDIQS5WQWs+fQIBblF9BhfD8cq5ff2N0VRKj8VEAaiy88nZuw4iq5dI2/sSwSffJ0EM298xq4GczMmb58MwIdtP8TS1JKCvCLWzj9KRnIe3caG4eZrZ+Q9UBTlcWfQgBBCdBFCnBFCnBdCTCvl/VFCiONCiCNCiF1CCOMMjvKQSSmJn/Efcg8fxuaV8XidmUWGiQPOo9Zi7eDK7H2ziUyJZE6rOfjY+VBUqGX9wmNci86iy/BgPAMcjb0LiqIohgsIIYQGWAA8CQQB/UsJgGVSyhApZT3gfaD83u91D5I+/5yMtWtxGjkU08v/w0TqKOj3M3auPvx69ldWnl/JyNCRtPZujU6rY+Pik8SeSaPDoDr4hboau3xFURTAsGcQjYHzUsqLUsoC4EfglpuCpZQZJSZtgIp1z20p0tetI2n+Z9j36Ep+/k84apM53X4x1WrX42TySWbvm01zz+aMDhuNlJJtP5zh0tEkWj4bQO0mHsYuX1HKDY1Gc+Puoh49ejy0MYsuX75McHDwQ9lWSbNmzcLLy4t69epRr149pk2746LJQ3PkyBHWr19vsO1fZ8iA8AKiS0zHFM+7hRBirBDiAvoziAkGrMfgco8cIf716Vg1bAA+kXjnnuWPwNk0a9OFtLw0Jm+bjIuVC3NbzcVEmLDntwuc3hNPeDc/wtr7GLt8RSlXrKysOHLkCCdOnMDZ2ZkFCxYYu6R/NWnSJI4cOcKRI0eYM6fsQ5Tc6xDglSEgykRKuUBKWQOYCvyntGWEECOEEBFCiIjr3ejLm4KYWKLHjsPUowrWbU3xStnNty4v0+u5YWh1Wqbtmsa13GvMazsPR0tHDv0ZxZFNVwhp603j7v7GLl9RyrVmzZoRGxsL6Du2dejQgQYNGhASEsKqVasA/ZlBnTp1GD58OHXr1qVz587k5uYCcPDgQcLCwggLC7slaPLy8njppZcICQmhfv36bNu2DdA/iKh379506tQJPz8/PvvsM+bNm0f9+vVp2rQpKSkpZa59y5Yt1K9fn5CQEIYMGXKjV7efnx9Tp06lQYMG/Pzzz1y4cIEuXbrQsGFDWrVqxenTpwH4+eefCQ4OJiwsjNatW1NQUMDMmTNZsWIF9erVY8WKFQ/+C74LQ/aDiAVKfi32Lp53Nz8CC0t7Q0q5CFgE+p7UD6vAh0WblUXM6NHIwkJcnw/B8dJilpg+R99h0wEdM/fMZHfsbmY2m0mwazAnd8ay9/eLBDSqQqtnA9Tge0q5Nnf/XE6nnH6o2wx0DmRq46n/viD6b9dbtmxh6NChgH7gu5UrV2Jvb09SUhJNmza9MfzGuXPnWL58OV999RXPPvssv/76KwMHDuT/27vzuKrqvIHjny+IooKgoLnggigaCIKYWKJhmVppZmlIbpjjvuQSM2bbxDM1Nc7U85RNZps5+mBlVqSlmcuUaZQLhhLuKBrugaKiAr/nj3vgQcJ1uFzhft+vFy/POfd3zvn+Lni/9/zOvd/fiBEjmD17Nt26dSM+Pr742G+88QYiQmpqKunp6fTs2bO4XtO2bdvYsmULeXl5tGrVipdffpktW7YwdepU5s+fz5QpU34X66uvvsqCBQsAePnll7nzzjuJi4tj1apVBAYGMmzYMN58883ifX18fNi8eTMAd999N3PmzKF169YkJyczfvx4Vq9eTUJCAitWrKBJkyZkZ2dTvXp1EhIS2LhxI7Nnz77B38C1secVxE9AaxHxF5HqwCAgqWQDEWldYvV+YJcd47ELk5/PoanTOL9vH43H9cR73zt8VHgXnUfMwtPdlWe+f4akPUlMDJvIwMCB7N50lLX/u4Pm7Xy4O+5WRIvvKVWmc+fOERYWRsOGDTly5Aj33HMPYPuU4MyZMwkNDaVHjx4cOnSouBS3v78/YWFhAERERJCRkUF2djbZ2dl062arkDx06NDic6xbt44hQ4YA0LZtW5o3b16cILp3746npyf169fHy8uLvn37AhASEnLZKU1LDjH16tWLHTt24O/vX1z/afjw4cUlwAFiYmIA21XR+vXrGThwIGFhYYwZM4asrCwAunTpQlxcHG+//XaFz0ZntysIY0y+iEwEVgCuwHvGmCqlb1EAABaGSURBVO0ikgBsNMYkARNFpAdwEfgNGG6veOzlyF9f4sx339Fw3ABq75/NNwXhVH/wVdo08uDp759m6d6lTAqfxOjQ0RxIO8HK97bTKMCLXqPb4erq8BE+pa7qWt/pl7eiexBnz56lV69evPHGG0yePJmFCxdy7NgxNm3ahJubGy1atCAvLw+4tDy3q6tr8RDTjSh5LBcXl+J1FxcX8vPzb/i4JdWuXRuAwsJCvL29SUlJ+V2bOXPmkJyczLJly4iIiGDTpk3lcu5rYddXKGPMl8aYQGNMgDHmBWvbs1ZywBjzuDEm2BgTZozpbozZbs94ytvJBQv5beFC6j3cizo5b7G1sCXJHf5O3/CmPPX9Uyzdu5TJ4ZMZHTqaw3tz+GpOKnUb1ub+8aG4VXd1dPhKVQq1atXitdde4x//+Af5+fnk5OTQoEED3NzcWLNmze8mASrN29sbb29v1q1bB8DChQuLH+vatWvx+s6dOzlw4ABt2rQpt9jbtGlDRkZGccnxf/3rX2WWAK9Tpw7+/v58/PHHgO0qqagM+J49e4iMjCQhIYH69euTmZmJp6cnp0+fLrc4L0ffwt6g3G+/5ciLL+IR1Yl6tZaQWVCX2Q3/wrQ+ocxcN5Nle5fxeIfHGRU6imOZp1k6eyu1vGrQd3J7atTS4ntKXY/w8HBCQ0NJTExk8ODBbNy4kZCQEObPn3/F6UiLvP/++0yYMIGwsLBLJv4ZP348hYWFhISEEBMTw7x58y65cvhPubu78/777zNw4EBCQkJwcXFh7NixZbYtmvOiffv2BAcHF998j4+PL57n+o477qB9+/Z0796dtLQ0u9+k1nLfNyBv5072xz6KW5NGNOuyh5xzZxjp9iJvTuzHqykJfJXxFVM6TGFkyEiOZJzii9dScKvhSv/pHajjW9OhsSt1LZyp3HdVVF7lvrWa63XKP36cg2PH4VKrJn5Rx7h47iSPXXyGp+N68krK8yzPWM60iGmMaDeCrN3ZfDF7KzU93Og3JVyTg1KqUtEhputQmJdH5oQJ5J88id/97rie28mo81MY0PdePtz/MsszljM9Yjoj2o3g4I7fSHp9K7W9atB/eoQmB6VUpaMJ4hoZY8iaOZO8rT/TuH8zaub9yPTzY2gYfg8peW+wImMFT3R8grh2cRzYfoKls7dSx8edB6eF65wOSqlKSRPENTr++mxOffkV9fsGU8es5R9mMHsa9ybfZwFf7/+a+I7xDA8ezr6tx1j25s/UbViLB6eFU9tLk4NSqnLSBHENcr74guP//CdeUbfiU2slS9zuZ4FrX/wCl7Aq8xviO8YzLHgYuzcdZflb2/D186TflHBqelR3dOhKKXXD9Cb1VZzdvJmsmU9RK6gFjRqtYrNHN/54ciC3d/mCdVnf8afb/sSQoCHsSD7MqnlpNAzwos+E9lSvqU+tUqpy0yuIK7iQmcnBCRNxq1+XJrdu5Nd6HXj0xDCCOywj5cR3zOg0gyFBQ0j7/le+mZdG48C69J0UpslBqXLy2WefISLFhetKi4uLY/HixVc8RlxcXHEJjrZt2/L888+Xe4xpaWnlesybhSaIyyg4dYrMseMw+Rfw67SX877Nue/oKBoFLWXv2WSe7PQkg28dTOrag6z5VzrNgurRZ0IobjX0G9JKlZfExESioqJITEz8j44za9as4hpJH3zwAfv27SunCDVBOB1z8SKHpkzlQkYGfl1P4+rrwYOnplC95VJOmM3MjJzJo7c+ypaVB/h20U782/ty39hQqmn5DKXKTW5uLuvWrePdd99l0aJFgO3ThBMnTqRNmzb06NGDo0ePFrdPSEjgtttuo127dowePZqyvgRcVLOpqAbS5UpxX277jBkzCAoKIjQ0lCeeeIL169eTlJREfHw8YWFh7Nmzx67PSUXTsZBSjDEcfuEFzqxfT6O7alCrwSnGuCRwuP4XUD2NpyKfYlDbQWz8MoPkpL0EdGjAPSODtPCeqrIOv/gi538p33LfNW5tS8OZM6/Y5vPPP6d3794EBgbi4+PDpk2b2L9/Pzt27CAtLY0jR44QFBTEY489BsDEiRN59tlnAVvF1qVLlxZXYI2Pj+cvf/kLu3fvZvLkyTRo0IC8vLwyS3GPHTu2zO1Dhw7l008/JT09HREhOzsbb29vHnjgAfr06cOAAQPK9Tm6GeirWim/zZ9P9qIP8enkiVejX3ntluf4rvoyqJ3G05FPE9MmhuSkvSQn7aVNZEN6anJQyi4SExMZNGgQAIMGDSIxMZFvv/2W2NhYXF1dady4MXfddVdx+zVr1hAZGUlISAirV69m+/b/r/1ZNMR0+PBhVq1axfr16y9bivty2728vHB3d2fkyJEsWbKEWrVqVeCz4Rh6BVHC6TVrOPLSy3i29aK+fzorg19izq/LqeaZzjOdn2Fg4EDWf7KblG8yCerSiOjBbXU+B1XlXe2dvj2cPHmS1atXk5qaiohQUFCAiNC/f/8y2+fl5TF+/Hg2btxI06ZN+fOf/1w8nFSSh4cH0dHRrFu3jl69el1XTNWqVePHH39k1apVLF68mNmzZ7N69eob6l9loW99LXnp6RyaPh33xp40bpfO7sinmJK1ypYcIp9lYOuBfLdoJynfZBIS7afJQSk7Wrx4MUOHDmX//v1kZGSQmZmJv78/Pj4+fPjhhxQUFJCVlVU8RWhRMvD19SU3N/eyn2zKz88nOTmZgICAy5bivtz23NxccnJyuO+++3j11VeLy3FXVOltR9AEAVw8epTMceNxrS74ddxJduexDMj8EVePdP4Y8TQDAgewdmE6qf8+RNg9zega01qTg1J2lJiY+LurhYcffpisrCxat25NUFAQw4YN4/bbbwdscz6MGjWKdu3a0atXL2677bZL9i26iRwaGkpISAgPPfTQZUtxX2776dOn6dOnD6GhoURFRfHKK68AtuGvWbNmER4eXuVuUjt9ue/Cc+fYP2w453em0yL6EIVd+9M95zRnq6UxNngG48JjWTX/F3YmH6HjfS3o1Ndf55BWVZ6W+67ctNx3OTCFhfw640nytm3DL+o3iIiib95Zzrj+woBm0xgbHsvX76axZ/NRIvu1pOO9LRwdslJKVRinThDH/uc1Tq9YQYMO53Dr2IpBNT04ciaVznXG8kzUcJa/tY2Mn4/TZUArwno0c3S4SilVoZw2QWR/+hkn3noL78ACat7mwWO3NGN3dipNC4Yz+95RfDnnZw5sP0m3QYGERPs5OlyllKpwTnmT+uxPP5H1zDPUaiJ43ZHP+IBQUrNTqZkdy/sPjmP5nFQOpJ2k+9C2mhyUUk7L6a4gLuzfz8GJk6juafCJymbSrV3YdHIbF488wgcDRrP+vV84vO8UPeKCaBPZ0NHhKqWUw9j1CkJEeovIDhHZLSIzynh8moikicjPIrJKRJrbM56CnBwyx4yBC7n4Rh3l8bBO/JS9g3O/PsKfo4aw5+N9HNl3ip4jgzU5KKWcnt0ShIi4Am8A9wJBQKyIBJVqtgXoaIwJBRYDf7NXPObiRQ4+/jgXMg/ge8dRpkZ25KdT+zh76BFiA/oia49y/OBpeo8NoVVEA3uFoZS6Di+88ALBwcGEhoYSFhbG888/z5NPPnlJm5SUlOKPdObm5jJmzBgCAgKIiIggOjqa5ORkR4ReJdhziKkTsNsYsxdARBYB/YDiurjGmDUl2v8ADLFHIMYYDickcPaHZHw6ZzO9awibzxyi4MggImp35db0c/x2PI/7x4XSLNjHHiEopa7Thg0bWLp0KZs3b6ZGjRocP36ctLQ04uLi+Otf/1rcbtGiRcTGxgLwhz/8AX9/f3bt2oWLiwv79u2rsqW4K4I9E0QTILPE+kEg8grtRwJflfWAiIwGRgM0a3b9Hzc9vXIl2R8vpk5QLn+8pxVbLhyn9qmhVDsbxn2nXDh9+jx9J7anSZu6131spZR9ZGVl4evrS40atnndfX196datG3Xr1iU5OZnISNvLyUcffcSKFSvYs2cPycnJLFy4EBcX2+CIv78//v7+DutDZXdT3KQWkSFAR+DOsh43xswF5oLtm9TXe3zP5uDTOZv47k1JKThFSzOK/QcDGI87F85f5IHJYTQK8PqP+qBUVfXdRzs5nplbrsf0bepB10cCr9imZ8+eJCQkEBgYSI8ePYiJieHOO+8kNjaWRYsWERkZyQ8//EC9evVo3bo1SUlJhIWF4eqq87KUF3vepD4ENC2x7mdtu4SI9ACeAh4wxpy3RyBnm4QS3zOQrXKB6LpT2LetJX+4WAsuFvLAlHBNDkrdhDw8PNi0aRNz586lfv36xMTEMG/ePGJiYli8eDGFhYWXDC+p8mfPK4ifgNYi4o8tMQwCHi3ZQETCgbeA3saYo78/RPnYkrObbQW5DAt4ioWf12bE+ZrUqO5Cvylh+Pp52uu0SlUJV3unb0+urq5ER0cTHR1NSEgIH3zwQfEc0//+97/55JNP2LBhAwDBwcFs3bqVgoICvYooJ3a7gjDG5AMTgRXAL8BHxpjtIpIgIg9YzWYBHsDHIpIiIkn2iCWqSRRzuy8m6cs6DD7jjqe7G/2nddDkoNRNbMeOHezatat4PSUlhebNbZ+Ej42NZerUqbRs2RI/P9uXWQMCAujYsSPPPfdc8XSjGRkZLFu2rOKDryLseg/CGPMl8GWpbc+WWO5hz/MXOXshn78tOEDfk9XwqlOdh6Z1wPuWqj8blFKVWW5uLpMmTSI7O5tq1arRqlUr5s6dC8DAgQOZPHkyr7/++iX7vPPOO0yfPp1WrVpRs2ZNfH19mTVrliPCrxJuipvU9rZ2w0Ei9+fj4VmdgfER1PGt6eiQlFJXERERwfr168t8zNfXl4sXL/5ue506dXj77bftHZrTcIoE0TWkIWc2nqD3iGA867k7OhyllKoUnCJBeNZzZ+D0CEeHoZRSlYpTVnNVSil1dZoglFJlqmzTESub8vy9aYJQSv2Ou7s7J06c0CRRyRhjOHHiBO7u5XOv1SnuQSilro+fnx8HDx7k2LFjjg5FXSd3d/fi74b8pzRBKKV+x83NTYvcKR1iUkopVTZNEEoppcqkCUIppVSZpLJ9SkFEjgH7b2BXX+B4OYdT2Tj7c6D91/47c//bGGOuq0JppbtJbYypfyP7ichGY0zH8o6nMnH250D7r/139v5f7z46xKSUUqpMmiCUUkqVyZkSxFxHB3ATcPbnQPvv3LT/16nS3aRWSilVMZzpCkIppdR1cIoEISK9RWSHiOwWkRmOjqciiUhTEVkjImkisl1EHnd0TI4gIq4iskVEljo6loomIt4islhE0kXkFxG53dExVSQRmWr97W8TkUQRqfKzhonIeyJyVES2ldhWT0RWisgu69+6VztOlU8QIuIKvAHcCwQBsSIS5NioKlQ+MN0YEwR0BiY4Wf+LPA784uggHOR/gOXGmLZAe5zoeRCRJsBkoKMxph3gCgxybFQVYh7Qu9S2GcAqY0xrYJW1fkVVPkEAnYDdxpi9xpgLwCKgn4NjqjDGmCxjzGZr+TS2F4cmjo2qYomIH3A/8I6jY6loIuIFdAPeBTDGXDDGZDs2qgpXDagpItWAWsCvDo7H7owx3wInS23uB3xgLX8APHi14zhDgmgCZJZYP4iTvUAWEZEWQDiQ7NhIKtx/A38ECh0diAP4A8eA960htndEpLajg6ooxphDwN+BA0AWkGOM+dqxUTnMLcaYLGv5MHDL1XZwhgShABHxAD4BphhjTjk6nooiIn2Ao8aYTY6OxUGqAR2AN40x4cAZrmFooaqwxtn7YUuUjYHaIjLEsVE5nrF9fPWqH2F1hgRxCGhaYt3P2uY0RMQNW3JYaIxZ4uh4KlgX4AERycA2vHiXiCxwbEgV6iBw0BhTdNW4GFvCcBY9gH3GmGPGmIvAEuAOB8fkKEdEpBGA9e/Rq+3gDAniJ6C1iPiLSHVsN6iSHBxThRERwTb+/Isx5hVHx1PRjDFPGmP8jDEtsP3uVxtjnOYdpDHmMJApIm2sTXcDaQ4MqaIdADqLSC3r/8LdONFN+lKSgOHW8nDg86vtUOmK9V0vY0y+iEwEVmD7BMN7xpjtDg6rInUBhgKpIpJibZtpjPnSgTGpijUJWGi9QdoLjHBwPBXGGJMsIouBzdg+0bcFJ/hGtYgkAtGAr4gcBJ4DXgI+EpGR2CpiP3LV4+g3qZVSSpXFGYaYlFJK3QBNEEoppcqkCUIppVSZNEEopZQqkyYIpZRSZdIEoaoMEVkrInafc1hEJltVURfa+TzRRdVnReSBK1Uitiq2ji+x3tj6eKdSN0wThFKAVcjtWo0H7jHGDL6B84iIXPf/O2NMkjHmpSs08bbiKmr/qzFmwPWeR6mSNEGoCiUiLax3329bNfq/FpGa1mPFVwAi4muVx0BE4kTkM6uGfYaITBSRaVbxuR9EpF6JUwwVkRSr9n8na//aVn38H619+pU4bpKIrMZW/rh0rNOs42wTkSnWtjlAS+ArEZlaqn2ciHxu9WOXiDxXos87RGQ+sA1oKiI9RWSDiGwWkY+tWllFc5eki8hm4KFSx55tLd8iIp+KyFbr5w5sX4IKsPo+yzrnNqu9u4i8LyKpVv+7lzjmEhFZbsX7N2u7q4jMs/qdWrqfyokYY/RHfyrsB2iB7RutYdb6R8AQa3kttrr9AL5AhrUcB+wGPIH6QA4w1nrsVWwFCIv2f9ta7gZss5ZfLHEOb2AnUNs67kGgXhlxRgCpVjsPYDsQbj2WAfiWsU8ctoqhPkBNbMmgo9XnQqBzib59C9S21v8EPAu4Y6s83BoQ67lZWuLYs63lD0v02RXwss6xrdTzXNT/6dgqCAC0xVZ+wt065l5rf3ds365tavV9ZYljeTv670Z/HPOjVxDKEfYZY4rKfmzC9mJ2NWuMMaeNMcewJYgvrO2ppfZPhOJ6+HVExBvoCcywSo2sxfZi2Mxqv9IYU7puPkAU8Kkx5owxJhdbkbeu1xDnSmPMCWPMOWufKGv7fmPMD9ZyZ2yTV31vxTQcaI7txXufMWaXMcYAlysqeBfwptXPAmNMzlViiio6ljEmHVsiCLQeW2WMyTHG5GGr0dQcW9JoKSKvi0hvwGmq/6pLVflaTOqmdL7EcgG2d9tgu7IoetNSelrIkvsUllgv5NK/49K1Ywy2d+MPG2N2lHxARCKxlb8uT2Wdn1LnEWyJJLZUPGHlHMu1KP27qGaM+U1E2gO9gLHYavY85oDYlIPpFYS6mWRgG94AuNEbrDEAIhKFbXKYHGyFGidZ1TwRkfBrOM53wINWFdDaQH9r29XcI7a5f2tim7Hr+zLa/AB0EZFWVjy1RSQQSAdaiEiA1S62jH3Bdr9knLWvq9hmjTuNbQjucn0ZbLUPxHb1tOMybRERX8DFGPMJ8DTOVR5claAJQt1M/g6ME5Et2Mbpb0Setf8cYKS17b8AN+BnEdlurV+RsU3TOg/4EdsMfO8YY7Zcw/l/xDb3xs/AJ8aYjWUc+xi28f9EEfkZ2AC0tYZ5RgPLrJvUl6vX/zjQXURSsQ3RBRljTmAbstomIrNKtf8n4GK1/xCIM8ac5/KaAGut4a8FwJPX0G9VBWk1V6XKiYjEYbvJPtHRsShVHvQKQimlVJn0CkIppVSZ9ApCKaVUmTRBKKWUKpMmCKWUUmXSBKGUUqpMmiCUUkqVSROEUkqpMv0ffi1F0LW78ZsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Classifier</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>Duration</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MLP</td>\n",
       "      <td>0.180548</td>\n",
       "      <td>0.312273</td>\n",
       "      <td>0.420257</td>\n",
       "      <td>0.509094</td>\n",
       "      <td>0.605208</td>\n",
       "      <td>0.691557</td>\n",
       "      <td>0.730998</td>\n",
       "      <td>0.775416</td>\n",
       "      <td>0.802604</td>\n",
       "      <td>0.815240</td>\n",
       "      <td>0:00:35.207844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>K Neighbors</td>\n",
       "      <td>0.195290</td>\n",
       "      <td>0.311890</td>\n",
       "      <td>0.424852</td>\n",
       "      <td>0.503925</td>\n",
       "      <td>0.567298</td>\n",
       "      <td>0.616887</td>\n",
       "      <td>0.650201</td>\n",
       "      <td>0.669922</td>\n",
       "      <td>0.678154</td>\n",
       "      <td>0.685238</td>\n",
       "      <td>0:00:02.227997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Random Forest</td>\n",
       "      <td>0.190312</td>\n",
       "      <td>0.293892</td>\n",
       "      <td>0.427915</td>\n",
       "      <td>0.525560</td>\n",
       "      <td>0.611143</td>\n",
       "      <td>0.704384</td>\n",
       "      <td>0.749952</td>\n",
       "      <td>0.784607</td>\n",
       "      <td>0.798583</td>\n",
       "      <td>0.804327</td>\n",
       "      <td>0:00:39.264837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AdaBoost</td>\n",
       "      <td>0.199119</td>\n",
       "      <td>0.322037</td>\n",
       "      <td>0.437105</td>\n",
       "      <td>0.541260</td>\n",
       "      <td>0.612100</td>\n",
       "      <td>0.688685</td>\n",
       "      <td>0.743442</td>\n",
       "      <td>0.781543</td>\n",
       "      <td>0.800689</td>\n",
       "      <td>0.808922</td>\n",
       "      <td>0:01:17.278591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>SVC</td>\n",
       "      <td>0.178442</td>\n",
       "      <td>0.279150</td>\n",
       "      <td>0.391729</td>\n",
       "      <td>0.509094</td>\n",
       "      <td>0.597549</td>\n",
       "      <td>0.683324</td>\n",
       "      <td>0.750144</td>\n",
       "      <td>0.795137</td>\n",
       "      <td>0.820218</td>\n",
       "      <td>0.820984</td>\n",
       "      <td>0:04:20.838741</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Classifier         1         2         3         4         5         6  \\\n",
       "1            MLP  0.180548  0.312273  0.420257  0.509094  0.605208  0.691557   \n",
       "2    K Neighbors  0.195290  0.311890  0.424852  0.503925  0.567298  0.616887   \n",
       "3  Random Forest  0.190312  0.293892  0.427915  0.525560  0.611143  0.704384   \n",
       "4       AdaBoost  0.199119  0.322037  0.437105  0.541260  0.612100  0.688685   \n",
       "5            SVC  0.178442  0.279150  0.391729  0.509094  0.597549  0.683324   \n",
       "\n",
       "          7         8         9        10        Duration  \n",
       "1  0.730998  0.775416  0.802604  0.815240  0:00:35.207844  \n",
       "2  0.650201  0.669922  0.678154  0.685238  0:00:02.227997  \n",
       "3  0.749952  0.784607  0.798583  0.804327  0:00:39.264837  \n",
       "4  0.743442  0.781543  0.800689  0.808922  0:01:17.278591  \n",
       "5  0.750144  0.795137  0.820218  0.820984  0:04:20.838741  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "\n",
    "RANDOM_SEED = 42\n",
    "\n",
    "classifiers = {\n",
    "    'Random Forest': RandomForestClassifier(n_estimators=len(paths), criterion='gini', random_state=RANDOM_SEED),\n",
    "    'SVC': SVC(probability=True, kernel='rbf', random_state=RANDOM_SEED),\n",
    "    'AdaBoost': AdaBoostClassifier(RandomForestClassifier(), random_state=RANDOM_SEED),\n",
    "    'MLP': MLPClassifier(alpha=0.0006, learning_rate_init=0.009, random_state=RANDOM_SEED),\n",
    "    'K Neighbors': KNeighborsClassifier(n_neighbors=PREDICTIONS_NUM * 2, algorithm='brute')\n",
    "}\n",
    "\n",
    "result_columns = ['Classifier'] + [i for i in range(1, PREDICTIONS_NUM + 1)] + ['Duration']\n",
    "result = []\n",
    "\n",
    "for name, classifier in classifiers.items():\n",
    "    start_time = datetime.now()\n",
    "    classifier.fit(X_train, y_train)\n",
    "    predictions = get_predictions(classifier, X_test)\n",
    "    end_time = datetime.now()\n",
    "    \n",
    "    accuracy, accuracy_cumulative = get_accuracy(y_test, predictions)\n",
    "    result.append([name] + accuracy_cumulative + [str(end_time - start_time)] )\n",
    "\n",
    "plot_accuracy(result, name)\n",
    "result = pd.DataFrame(result, columns=result_columns)\n",
    "result.index += 1\n",
    "display(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusion\n",
    "Most of the displayed models have similar accuracy and they flatten as we approach higher numbers of predictions. AdaBoost has the best start but others are close, even KNN, ~20% for one and ~42% for three predictions which is ok because we can't expect more than a few predictions in a lot of real-life scenarios. KNN also delivers fast, brute-force search does well on small and simple datasets, it could be used as a fallback model. AdaBoost with Random Forest as a weak classifier would be interesting to try to develop optimally from the scratch.\n",
    "\n",
    "#### What's next? \n",
    "Get more data! Testing these approaches on different datasets would give us a better insight in how they would behave in the production and establish some correlations between model hyperparameters and nature of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:tensorflow3.5]",
   "language": "python",
   "name": "conda-env-tensorflow3.5-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}