Time_Series_Forecasting_With_CNNs.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Time Series Analysis and Forecasting with CNNs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convolutional Neural Network models, or CNNs for short, can be applied to time series forecasting. I will be presenting different type of scenarios that we usually come across during solving problems related with time series and the variety of CNN architectures that we can use to tackle them.\n",
    "\n",
    "Variety of time series related problems:\n",
    "- CNN Models for Univariate time series\n",
    "- CNN Models for Multivariate time series\n",
    "- CNN Models for Multistep time series\n",
    "- CNN Models for Multivariate and Multisteps time series\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- Imports\n",
    "- Data Description\n",
    "- Load and Prepare the Dataset\n",
    "- Problem Framing\n",
    "- Solution Approach\n",
    "- Evaluation Metric \n",
    "- Train and Test sets\n",
    "- Modeling and Evaluation\n",
    "    - Univariate CNN Model\n",
    "    - Multi-channel CNN Model\n",
    "    - Multi-headed CNN  Model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F \n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Description\n",
    "\n",
    "The Household Power Consumption dataset is a multivariate time series dataset that describes the electricity consumption for a single household over four years. The data was collected between December 2006 and November 2010 and observations of power consumption within the household were collected every minute. \n",
    "\n",
    "It is a multivariate series comprised of seven variables\n",
    "(besides the date and time); they are:\n",
    "\n",
    "- `global active power`: The total active power consumed by the household (kilowatts).\n",
    "- `global reactive power`: The total reactive power consumed by the household (kilowatts).  voltage: Average voltage (volts).\n",
    "- `global intensity`: Average current intensity (amps).\n",
    "- `sub metering 1`: Active energy for kitchen (watt-hours of active energy).\n",
    "- `sub metering 2`: Active energy for laundry (watt-hours of active energy).\n",
    "- `sub metering 3`: Active energy for climate control systems (watt-hours of active energy).\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem Framing\n",
    "\n",
    "> `Given some number of prior days of total daily power consumption, predict the next standard week of daily power consumption?`\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solution Approach\n",
    "\n",
    "Since we have to do some kind of prediction for whole week or for each day for next 7 days, this framing of the problem is referred to as a multi-step time series forecasting problem, given the multiple forecast steps.\n",
    "\n",
    "A model that makes use of multiple input variables may be referred to as a multivariate multi-step time series forecasting model.\n",
    "\n",
    "This framing of the dataset also suggests that it would be useful to downsample the per-minute observations of power consumption to daily totals.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation Metric Selection\n",
    "\n",
    "A forecast will be comprised of seven values, one for each day of the week ahead. It is common with multi-step forecasting problems to evaluate each forecasted time step separately. \n",
    "\n",
    "This is helpful for a few reasons:\n",
    "\n",
    "- To comment on the skill at a specific lead time (e.g. +1 day vs +3 days).\n",
    "- To contrast models based on their skills at different lead times (e.g. models good at +1 day vs models good at days +5).\n",
    "\n",
    "The units of the total power are kilowatts and it would be useful to have an error metric that was also in the same units. Both Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) fit this bill, although RMSE is more commonly used.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and Test Sets\n",
    "\n",
    "We will use the first 3 years of data for training predictive models and the final year for evaluating models. \n",
    "\n",
    "The data in a given dataset will be divided into standard weeks. These are weeks that begin on a Sunday and end on a Saturday. This is a realistic and useful way for using the chosen framing of the model, where the power consumption for the week ahead can be predicted. \n",
    "\n",
    "It is also helpful with modeling, where models can be used to predict a specific day (e.g. Wednesday) or the entire sequence.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and Prepare the Dataset"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Data Loading**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 459,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2024-01-28 14:12:17--  https://archive.ics.uci.edu/static/public/235/individual+household+electric+power+consumption.zip\n",
      "Resolving archive.ics.uci.edu (archive.ics.uci.edu)... 128.195.10.252\n",
      "Connecting to archive.ics.uci.edu (archive.ics.uci.edu)|128.195.10.252|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: unspecified\n",
      "Saving to: 'individual+household+electric+power+consumption.zip'\n",
      "\n",
      "tion.zip                [               <=>  ]   4.23M   284KB/s               ^C\n"
     ]
    }
   ],
   "source": [
    "!wget https://archive.ics.uci.edu/static/public/235/individual+household+electric+power+consumption.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('data/household_power_consumption.txt', sep=';', header=0, low_memory=False, infer_datetime_format=True, parse_dates={'datetime':[0,1]}, index_col=['datetime'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Global_active_power</th>\n",
       "      <th>Global_reactive_power</th>\n",
       "      <th>Voltage</th>\n",
       "      <th>Global_intensity</th>\n",
       "      <th>Sub_metering_1</th>\n",
       "      <th>Sub_metering_2</th>\n",
       "      <th>Sub_metering_3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datetime</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2006-12-16 17:24:00</th>\n",
       "      <td>4.216</td>\n",
       "      <td>0.418</td>\n",
       "      <td>234.840</td>\n",
       "      <td>18.400</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006-12-16 17:25:00</th>\n",
       "      <td>5.360</td>\n",
       "      <td>0.436</td>\n",
       "      <td>233.630</td>\n",
       "      <td>23.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006-12-16 17:26:00</th>\n",
       "      <td>5.374</td>\n",
       "      <td>0.498</td>\n",
       "      <td>233.290</td>\n",
       "      <td>23.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>2.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006-12-16 17:27:00</th>\n",
       "      <td>5.388</td>\n",
       "      <td>0.502</td>\n",
       "      <td>233.740</td>\n",
       "      <td>23.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006-12-16 17:28:00</th>\n",
       "      <td>3.666</td>\n",
       "      <td>0.528</td>\n",
       "      <td>235.680</td>\n",
       "      <td>15.800</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    Global_active_power Global_reactive_power  Voltage  \\\n",
       "datetime                                                                 \n",
       "2006-12-16 17:24:00               4.216                 0.418  234.840   \n",
       "2006-12-16 17:25:00               5.360                 0.436  233.630   \n",
       "2006-12-16 17:26:00               5.374                 0.498  233.290   \n",
       "2006-12-16 17:27:00               5.388                 0.502  233.740   \n",
       "2006-12-16 17:28:00               3.666                 0.528  235.680   \n",
       "\n",
       "                    Global_intensity Sub_metering_1 Sub_metering_2  \\\n",
       "datetime                                                             \n",
       "2006-12-16 17:24:00           18.400          0.000          1.000   \n",
       "2006-12-16 17:25:00           23.000          0.000          1.000   \n",
       "2006-12-16 17:26:00           23.000          0.000          2.000   \n",
       "2006-12-16 17:27:00           23.000          0.000          1.000   \n",
       "2006-12-16 17:28:00           15.800          0.000          1.000   \n",
       "\n",
       "                     Sub_metering_3  \n",
       "datetime                             \n",
       "2006-12-16 17:24:00            17.0  \n",
       "2006-12-16 17:25:00            16.0  \n",
       "2006-12-16 17:26:00            17.0  \n",
       "2006-12-16 17:27:00            17.0  \n",
       "2006-12-16 17:28:00            17.0  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2075259, 7)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Global_active_power', 'Global_reactive_power', 'Voltage',\n",
       "       'Global_intensity', 'Sub_metering_1', 'Sub_metering_2',\n",
       "       'Sub_metering_3'],\n",
       "      dtype='object')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Missing Values Imputation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Global_active_power          0\n",
       "Global_reactive_power        0\n",
       "Voltage                      0\n",
       "Global_intensity             0\n",
       "Sub_metering_1               0\n",
       "Sub_metering_2               0\n",
       "Sub_metering_3           25979\n",
       "dtype: int64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a lot of missing values in `Sub_metering_3` column. Before proceeding further we need to impute them with some reasonable values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Global_active_power      None\n",
       "Global_reactive_power    None\n",
       "Voltage                  None\n",
       "Global_intensity         None\n",
       "Sub_metering_1           None\n",
       "Sub_metering_2           None\n",
       "Sub_metering_3           None\n",
       "dtype: object"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.replace('?', None, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([17., 16., 18.,  9.,  0.,  5., 19., 10., 20.,  8.,  4.,  2.,  3.,\n",
       "       11., 13.,  6., 15., nan, 12.,  1., 14.,  7., 23., 25., 26., 28.,\n",
       "       27., 21., 22., 29., 24., 30., 31.])"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['Sub_metering_3'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Sub_metering_3'] = df[['Sub_metering_3']].fillna(method=\"ffill\")\n",
    "df['Sub_metering_3'] = df[['Sub_metering_3']].fillna(method=\"bfill\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Global_active_power      0\n",
       "Global_reactive_power    0\n",
       "Voltage                  0\n",
       "Global_intensity         0\n",
       "Sub_metering_1           0\n",
       "Sub_metering_2           0\n",
       "Sub_metering_3           0\n",
       "dtype: int64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok so now we don't have any null values. Let's check for the data type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Global_active_power       object\n",
       "Global_reactive_power     object\n",
       "Voltage                   object\n",
       "Global_intensity          object\n",
       "Sub_metering_1            object\n",
       "Sub_metering_2            object\n",
       "Sub_metering_3           float64\n",
       "dtype: object"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Datatype Conversion**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most of the ML models expect data to be of `float32` type. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.astype('float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Global_active_power      float32\n",
       "Global_reactive_power    float32\n",
       "Voltage                  float32\n",
       "Global_intensity         float32\n",
       "Sub_metering_1           float32\n",
       "Sub_metering_2           float32\n",
       "Sub_metering_3           float32\n",
       "dtype: object"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now done with data preparation part. We can move to do modeling."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Downsampling data to day level**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Global_active_power</th>\n",
       "      <th>Global_reactive_power</th>\n",
       "      <th>Voltage</th>\n",
       "      <th>Global_intensity</th>\n",
       "      <th>Sub_metering_1</th>\n",
       "      <th>Sub_metering_2</th>\n",
       "      <th>Sub_metering_3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datetime</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2006-12-16</th>\n",
       "      <td>1209.176025</td>\n",
       "      <td>34.922001</td>\n",
       "      <td>93552.53125</td>\n",
       "      <td>5180.799805</td>\n",
       "      <td>0.0</td>\n",
       "      <td>546.0</td>\n",
       "      <td>4926.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006-12-17</th>\n",
       "      <td>3390.459961</td>\n",
       "      <td>226.005997</td>\n",
       "      <td>345725.31250</td>\n",
       "      <td>14398.599609</td>\n",
       "      <td>2033.0</td>\n",
       "      <td>4187.0</td>\n",
       "      <td>13341.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006-12-18</th>\n",
       "      <td>2203.825928</td>\n",
       "      <td>161.792007</td>\n",
       "      <td>347373.62500</td>\n",
       "      <td>9247.200195</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>2621.0</td>\n",
       "      <td>14018.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Global_active_power  Global_reactive_power       Voltage  \\\n",
       "datetime                                                               \n",
       "2006-12-16          1209.176025              34.922001   93552.53125   \n",
       "2006-12-17          3390.459961             226.005997  345725.31250   \n",
       "2006-12-18          2203.825928             161.792007  347373.62500   \n",
       "\n",
       "            Global_intensity  Sub_metering_1  Sub_metering_2  Sub_metering_3  \n",
       "datetime                                                                      \n",
       "2006-12-16       5180.799805             0.0           546.0          4926.0  \n",
       "2006-12-17      14398.599609          2033.0          4187.0         13341.0  \n",
       "2006-12-18       9247.200195          1063.0          2621.0         14018.0  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# resample data to daily\n",
    "\n",
    "daily_df = df.resample('D')\n",
    "\n",
    "dataset = daily_df.sum()\n",
    "\n",
    "dataset.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.20917603e+03, 3.49220009e+01, 9.35525312e+04, ...,\n",
       "        0.00000000e+00, 5.46000000e+02, 4.92600000e+03],\n",
       "       [3.39045996e+03, 2.26005997e+02, 3.45725312e+05, ...,\n",
       "        2.03300000e+03, 4.18700000e+03, 1.33410000e+04],\n",
       "       [2.20382593e+03, 1.61792007e+02, 3.47373625e+05, ...,\n",
       "        1.06300000e+03, 2.62100000e+03, 1.40180000e+04],\n",
       "       ...,\n",
       "       [1.79624805e+03, 1.32460007e+02, 3.45644594e+05, ...,\n",
       "        1.09600000e+03, 2.84800000e+03, 1.22240000e+04],\n",
       "       [1.43116394e+03, 1.16127998e+02, 3.47812219e+05, ...,\n",
       "        1.07600000e+03, 4.26000000e+02, 5.07200000e+03],\n",
       "       [1.48810400e+03, 1.20826004e+02, 3.03487562e+05, ...,\n",
       "        1.08000000e+03, 3.85000000e+02, 9.98900000e+03]], dtype=float32)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.values"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Train and Test Split**\n",
    "\n",
    "The final year of the data is in 2010 and the first Sunday for 2010 was January 3rd. The data ends in mid November 2010 and the closest final Saturday in the data is November 20th. This gives 46 weeks of test data.\n",
    "\n",
    "The function split dataset() below splits the daily data into train and test sets and organizes each into standard weeks. Specific row offsets are used to split the data using knowledge of the dataset.\n",
    "\n",
    "The split datasets are then organized into weekly data using the `NumPy` `split()` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy import array, split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "def split_dataset(data):\n",
    "    #split into standard weeks\n",
    "    train, test = data[1:-328], data[-328:-6]\n",
    "    train = torch.tensor(split(train, len(train)/7))\n",
    "    test = torch.tensor(split(test, len(test)/7))\n",
    "\n",
    "    return train, test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([159, 7, 7])\n",
      "tensor(3390.4600) tensor(1311.8060)\n",
      "torch.Size([46, 7, 7])\n",
      "tensor(2083.4541) tensor(2197.0061)\n"
     ]
    }
   ],
   "source": [
    "\n",
    "train, test = split_dataset(dataset.values)\n",
    "\n",
    "# validate train data\n",
    "print(train.shape)\n",
    "print(train[0, 0, 0], train[-1, -1, 0])\n",
    "# validate test\n",
    "print(test.shape)\n",
    "print(test[0, 0, 0], test[-1, -1, 0])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Univariate CNN Model\n",
    "\n",
    "In this section, we will develop a convolutional neural network for multi-step time series forecasting using only the univariate sequence of daily power consumption.\n",
    "\n",
    "Basically here we want to answer:\n",
    "\n",
    "> `Given some number of prior days of total daily power consumption, predict the next standard week of daily power consumption?`\n",
    "\n",
    "**Time Series To Supervised Learning Data Preparation**\n",
    "\n",
    "A `1D` CNN model expects data to have the shape of: `[samples, timesteps, features]`. One sample will be comprised of seven time steps with one feature for the seven days of total daily power consumed. The training dataset has 159 weeks of data, so the shape of the training dataset would be: `[159, 7, 1]`.\n",
    "\n",
    "The training data is provided in standard weeks with eight variables, specifically in the shape `[159, 7, 8]`. The first step is to flatten the data so that we have eight time series sequences.\n",
    "\n",
    "We then need to iterate over the time steps and divide the data into overlapping windows; each iteration moves along one time step and predicts the subsequent seven days. \n",
    "\n",
    "For example:\n",
    "\n",
    "`Input, Output`\n",
    "\n",
    "`[d01, d02, d03, d04, d05, d06, d07], [d08, d09, d10, d11, d12, d13, d14]`\n",
    "\n",
    "`[d02, d03, d04, d05, d06, d07, d08], [d09, d10, d11, d12, d13, d14, d15] ...`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([1, 2]), array([3, 4])]"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = array([1, 2, 3, 4])\n",
    "x.shape\n",
    "split(x, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "def to_supervised(train, n_input, n_out=7):\n",
    "    \"\"\"\n",
    "    \n",
    "    input - a list of weeks (history) and the number of time steps to use as inputs outputs \n",
    "    returns - the data in the overlapping moving window format.\n",
    "    \n",
    "    \"\"\"\n",
    "    #flatten data\n",
    "    data = train.reshape(train.shape[0]*train.shape[1], train.shape[2])\n",
    "    X, y = [], []\n",
    "    in_start = 0\n",
    "\n",
    "    for _ in range(len(data)):\n",
    "\n",
    "        #define the end of the input sequence\n",
    "        in_end = in_start + n_input\n",
    "        out_end = in_end + n_out\n",
    "\n",
    "        #ensure we have all the data for that window\n",
    "        if out_end < len(data):\n",
    "            x_input = data[in_start:in_end, 0]\n",
    "            x_input = x_input.view((len(x_input), 1))\n",
    "            X.append(x_input)\n",
    "            y.append(data[in_end:out_end, 0])\n",
    "        # move along one time step\n",
    "        in_start += 1\n",
    "    return torch.tensor(X), torch.tensor(y)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1099, 7, 1])"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X, y = to_supervised(train, 7, 7)\n",
    "X.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we run the above function on the entire training dataset, we transform `159` samples into `1,099`; specifically, the transformed dataset has the shapes `X=[1099, 7, 1]` and `y=[1099, 7]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3.3904600e+03, 2.2600600e+02, 3.4572531e+05, 1.4398600e+04,\n",
       "        2.0330000e+03, 4.1870000e+03, 1.3341000e+04],\n",
       "       [2.2038259e+03, 1.6179201e+02, 3.4737362e+05, 9.2472002e+03,\n",
       "        1.0630000e+03, 2.6210000e+03, 1.4018000e+04],\n",
       "       [1.6661940e+03, 1.5094200e+02, 3.4847900e+05, 7.0940000e+03,\n",
       "        8.3900000e+02, 7.6020000e+03, 6.1970000e+03],\n",
       "       [2.2257480e+03, 1.6099800e+02, 3.4892362e+05, 9.3130000e+03,\n",
       "        0.0000000e+00, 2.6480000e+03, 1.4063000e+04],\n",
       "       [1.7171121e+03, 1.4416600e+02, 3.4710088e+05, 7.2406001e+03,\n",
       "        1.7650000e+03, 2.6230000e+03, 1.0421000e+04],\n",
       "       [2.3413379e+03, 1.8690601e+02, 3.4730575e+05, 9.8970000e+03,\n",
       "        3.1510000e+03, 3.5000000e+02, 1.1131000e+04],\n",
       "       [4.7733862e+03, 2.2147000e+02, 3.4579594e+05, 2.0200400e+04,\n",
       "        2.6690000e+03, 4.2500000e+02, 1.4726000e+04]], dtype=float32)"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train[0]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can define and fit the CNN model on the training data. This multi-step time series forecasting problem is an autoregression\n",
    "\n",
    "**Model Architecture**\n",
    "\n",
    "- convolution layer with 16 filters and kernel size = 3\n",
    "- Relu activation function\n",
    "- A pooling layer\n",
    "- Flattening\n",
    "- Linear layer with 10 nodes\n",
    "- Relu activation function\n",
    "- Linear layer with `n_out` nodes\n",
    "- `MSE` as loss function\n",
    "- Optimizer - `Adam`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [],
   "source": [
    "#define model\n",
    "\n",
    "class UnivariateCNNModel(nn.Module):\n",
    "\n",
    "    def __init__(self, n_timestamp, n_features) -> None:\n",
    "        super().__init__()\n",
    "        self.conv_net = nn.Sequential(\n",
    "                nn.Conv1d(in_channels=n_features, out_channels=16, kernel_size=3),\n",
    "                nn.ReLU(),\n",
    "                nn.MaxPool1d(kernel_size=2),\n",
    "            )\n",
    "        self.flt_layer = nn.Flatten()\n",
    "        self.ln_layer = nn.Sequential(\n",
    "                nn.Linear(in_features=32, out_features=10), \n",
    "                nn.ReLU(), \n",
    "                nn.Linear(in_features=10, out_features=n_timestamp)\n",
    "            )\n",
    "    def forward(self, X):\n",
    "        #X -> (B, T, C)\n",
    "        B, T, C = X.shape\n",
    "        X = X.view(B, C, T)\n",
    "        X = self.conv_net(X)\n",
    "        # print(X.shape)\n",
    "        X = self.flt_layer(X)\n",
    "        X = self.ln_layer(X)\n",
    "\n",
    "        return X\n",
    "\n",
    "#define variables\n",
    "n_timestamp = 7\n",
    "n_features = 1\n",
    "\n",
    "model = UnivariateCNNModel(n_timestamp, n_features)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ -88.9717,   -7.9742, -253.7858,  ...,    9.3355,  174.4473,\n",
       "          -49.3912],\n",
       "        [ -58.1965,  -61.4020, -247.2206,  ...,   45.9940,   78.5566,\n",
       "          -46.1676],\n",
       "        [ -44.2537,  -41.0751, -225.0095,  ...,   33.9901,   94.1954,\n",
       "          -52.4452],\n",
       "        ...,\n",
       "        [ -98.6096,  -11.7230, -228.2829,  ...,   11.5513,  130.4908,\n",
       "          -65.4323],\n",
       "        [ -48.2699,    8.9337, -159.6686,  ...,   15.1565,  108.5201,\n",
       "          -36.7259],\n",
       "        [-112.3470,    9.1581, -229.3958,  ...,   20.7558,  136.7785,\n",
       "          -46.3908]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#check the output from untrained mode just for sanity check\n",
    "model(X)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Define the optimizer and loss**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\n",
    "\n",
    "criterion = torch.nn.MSELoss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(2977705.7500, grad_fn=<MseLossBackward0>)"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#check loss for untrained model\n",
    "y_pred = model(X)\n",
    "criterion(y_pred, y)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Model's Training**\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_timesteps : 7, n_features : 1, n_outputs : 7\n"
     ]
    }
   ],
   "source": [
    "\n",
    "n_timesteps, n_features, n_outputs = X.shape[1], X.shape[2], y.shape[1]\n",
    "\n",
    "print(f\"n_timesteps : {n_timesteps}, n_features : {n_features}, n_outputs : {n_outputs}\")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Training Loop**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2977705.75\n",
      "2965815.5\n",
      "2953951.25\n",
      "2942164.75\n",
      "2930417.5\n",
      "2918724.75\n",
      "2907119.0\n",
      "2895588.25\n",
      "2884119.75\n",
      "2872727.25\n",
      "2861421.0\n",
      "2850182.5\n",
      "2839007.25\n",
      "2827873.0\n",
      "2816739.5\n",
      "2805646.0\n",
      "2794545.5\n",
      "2783444.75\n",
      "2772320.5\n",
      "2761181.75\n",
      "2750000.25\n",
      "2738749.5\n",
      "2727419.25\n",
      "2715972.75\n",
      "2704384.5\n",
      "2692636.0\n",
      "2680690.75\n",
      "2668549.25\n",
      "2656195.75\n",
      "2643591.0\n",
      "2630704.75\n",
      "2617522.0\n",
      "2603997.25\n",
      "2590070.25\n",
      "2575726.5\n",
      "2560923.75\n",
      "2545606.25\n",
      "2529744.0\n",
      "2513264.5\n",
      "2496127.75\n",
      "2478281.75\n",
      "2459632.25\n",
      "2440153.25\n",
      "2419811.25\n",
      "2398541.75\n",
      "2376347.25\n",
      "2353208.75\n",
      "2329074.5\n",
      "2303975.5\n",
      "2277922.25\n",
      "2250898.0\n",
      "2222897.75\n",
      "2193913.0\n",
      "2163932.0\n",
      "2132949.5\n",
      "2100957.75\n",
      "2067946.625\n",
      "2033946.0\n",
      "1998937.0\n",
      "1962917.625\n",
      "1925897.375\n",
      "1887916.5\n",
      "1848983.0\n",
      "1809121.5\n",
      "1768341.875\n",
      "1726718.75\n",
      "1684257.625\n",
      "1641018.25\n",
      "1597070.25\n",
      "1552476.75\n",
      "1507299.25\n",
      "1461580.625\n",
      "1415377.0\n",
      "1368796.0\n",
      "1321985.25\n",
      "1275028.125\n",
      "1228048.75\n",
      "1181188.375\n",
      "1134563.0\n",
      "1088318.5\n",
      "1042561.9375\n",
      "997463.625\n",
      "953177.875\n",
      "909842.25\n",
      "867625.3125\n",
      "826670.5625\n",
      "787105.875\n",
      "749079.1875\n",
      "712694.9375\n",
      "678069.4375\n",
      "645253.375\n",
      "614356.3125\n",
      "585416.75\n",
      "558553.1875\n",
      "533816.375\n",
      "511223.75\n",
      "490811.875\n",
      "472605.03125\n",
      "456567.46875\n",
      "442616.125\n",
      "430630.875\n",
      "420450.15625\n",
      "411860.78125\n",
      "404642.75\n",
      "398555.25\n",
      "393355.84375\n",
      "388812.40625\n",
      "384714.9375\n",
      "380878.28125\n",
      "377149.09375\n",
      "373420.78125\n",
      "369626.09375\n",
      "365731.0\n",
      "361730.0\n",
      "357647.0625\n",
      "353511.09375\n",
      "349368.21875\n",
      "345267.0625\n",
      "341250.28125\n",
      "337360.21875\n",
      "333637.15625\n",
      "330108.15625\n",
      "326795.21875\n",
      "323712.15625\n",
      "320862.71875\n",
      "318245.8125\n",
      "315853.9375\n",
      "313675.25\n",
      "311695.53125\n",
      "309898.3125\n",
      "308266.5\n",
      "306783.09375\n",
      "305432.0625\n",
      "304198.75\n",
      "303069.65625\n",
      "302033.0\n",
      "301079.59375\n",
      "300200.71875\n",
      "299388.4375\n",
      "298641.5\n",
      "297953.5\n",
      "297319.5\n",
      "296737.90625\n",
      "296206.53125\n",
      "295723.25\n",
      "295285.96875\n",
      "294892.5\n",
      "294540.53125\n",
      "294227.4375\n",
      "293950.5625\n",
      "293706.90625\n",
      "293493.3125\n",
      "293306.0625\n",
      "293142.4375\n",
      "292999.25\n",
      "292873.53125\n",
      "292762.46875\n",
      "292663.4375\n",
      "292574.125\n",
      "292492.5\n",
      "292416.84375\n",
      "292345.96875\n",
      "292278.71875\n",
      "292214.34375\n",
      "292152.34375\n",
      "292092.28125\n",
      "292033.875\n",
      "291977.1875\n",
      "291922.0625\n",
      "291868.53125\n",
      "291816.5625\n",
      "291766.21875\n",
      "291717.34375\n",
      "291669.84375\n",
      "291623.625\n",
      "291578.5625\n",
      "291534.59375\n",
      "291491.34375\n",
      "291448.71875\n",
      "291406.5625\n",
      "291364.71875\n",
      "291322.9375\n",
      "291281.15625\n",
      "291239.21875\n",
      "291196.96875\n",
      "291154.5\n",
      "291111.6875\n",
      "291068.5625\n",
      "291025.15625\n",
      "290981.40625\n",
      "290937.46875\n",
      "290893.46875\n",
      "290849.34375\n",
      "290805.03125\n",
      "290760.53125\n",
      "290715.90625\n",
      "290671.15625\n",
      "290626.40625\n",
      "290581.53125\n",
      "290536.6875\n",
      "290491.71875\n",
      "290446.6875\n",
      "290401.5625\n",
      "290356.3125\n",
      "290310.96875\n",
      "290265.53125\n",
      "290219.9375\n",
      "290174.28125\n",
      "290128.53125\n",
      "290082.53125\n",
      "290036.3125\n",
      "289989.96875\n",
      "289943.34375\n",
      "289896.46875\n",
      "289849.34375\n",
      "289801.90625\n",
      "289754.15625\n",
      "289706.28125\n",
      "289658.25\n",
      "289609.96875\n",
      "289561.3125\n",
      "289512.53125\n",
      "289463.5625\n",
      "289414.3125\n",
      "289364.6875\n",
      "289314.875\n",
      "289264.75\n",
      "289214.375\n",
      "289163.6875\n",
      "289112.75\n",
      "289061.6875\n",
      "289010.3125\n",
      "288958.71875\n",
      "288906.5\n",
      "288853.71875\n",
      "288799.90625\n",
      "288745.3125\n",
      "288689.875\n",
      "288633.75\n",
      "288576.6875\n",
      "288518.8125\n",
      "288460.125\n",
      "288400.625\n",
      "288340.625\n",
      "288280.0625\n",
      "288218.75\n",
      "288156.625\n",
      "288093.6875\n",
      "288029.625\n",
      "287966.75\n",
      "287904.65625\n",
      "287842.5625\n",
      "287779.53125\n",
      "287715.40625\n",
      "287648.96875\n",
      "287581.15625\n",
      "287511.15625\n",
      "287439.84375\n",
      "287366.75\n",
      "287292.15625\n",
      "287214.625\n",
      "287133.125\n",
      "287050.09375\n",
      "286962.34375\n",
      "286871.3125\n",
      "286776.625\n",
      "286678.8125\n",
      "286577.6875\n",
      "286474.46875\n",
      "286374.125\n",
      "286280.90625\n",
      "286185.75\n",
      "286090.625\n",
      "285990.90625\n",
      "285890.96875\n",
      "285792.5\n",
      "285694.28125\n",
      "285594.0625\n",
      "285492.5\n",
      "285389.21875\n",
      "285288.34375\n",
      "285188.8125\n",
      "285089.0625\n",
      "284987.34375\n",
      "284882.78125\n",
      "284778.125\n",
      "284674.71875\n",
      "284569.4375\n",
      "284462.90625\n",
      "284355.09375\n",
      "284246.125\n",
      "284136.53125\n",
      "284026.25\n",
      "283915.09375\n",
      "283806.5625\n",
      "283698.5625\n",
      "283590.875\n",
      "283483.0\n",
      "283375.15625\n",
      "283269.15625\n",
      "283164.75\n",
      "283060.6875\n",
      "282956.90625\n",
      "282855.8125\n",
      "282755.375\n",
      "282656.71875\n",
      "282558.28125\n",
      "282460.5625\n",
      "282363.40625\n",
      "282266.8125\n",
      "282171.375\n",
      "282077.1875\n",
      "281984.21875\n",
      "281892.75\n",
      "281803.4375\n",
      "281714.875\n",
      "281627.34375\n",
      "281540.53125\n",
      "281455.25\n",
      "281374.6875\n",
      "281295.0625\n",
      "281216.28125\n",
      "281138.3125\n",
      "281061.15625\n",
      "280984.78125\n",
      "280909.09375\n",
      "280834.28125\n",
      "280760.09375\n",
      "280686.625\n",
      "280613.84375\n",
      "280541.71875\n",
      "280470.96875\n",
      "280401.1875\n",
      "280332.3125\n",
      "280264.0\n",
      "280196.25\n",
      "280129.15625\n",
      "280062.84375\n",
      "279997.1875\n",
      "279931.9375\n",
      "279867.28125\n",
      "279803.34375\n",
      "279740.125\n",
      "279677.375\n",
      "279615.0625\n",
      "279553.46875\n",
      "279492.53125\n",
      "279432.375\n",
      "279372.78125\n",
      "279313.78125\n",
      "279255.84375\n",
      "279198.53125\n",
      "279141.78125\n",
      "279085.8125\n",
      "279030.65625\n",
      "278975.90625\n",
      "278921.53125\n",
      "278867.6875\n",
      "278814.4375\n",
      "278761.75\n",
      "278709.75\n",
      "278658.5\n",
      "278607.75\n",
      "278557.625\n",
      "278508.1875\n",
      "278459.0625\n",
      "278410.40625\n",
      "278362.1875\n",
      "278314.46875\n",
      "278267.375\n",
      "278220.625\n",
      "278174.3125\n",
      "278128.4375\n",
      "278082.9375\n",
      "278037.875\n",
      "277993.21875\n",
      "277948.9375\n",
      "277905.09375\n",
      "277861.65625\n",
      "277818.625\n",
      "277776.0\n",
      "277733.71875\n",
      "277692.0\n",
      "277650.59375\n",
      "277609.40625\n",
      "277568.75\n",
      "277528.46875\n",
      "277488.5\n",
      "277448.9375\n",
      "277409.625\n",
      "277370.6875\n",
      "277332.34375\n",
      "277294.0625\n",
      "277256.0625\n",
      "277218.5625\n",
      "277181.34375\n",
      "277144.375\n",
      "277107.78125\n",
      "277071.40625\n",
      "277035.625\n",
      "277000.0\n",
      "276964.46875\n",
      "276929.4375\n",
      "276894.65625\n",
      "276860.09375\n",
      "276825.90625\n",
      "276791.84375\n",
      "276758.28125\n",
      "276725.0\n",
      "276691.9375\n",
      "276659.125\n",
      "276626.5\n",
      "276594.15625\n",
      "276562.0625\n",
      "276530.3125\n",
      "276498.65625\n",
      "276467.21875\n",
      "276435.96875\n",
      "276404.96875\n",
      "276374.40625\n",
      "276343.9375\n",
      "276313.75\n",
      "276284.0625\n",
      "276254.59375\n",
      "276225.3125\n",
      "276196.34375\n",
      "276167.5625\n",
      "276139.03125\n",
      "276110.78125\n",
      "276082.6875\n",
      "276054.75\n",
      "276027.09375\n",
      "275999.65625\n",
      "275972.3125\n",
      "275945.25\n",
      "275918.40625\n",
      "275891.75\n",
      "275865.1875\n",
      "275838.96875\n",
      "275812.90625\n",
      "275786.90625\n",
      "275760.96875\n",
      "275735.3125\n",
      "275709.875\n",
      "275684.625\n",
      "275659.53125\n",
      "275634.5\n",
      "275609.59375\n",
      "275585.09375\n",
      "275560.53125\n",
      "275535.9375\n",
      "275511.71875\n",
      "275487.59375\n",
      "275463.5\n",
      "275439.71875\n",
      "275415.84375\n",
      "275391.96875\n",
      "275368.40625\n",
      "275344.875\n",
      "275321.4375\n",
      "275298.375\n",
      "275275.28125\n",
      "275252.03125\n",
      "275229.1875\n",
      "275206.3125\n",
      "275183.6875\n",
      "275160.96875\n",
      "275138.625\n",
      "275116.28125\n",
      "275093.875\n",
      "275071.5625\n",
      "275049.34375\n",
      "275027.46875\n",
      "275005.71875\n",
      "274984.0\n",
      "274962.34375\n",
      "274940.75\n",
      "274919.5625\n",
      "274898.4375\n",
      "274877.375\n",
      "274856.5\n",
      "274835.6875\n",
      "274814.9375\n",
      "274794.25\n",
      "274773.9375\n",
      "274753.5625\n",
      "274733.1875\n",
      "274713.21875\n",
      "274693.1875\n",
      "274673.09375\n",
      "274653.0625\n",
      "274633.46875\n",
      "274613.75\n",
      "274593.9375\n",
      "274574.40625\n",
      "274554.8125\n",
      "274535.3125\n",
      "274515.84375\n",
      "274496.6875\n",
      "274477.5625\n",
      "274458.46875\n",
      "274439.6875\n",
      "274421.09375\n",
      "274402.5625\n",
      "274383.9375\n",
      "274365.46875\n",
      "274347.03125\n",
      "274328.84375\n",
      "274310.4375\n",
      "274292.3125\n",
      "274274.15625\n",
      "274256.21875\n",
      "274238.25\n",
      "274220.40625\n",
      "274202.59375\n",
      "274184.8125\n",
      "274167.125\n",
      "274149.59375\n",
      "274131.96875\n",
      "274114.46875\n",
      "274097.0625\n",
      "274079.71875\n",
      "274062.5\n",
      "274045.28125\n",
      "274028.15625\n",
      "274011.0\n",
      "273993.90625\n",
      "273976.84375\n",
      "273960.09375\n",
      "273943.03125\n",
      "273926.25\n",
      "273909.625\n",
      "273892.9375\n",
      "273876.3125\n",
      "273859.625\n",
      "273843.0\n",
      "273826.46875\n",
      "273810.1875\n",
      "273793.84375\n",
      "273777.53125\n",
      "273761.375\n",
      "273745.34375\n",
      "273729.3125\n",
      "273713.3125\n",
      "273697.34375\n",
      "273681.40625\n",
      "273665.5\n",
      "273649.75\n",
      "273634.375\n",
      "273618.75\n",
      "273603.125\n",
      "273587.75\n",
      "273572.40625\n",
      "273557.0625\n",
      "273541.71875\n",
      "273526.34375\n",
      "273510.96875\n",
      "273495.9375\n",
      "273480.6875\n",
      "273465.40625\n",
      "273450.375\n",
      "273435.3125\n",
      "273420.3125\n",
      "273405.25\n",
      "273390.21875\n",
      "273375.3125\n",
      "273360.34375\n",
      "273345.46875\n",
      "273330.6875\n",
      "273315.875\n",
      "273301.1875\n",
      "273286.4375\n",
      "273271.71875\n",
      "273256.96875\n",
      "273242.3125\n",
      "273228.0625\n",
      "273213.28125\n",
      "273198.8125\n",
      "273184.375\n",
      "273170.03125\n",
      "273155.625\n",
      "273141.28125\n",
      "273126.9375\n",
      "273112.625\n",
      "273098.34375\n",
      "273084.125\n",
      "273069.875\n",
      "273055.71875\n",
      "273041.59375\n",
      "273027.46875\n",
      "273013.3125\n",
      "272999.1875\n",
      "272985.09375\n",
      "272971.0\n",
      "272956.96875\n",
      "272942.96875\n",
      "272928.96875\n",
      "272915.03125\n",
      "272901.125\n",
      "272887.21875\n",
      "272873.34375\n",
      "272859.46875\n",
      "272845.625\n",
      "272831.75\n",
      "272817.9375\n",
      "272804.15625\n",
      "272790.375\n",
      "272776.625\n",
      "272762.90625\n",
      "272749.15625\n",
      "272735.3125\n",
      "272721.53125\n",
      "272707.78125\n",
      "272693.96875\n",
      "272680.1875\n",
      "272666.46875\n",
      "272652.71875\n",
      "272639.09375\n",
      "272625.46875\n",
      "272611.96875\n",
      "272598.46875\n",
      "272584.96875\n",
      "272571.375\n",
      "272557.78125\n",
      "272544.28125\n",
      "272530.8125\n",
      "272517.40625\n",
      "272504.0\n",
      "272490.625\n",
      "272477.25\n",
      "272463.875\n",
      "272450.53125\n",
      "272437.3125\n",
      "272424.09375\n",
      "272410.875\n",
      "272397.71875\n",
      "272384.625\n",
      "272371.59375\n",
      "272358.5625\n",
      "272345.5625\n",
      "272332.59375\n",
      "272319.59375\n",
      "272306.59375\n",
      "272293.625\n",
      "272280.71875\n",
      "272267.75\n",
      "272254.84375\n",
      "272241.9375\n",
      "272229.03125\n",
      "272216.1875\n",
      "272203.40625\n",
      "272190.59375\n",
      "272177.78125\n",
      "272164.96875\n",
      "272152.125\n",
      "272139.3125\n",
      "272126.53125\n",
      "272113.71875\n",
      "272100.9375\n",
      "272088.21875\n",
      "272075.46875\n",
      "272062.71875\n",
      "272050.03125\n",
      "272037.28125\n",
      "272024.59375\n",
      "272011.84375\n",
      "271999.09375\n",
      "271986.4375\n",
      "271973.625\n",
      "271960.75\n",
      "271947.78125\n",
      "271934.78125\n",
      "271921.71875\n",
      "271908.5625\n",
      "271895.4375\n",
      "271882.15625\n",
      "271868.65625\n",
      "271854.9375\n",
      "271840.40625\n",
      "271825.5\n",
      "271810.3125\n",
      "271795.15625\n",
      "271779.96875\n",
      "271765.09375\n",
      "271750.71875\n",
      "271736.5625\n",
      "271722.34375\n",
      "271708.3125\n",
      "271694.1875\n",
      "271680.1875\n",
      "271666.4375\n",
      "271653.78125\n",
      "271641.3125\n",
      "271628.71875\n",
      "271616.0625\n",
      "271603.21875\n",
      "271590.09375\n",
      "271576.875\n",
      "271563.5\n",
      "271550.0\n",
      "271536.4375\n",
      "271522.8125\n",
      "271509.25\n",
      "271495.65625\n",
      "271482.0625\n",
      "271468.53125\n",
      "271454.9375\n",
      "271441.21875\n",
      "271427.59375\n",
      "271413.9375\n",
      "271400.1875\n",
      "271386.3125\n",
      "271372.28125\n",
      "271358.34375\n",
      "271344.28125\n",
      "271330.125\n",
      "271315.96875\n",
      "271301.8125\n",
      "271287.59375\n",
      "271273.34375\n",
      "271259.09375\n",
      "271244.875\n",
      "271230.625\n",
      "271216.3125\n",
      "271201.96875\n",
      "271187.5625\n",
      "271173.09375\n",
      "271158.6875\n",
      "271144.28125\n",
      "271129.84375\n",
      "271115.375\n",
      "271100.84375\n",
      "271086.1875\n",
      "271071.59375\n",
      "271057.1875\n",
      "271042.78125\n",
      "271028.375\n",
      "271014.0\n",
      "270999.53125\n",
      "270985.0625\n",
      "270970.5625\n",
      "270956.0625\n",
      "270941.625\n",
      "270927.28125\n",
      "270912.875\n",
      "270898.4375\n",
      "270883.9375\n",
      "270869.34375\n",
      "270854.65625\n",
      "270839.96875\n",
      "270825.4375\n",
      "270810.9375\n",
      "270796.34375\n",
      "270781.75\n",
      "270767.15625\n",
      "270752.5625\n",
      "270738.0\n",
      "270723.40625\n",
      "270708.84375\n",
      "270694.28125\n",
      "270679.71875\n",
      "270665.125\n",
      "270650.53125\n",
      "270636.0\n",
      "270621.53125\n",
      "270607.25\n",
      "270592.9375\n",
      "270578.6875\n",
      "270564.4375\n",
      "270550.21875\n",
      "270536.15625\n",
      "270522.125\n",
      "270508.1875\n",
      "270494.21875\n",
      "270480.3125\n",
      "270466.4375\n",
      "270452.5625\n",
      "270438.6875\n",
      "270424.84375\n",
      "270411.0\n",
      "270397.0\n",
      "270383.0\n",
      "270369.03125\n",
      "270355.03125\n",
      "270340.9375\n",
      "270326.875\n",
      "270312.75\n",
      "270298.71875\n",
      "270284.625\n",
      "270270.5625\n",
      "270256.5\n",
      "270242.4375\n",
      "270228.34375\n",
      "270214.28125\n",
      "270200.1875\n",
      "270186.15625\n",
      "270172.0625\n",
      "270158.0\n",
      "270144.0\n",
      "270129.96875\n",
      "270115.875\n",
      "270101.84375\n",
      "270087.75\n",
      "270073.625\n",
      "270059.46875\n",
      "270045.40625\n",
      "270031.1875\n",
      "270017.03125\n",
      "270002.90625\n",
      "269988.75\n",
      "269974.59375\n",
      "269960.40625\n",
      "269946.21875\n",
      "269932.03125\n",
      "269917.78125\n",
      "269903.59375\n",
      "269889.25\n",
      "269874.84375\n",
      "269860.375\n",
      "269846.0\n",
      "269831.625\n",
      "269817.25\n",
      "269802.9375\n",
      "269788.6875\n",
      "269774.4375\n",
      "269760.21875\n",
      "269745.9375\n",
      "269731.625\n",
      "269717.375\n",
      "269703.15625\n",
      "269689.46875\n",
      "269675.71875\n",
      "269661.9375\n",
      "269648.1875\n",
      "269634.46875\n",
      "269620.90625\n",
      "269607.40625\n",
      "269593.9375\n",
      "269580.4375\n",
      "269566.90625\n",
      "269553.375\n",
      "269539.90625\n",
      "269526.4375\n",
      "269512.96875\n",
      "269499.53125\n",
      "269486.0\n",
      "269472.46875\n",
      "269459.0\n",
      "269445.53125\n",
      "269432.03125\n",
      "269418.53125\n",
      "269405.0\n",
      "269391.5\n",
      "269378.03125\n",
      "269364.5\n",
      "269351.03125\n",
      "269337.59375\n",
      "269324.125\n",
      "269310.65625\n",
      "269297.1875\n",
      "269283.71875\n",
      "269270.28125\n",
      "269256.875\n",
      "269243.34375\n",
      "269229.84375\n",
      "269216.34375\n",
      "269202.8125\n",
      "269189.28125\n",
      "269175.6875\n",
      "269162.0625\n",
      "269148.40625\n",
      "269134.65625\n",
      "269120.96875\n",
      "269107.21875\n",
      "269093.375\n",
      "269079.625\n",
      "269065.84375\n",
      "269052.1875\n",
      "269038.625\n",
      "269025.09375\n",
      "269011.5625\n",
      "268998.09375\n",
      "268984.59375\n",
      "268971.09375\n",
      "268957.59375\n",
      "268944.03125\n",
      "268930.5\n",
      "268917.0625\n",
      "268903.75\n",
      "268890.375\n",
      "268877.0625\n",
      "268863.71875\n",
      "268850.34375\n",
      "268837.0\n",
      "268823.65625\n",
      "268810.28125\n",
      "268796.9375\n",
      "268783.625\n",
      "268770.25\n",
      "268756.9375\n",
      "268743.625\n",
      "268730.3125\n",
      "268717.0\n",
      "268703.6875\n",
      "268690.34375\n",
      "268677.03125\n",
      "268663.71875\n",
      "268650.40625\n",
      "268637.125\n",
      "268623.875\n",
      "268610.625\n",
      "268597.4375\n",
      "268584.21875\n",
      "268571.0625\n",
      "268557.875\n",
      "268544.71875\n",
      "268531.53125\n",
      "268518.375\n",
      "268505.1875\n",
      "268492.0625\n",
      "268478.90625\n",
      "268465.71875\n",
      "268452.59375\n",
      "268439.40625\n",
      "268426.21875\n",
      "268413.0625\n",
      "268399.90625\n",
      "268386.71875\n",
      "268373.8125\n",
      "268360.75\n",
      "268347.5625\n",
      "268334.90625\n",
      "268322.0\n",
      "268308.6875\n",
      "268295.84375\n",
      "268283.03125\n",
      "268270.125\n",
      "268257.03125\n",
      "268243.875\n",
      "268230.78125\n",
      "268217.90625\n",
      "268204.9375\n",
      "268191.78125\n",
      "268178.90625\n",
      "268165.96875\n",
      "268153.03125\n",
      "268140.0\n",
      "268126.96875\n",
      "268114.15625\n",
      "268101.1875\n",
      "268088.15625\n",
      "268075.3125\n",
      "268062.40625\n",
      "268049.46875\n",
      "268036.53125\n",
      "268023.65625\n",
      "268010.6875\n",
      "267997.84375\n",
      "267984.9375\n",
      "267972.03125\n",
      "267959.25\n",
      "267946.25\n",
      "267933.4375\n",
      "267920.53125\n",
      "267907.625\n",
      "267894.78125\n",
      "267881.78125\n",
      "267868.90625\n",
      "267855.9375\n",
      "267842.90625\n",
      "267829.78125\n",
      "267816.6875\n",
      "267803.5\n",
      "267790.40625\n",
      "267777.25\n",
      "267764.25\n",
      "267751.0625\n",
      "267737.9375\n",
      "267724.78125\n",
      "267711.59375\n",
      "267698.5\n",
      "267685.25\n",
      "267672.03125\n",
      "267658.8125\n",
      "267645.5625\n",
      "267632.3125\n",
      "267619.09375\n",
      "267605.96875\n",
      "267592.65625\n",
      "267579.46875\n",
      "267566.3125\n",
      "267553.15625\n",
      "267539.875\n",
      "267526.59375\n",
      "267513.4375\n",
      "267500.25\n",
      "267487.125\n",
      "267473.9375\n",
      "267460.78125\n",
      "267447.625\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Training Loss')"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAERCAYAAAB2CKBkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAgS0lEQVR4nO3deZxddX3/8dd77myZzGSSzEwWshCWUBuQzUhBsaWoLVJ/4I9qK7+2WkV5tHXBVm1t+/vVpY9HtS1aS7UqrahQpXVBRQqlFFFwAybsSUQiEEgIySSELGSb5fP745xJbiaz3JnMmTPn3vfz4X3ce8/5zjmfMwfznu9ZvkcRgZmZ1a66vAswM7N8OQjMzGqcg8DMrMY5CMzMapyDwMysxjkIzMxqXCGDQNI1krZIeqTC9r8laY2k1ZK+knV9ZmZFoiLeRyDpl4HdwLURccoYbZcDXwXOj4jtkuZFxJapqNPMrAgK2SOIiDuB58qnSTpB0n9JWiXpLkkvSme9Hfh0RGxPf9YhYGZWppBBMIKrgXdFxEuA9wH/nE4/CThJ0g8l/UTSBblVaGY2DdXnXcBkkNQKvAz4mqTByU3pez2wHDgPWAzcKenFEfH8FJdpZjYtVUUQkPRsno+I04eZtwG4OyJ6gSck/YwkGO6dwvrMzKatqjg0FBE7Sf6RfwOAEqels79F0htAUifJoaLHcyjTzGxaKmQQSLoe+DHwC5I2SLoM+B3gMkkPAquBi9PmtwLbJK0B7gDeHxHb8qjbzGw6KuTlo2ZmNnkK2SMwM7PJU7iTxZ2dnbFs2bK8yzAzK5RVq1ZtjYiu4eYVLgiWLVtGd3d33mWYmRWKpPUjzfOhITOzGucgMDOrcQ4CM7Mal1kQSGqWdI+kB9Phnz88TJsmSf8haZ2kuyUty6oeMzMbXpY9gv0kQz+fBpwOXCDp7CFtLgO2R8SJwD8Af5thPWZmNozMgiASu9OvDelr6N1rFwNfSj9/HXilykaNMzOz7GV6jkBSSdIDwBbgtoi4e0iTRcDTABHRB+wAOrKsyczMDpdpEEREfzoi6GLgLEmjPk1sJJIul9Qtqbunp2dCtazbsouPfGcNB/oGJvTzZmbVakquGkrH/r8DGPpQmI3AEgBJ9UA7cMSAcBFxdUSsjIiVXV3D3hg3pqef28s1P3yC7/9sYkFiZlatsrxqqEvS7PTzDODVwE+HNLsReHP6+fXAdyOjUfDOXd5Jx8xGvnn/hiwWb2ZWWFn2CBYCd0h6iOQhMLdFxE2SPiLporTN54EOSeuAPwE+kFUxDaU6/tdpx/A/a7ewY29vVqsxMyuczMYaioiHgDOGmf5XZZ/3AW/Iqoah/vcZi/jij57kloc38cazlk7Vas3MprWaurP41MXtHN81k2/evzHvUszMpo2aCgJJXHLGIu5+4jk2bN+TdzlmZtNCTQUBwMWnLwLg2w88k3MlZmbTQ80FwZK5LZy1bC7fvH8jfkynmVkNBgHA685YxLotu3lk4868SzEzy11NBsFvvHghjaU6nzQ2M6NGg6C9pYHzXzSPGx/cSF+/h5wws9pWk0EA8JsvWczW3Qe467GteZdiZparmg2CXzmpi7kzG/nGfR5ywsxqW80GQWN9HReddgz/vWazh5wws5pWs0EAcMmZizjQN8DND2/KuxQzs9zUdBC8eFE7J85r5QYfHjKzGlbTQSCJS85cxL1Pbmf9thfyLsfMLBc1HQQArzt9ERK+p8DMalbNB8Exs2fwshM6uOE+DzlhZrWp5oMA4JIzFvPUc3voXr8971LMzKacgwC44JQFzGgo+aSxmdUkBwEws6me15yygJse2sS+3v68yzEzm1IOgtQlZy5m174+bl+7Je9SzMymlIMgdc4JHXS2NnLLI765zMxqi4MgVaoTr14xnzt+usWHh8yspjgIyvz6yQt44UA/P1znEUnNrHY4CMq87IRO2prquXX1s3mXYmY2ZRwEZRrr63jlL87jtjWb/cAaM6sZDoIhfv3kBWzf08sq31xmZjXCQTDEy5d3UqqTn1xmZjXDQTDErOYGzlgym7t8wtjMaoSDYBjnLu/koQ3P8/yeA3mXYmaWucyCQNISSXdIWiNptaQrhmlznqQdkh5IX3+VVT3j8YrlXUTAD9dty7sUM7PM1We47D7gvRFxn6Q2YJWk2yJizZB2d0XEazOsY9xOW9xOW3M9dz3Ww2+cujDvcszMMpVZjyAiNkXEfennXcBaYFFW65tM9aU6fum4Du5+4rm8SzEzy9yUnCOQtAw4A7h7mNnnSHpQ0i2STh7h5y+X1C2pu6enJ8tSD1q5bA5PbH2Bbbv3T8n6zMzyknkQSGoFvgG8JyJ2Dpl9H3BsRJwG/BPwreGWERFXR8TKiFjZ1dWVab2DVh47B8D3E5hZ1cs0CCQ1kITAlyPihqHzI2JnROxOP98MNEjqzLKmSp2yqJ3GUp2DwMyqXpZXDQn4PLA2Ij4xQpsFaTsknZXWMy0u1WluKHHKolkOAjOrelleNfRy4PeAhyU9kE77C2ApQER8Fng98IeS+oC9wBtjGj1B/sylc7j2J+vp7R+goeRbLsysOmUWBBHxA0BjtPkU8KmsajhaL17czoG+AdZt2c0vLpyVdzlmZpnwn7mjOPmY5B//1c8MPcdtZlY9HASjOK6zlRkNJVY/syPvUszMMuMgGEWpTvziwjZWb3SPwMyql4NgDCcf086aTTsZGJg257DNzCaVg2AMJx8zi937+3jquT15l2JmlgkHwRhelF4t9LPNu3KuxMwsGw6CMZzQNROAx7bszrkSM7NsOAjG0NbcwML2Zn7uIDCzKuUgqMCJ81rdIzCzquUgqMDyeW2s27LbVw6ZWVVyEFTgxHmt7O3tZ+Pze/Muxcxs0jkIKrB8fisA63x4yMyqkIOgAid2JUHw8x4HgZlVHwdBBWa3NDCruZ7123xTmZlVHwdBBSRxbMdMntz2Qt6lmJlNOgdBhY7taPEwE2ZWlRwEFVrWMZMN2/fS2z+QdylmZpPKQVChpR0t9A8EG7f7ElIzqy4Oggot60jGHFrvw0NmVmUcBBU6tqMFgPU+YWxmVcZBUKF5bU00N9T5ElIzqzoOggpJYlnHTPcIzKzqOAjGYencFp50j8DMqoyDYByWzm3h6ef2EOFRSM2sejgIxmHRnBns7xtg2wsH8i7FzGzSOAjGYdHsGQA84+GozayKOAjGYdGcJAh8U5mZVZNxBYGkOkmzKmy7RNIdktZIWi3pimHaSNJVktZJekjSmeOpZ6oN9gj8gBozqyZjBoGkr0iaJWkm8AiwRtL7K1h2H/DeiFgBnA28Q9KKIW1eAyxPX5cDnxlX9VOsfUYDMxtLDgIzqyqV9AhWRMRO4HXALcBxwO+N9UMRsSki7ks/7wLWAouGNLsYuDYSPwFmS1o4jvqnlCSOmT3Dh4bMrKpUEgQNkhpIguDGiOgFxnX9pKRlwBnA3UNmLQKeLvu+gSPDYlpZNGcGz+xwEJhZ9agkCD4HPAnMBO6UdCyws9IVSGoFvgG8J+1ZjJukyyV1S+ru6emZyCImjXsEZlZtxgyCiLgqIhZFxIXpIZz1wK9WsvC0J/EN4MsRccMwTTYCS8q+L06nDa3h6ohYGREru7q6Kll1ZhbNnsH2Pb3sOdCXax1mZpOlkpPFV6QniyXp85LuA86v4OcEfB5YGxGfGKHZjcCb0mWfDeyIiE3j2YCptniO7yUws+pSyaGht6aHdH4NmENyovhjFfzcy9O250t6IH1dKOkPJP1B2uZm4HFgHfAvwB+Newum2DHpJaQbfHjIzKpEfQVtlL5fCFwXEavTv/ZHFRE/KPvZkdoE8I4Kapg2fC+BmVWbSnoEqyT9N0kQ3CqpDajZB/fOa2uiTrB5x768SzEzmxSV9AguA04HHo+IPZI6gLdkWtU0Vl+qo7O1iWd3OgjMrDqMGQQRMSBpMfB/0iNC34+I72Re2TS2oL2ZzTv3512GmdmkqOSqoY8BVwBr0te7Jf1N1oVNZ/PamtnsHoGZVYlKDg1dCJweEQMAkr4E3A/8RZaFTWcL2pvoXv9c3mWYmU2KSkcfnV32uT2DOgplflszz+/pZV9vf96lmJkdtUp6BB8F7pd0B8nloL8MfCDTqqa5+e3NAGzZuZ+lHS05V2NmdnQqGWLiepJhpG8gGS7iHJKxh2rW/FlJEGze5fMEZlZ8lfQISId9uHHwu6R7gKVZFTXdLUiD4FnfS2BmVWCij6oc887iajYYBL5yyMyqwUSDYFzPI6g2s2bU01Rf5yAws6ow4qEhSd9h+H/wBXRkVlEBSPJNZWZWNUY7R3DlBOfVhPltzR5mwsyqwohBEBHfn8pCimZ+ezMPb3g+7zLMzI7aRM8R1Lz5bcnAc8lI2mZmxeUgmKAF7c3s6x1g5z4/stLMis1BMEHzfQmpmVWJMW8oG+HqoR1AN/C5iKjJfwnnl91UdtL8tpyrMTObuEp6BI8Du0meKfwvwE5gF3BS+r0mdbU1AbB1ty8hNbNiq2SIiZdFxEvLvn9H0r0R8VJJq7MqbLobDIKeXQ4CMyu2SnoErZIOjiuUfm5Nvx7IpKoCmNlYYkZDyUFgZoVXSY/gvcAPJP2c5K7i44A/kjQT+FKWxU1nkuhqa/KhITMrvEqeWXyzpOXAi9JJj5adIP5kVoUVQWdrIz0OAjMruIqGoQZeAixL258miYi4NrOqCqKrrYkntr6QdxlmZkelkstHrwNOAB4ABp/NGICDoK2Je57ws4vNrNgq6RGsBFaEx1I4QldrM9v39NLbP0BDyffmmVkxVfKv1yPAgqwLKaLOtkYAtu2u2YunzKwKVNIj6ATWpI+nPHhmNCIuyqyqguhqPXQvwYL0gfZmZkVTSRB8aCILlnQN8FpgS0ScMsz884BvA0+kk26IiI9MZF158d3FZlYNKrl8dKLPJfgi8ClGP6l8V0S8doLLz11nq+8uNrPiG/EcgaQfpO+7JO0se+2StHOsBUfEnUBVX1JzcJgJ9wjMrMBGDIKIODd9b4uIWWWvtoiYNUnrP0fSg5JukXTySI0kXS6pW1J3T0/PJK366DU3lGhrrnePwMwKraIbyiSVgPnl7SPiqaNc933AsRGxW9KFwLeA5cM1jIirgasBVq5cOa0uY+1qa3KPwMwKrZIbyt4FfBDYDAykkwM49WhWHBE7yz7fLOmfJXVGxNajWe5U62xtco/AzAqtkh7BFcAvRMS2yVyxpAXA5ogISWeRHKaa1HVMha62JtY+M+YpEzOzaauSIHia5Ilk4yLpeuA8oFPSBpJeRQNARHwWeD3wh5L6gL3AG4t493JXaxN3ukdgZgVWSRA8DnxP0n9y+A1lnxjthyLi0jHmf4rk8tJC62prYtf+Pvb19tPcUMq7HDOzcaskCJ5KX43py8qU3128ZG5LztWYmY1fJTeUfXgqCimq8nsJHARmVkQjBoGkT0bEeyR9h+QqocN4rKHEwWEmfJ7AzApqtB7Bden7lVNRSFENDjOxxUFgZgU1YhBExKr0faJjDdWEjtbktIkHnjOzoqrkhrLlwEeBFcDBsZYj4vgM6yqMhlIdc1oaHARmVliVPJjmC8BngD7gV0lGE/23LIsqmq42311sZsVVSRDMiIjbAUXE+oj4EPAb2ZZVLJ2tTWz1U8rMrKAquY9gv6Q64DFJ7wQ2Aq3ZllUsXW1N3P/U83mXYWY2IZX0CK4AWoB3Ay8Bfhd4c5ZFFU3SI/ChITMrplF7BOnw078dEe8DdgNvmZKqCqarrYk9B/p5YX8fM5sqGtnbzGzaGO0JZfUR0Q+cO4X1FNLgvQTuFZhZEY325+s9wJnA/ZJuBL4GvDA4MyJuyLi2wjg4zMSu/RzbMTPnaszMxqeS4xjNJM8JOJ9kqAml7w6CVKdvKjOzAhstCOZJ+hPgEQ4FwKDCPTcgS+U9AjOzohktCEokl4lqmHkOgjJzWxqRoMf3EphZAY0WBJsi4iNTVkmB1Zfq6JjZ6B6BmRXSaPcRDNcTsBH4XgIzK6rRguCVU1ZFFehqcxCYWTGNGAQR8dxUFlJ0na0eeM7MiqmSISasAp2tjWzdvZ8In0c3s2JxEEySrrYm9vUOsHt/X96lmJmNi4NgkhwaZsKXkJpZsTgIJolvKjOzonIQTBIPPGdmReUgmCTuEZhZUTkIJsmclkbq5B6BmRVPZkEg6RpJWyQ9MsJ8SbpK0jpJD0k6M6tapkKpTnT4XgIzK6AsewRfBC4YZf5rgOXp63LgMxnWMiU8zISZFVFmQRARdwKj3Z18MXBtJH4CzJa0MKt6pkJXm3sEZlY8eZ4jWAQ8XfZ9QzrtCJIul9Qtqbunp2dKipuI5O5i30dgZsVSiJPFEXF1RKyMiJVdXV15lzOiwR6Bh5kwsyLJMwg2AkvKvi9OpxVWV2sTB/oH2LnPw0yYWXHkGQQ3Am9Krx46G9gREZtyrOeo+V4CMyuiSh5ePyGSrgfOAzolbQA+CDQARMRngZuBC4F1wB7gLVnVMlXK7y4+cV5rztWYmVUmsyCIiEvHmB/AO7Jafx4Gg8A9AjMrkkKcLC6KwUNDvpfAzIrEQTCJZs9ooFQn9wjMrFAcBJOork4Hn1RmZlYUDoJJlgwz4ZvKzKw4HASTbF5bE5t37su7DDOzijkIJtmC9hkOAjMrFAfBJFswq5mtuw+wv68/71LMzCriIJhkC9ubAdiy0yeMzawYHASTbH4aBM/68JCZFYSDYJIN9gie3eEgMLNicBBMsgUOAjMrGAfBJGtrqqelscQmB4GZFYSDYJJJYkF7sy8hNbPCcBBkYMGsZjbt2Jt3GWZmFXEQZCDpEfjyUTMrBgdBBha2N/Pszn309Q/kXYqZ2ZgcBBlYMqeF/oHwvQRmVggOggwsndsCwFPP7cm5EjOzsTkIMrAkDYKnHQRmVgAOggwsbG+mVCeefs5XDpnZ9OcgyEB9qY5jZjf70JCZFYKDICNL57Y4CMysEBwEGVk6t4UN2x0EZjb9OQgysnhOC1t3H+CF/X15l2JmNioHQUZ8CamZFYWDICMndLUCsG7L7pwrMTMbnYMgI8d3zaRO8NjmXXmXYmY2KgdBRpobSizrmMmjDgIzm+YyDQJJF0h6VNI6SR8YZv7vS+qR9ED6eluW9Uy15fNbeWyzDw2Z2fSWWRBIKgGfBl4DrAAulbRimKb/ERGnp69/zaqePJw0v40nt73Avt7+vEsxMxtRlj2Cs4B1EfF4RBwA/h24OMP1TTsrFs5iIGDtpp15l2JmNqIsg2AR8HTZ9w3ptKF+U9JDkr4uaclwC5J0uaRuSd09PT1Z1JqJM4+dA8Cq9dtzrsTMbGR5nyz+DrAsIk4FbgO+NFyjiLg6IlZGxMqurq4pLfBozJ/VzOI5MxwEZjatZRkEG4Hyv/AXp9MOiohtETH4TMd/BV6SYT25WHnsHLrXbyci8i7FzGxYWQbBvcByScdJagTeCNxY3kDSwrKvFwFrM6wnF2cf30HPrv2s3eTLSM1sesosCCKiD3gncCvJP/BfjYjVkj4i6aK02bslrZb0IPBu4Pezqicvr1oxnzrBLY9syrsUM7NhqWiHLFauXBnd3d15lzEub7rmHn66aSc/+LPzaazP+7SMmdUiSasiYuVw8/yv0hR4+yuOY8uu/Xzhh0/kXYqZ2REcBFPg3BM7efWK+Vz5349y58+Kc/mrmdUGB8EUkMSVbziNE7paedu13dzysM8XmNn04SCYIu0zGrj+7WdzyjGz+KOv3Md1P34y75LMzAAHwZSaM7ORL7/tbF75onn8v2+v5spbH/X9BWaWOwfBFJvRWOKzv/sS3vjSJXzqjnX82Tceoq9/IO+yzKyG1eddQC2qL9Xx0UtezLxZzVx1+2Ns2rGPj1x8Csd1zsy7NDOrQQ6CnEjiT159Egvbm/nrm9bwyo9/j3OXd/HqFfM5c+lsTuhqpbmhlHeZZlYDfEPZNLBl1z6u/dF6vvXARjZs33twemdrE/NnNdHSWGJGYz0tDSVKJVGSKNWJOolSHWWfk/f6uvRzXdK2ru7QtMPeS3XDTK87+L2+VPa9NHK7Up1oKNVRXxINdel7qY6GkpCU42/WzAaNdkOZg2AaiQg2bN/LfU9tZ/22PWzcvpee3fvZc6CPvQf62dvbT99AMDAQ9EcwMAD9Bz8n7/0Dh14DMfie3zYNhsVgUNTXJQFRHhqHptUN27Y86CSok6hL35V+HmwzdH5d3ZFtD85Pf2a4+RIIQELJG+LQ9MHvHPxe1q6sLUPnDVkGh30/chnDLp/y+oYuc5TPDFdb2c8PrvPg9MNrTFd3xLI42G6w9sGpw9ebLvGI2srbo0M1jPQ7ZJh1DFuj/xgBRg8CHxqaRiSxZG4LS+a2TOpyI5Iw6BsYoH8g6BsI+vvT94Ggb2CAviHfD7YbCPr6R5g+EPT1D9A3EPT2J8voTb/39Q/Q239o2YOfe/uP/JmD09P3PQf60vlJ28FQGwgYiCDS9yToBrfvyPn9A4e3zTMQbXoYM3woC7URwuqwcB4hrI4MzyGByKGAGjmENSQ84dKzlvK2Vxw/yb8VB0FNkERJUKrzOYco6yUNDZWBgIE0dAKIgCBI/3fwexz8nnxmpHnpdA6bXtau7PO4l1FBjYPbwRHrHHnZI9XBcPMG6y5b3uC6DrUdZjvTmeXbcmg5Y6yT8vWW/+5iSD1jrHOEGjlsGRWsc4waD/5uh6ln6L4rX+bQ38Pg77aztYksOAispkjJYSkzO8T3EZiZ1TgHgZlZjXMQmJnVOAeBmVmNcxCYmdU4B4GZWY1zEJiZ1TgHgZlZjSvcWEOSeoD1E/zxTmDrJJZTBN7m2uBtrg1Hs83HRkTXcDMKFwRHQ1L3SIMuVStvc23wNteGrLbZh4bMzGqcg8DMrMbVWhBcnXcBOfA21wZvc23IZJtr6hyBmZkdqdZ6BGZmNoSDwMysxtVMEEi6QNKjktZJ+kDe9UwWSUsk3SFpjaTVkq5Ip8+VdJukx9L3Oel0Sboq/T08JOnMfLdgYiSVJN0v6ab0+3GS7k636z8kNabTm9Lv69L5y3It/ChImi3p65J+KmmtpHOqeT9L+uP0v+lHJF0vqbka97OkayRtkfRI2bRx71dJb07bPybpzeOpoSaCQFIJ+DTwGmAFcKmkFflWNWn6gPdGxArgbOAd6bZ9ALg9IpYDt6ffIfkdLE9flwOfmfqSJ8UVwNqy738L/ENEnAhsBy5Lp18GbE+n/0Parqj+EfiviHgRcBrJ9lflfpa0CHg3sDIiTgFKwBupzv38ReCCIdPGtV8lzQU+CPwScBbwwcHwqEjyXM7qfgHnALeWff9z4M/zriujbf028GrgUWBhOm0h8Gj6+XPApWXtD7YrygtYnP6f43zgJpLnem8F6ofub+BW4Jz0c33aTnlvwwS2uR14Ymjt1bqfgUXA08DcdL/dBPx6te5nYBnwyET3K3Ap8Lmy6Ye1G+tVEz0CDv1HNWhDOq2qpN3hM4C7gfkRsSmd9SwwP/1cDb+LTwJ/Cgyk3zuA5yOiL/1evk0HtzedvyNtXzTHAT3AF9JDYv8qaSZVup8jYiNwJfAUsIlkv62i+vfzoPHu16Pa37USBFVPUivwDeA9EbGzfF4kfyJUxXXCkl4LbImIVXnXMsXqgTOBz0TEGcALHDpcAFTdfp4DXEwSgMcAMzny8ElNmIr9WitBsBFYUvZ9cTqtKkhqIAmBL0fEDenkzZIWpvMXAlvS6UX/XbwcuEjSk8C/kxwe+kdgtqT6tE35Nh3c3nR+O7BtKgueJBuADRFxd/r96yTBUK37+VXAExHRExG9wA0k+77a9/Og8e7Xo9rftRIE9wLL0ysOGklOOt2Yc02TQpKAzwNrI+ITZbNuBAavHHgzybmDwelvSq8+OBvYUdYFnfYi4s8jYnFELCPZj9+NiN8B7gBenzYbur2Dv4fXp+0L91dzRDwLPC3pF9JJrwTWUKX7meSQ0NmSWtL/xge3t6r3c5nx7tdbgV+TNCftTf1aOq0yeZ8kmcKTMRcCPwN+Dvxl3vVM4nadS9JtfAh4IH1dSHJ89HbgMeB/gLlpe5FcQfVz4GGSqzJy344Jbvt5wE3p5+OBe4B1wNeApnR6c/p9XTr/+LzrPortPR3oTvf1t4A51byfgQ8DPwUeAa4DmqpxPwPXk5wH6SXp+V02kf0KvDXd/nXAW8ZTg4eYMDOrcbVyaMjMzEbgIDAzq3EOAjOzGucgMDOrcQ4CM7Ma5yCwqiIpJH287Pv7JH1oEpbbJOl/JD0g6beHzPuipCfSeQ9I+tHRrm/I8r8nqaYe0m5Tq37sJmaFsh+4RNJHI2LrJC73DICIOH2E+e+PiK9P4vrMpox7BFZt+kie6/rHQ2dIWibpu+k47rdLWjpMm7mSvpW2+YmkUyXNA/4NeGn6F/8JlRQi6UOSrpP043SM+Len0yXp79Nx9h8u72FI+rN02oOSPla2uDdIukfSzyS9Im17cjrtgbTe5eP6TZml3COwavRp4CFJfzdk+j8BX4qIL0l6K3AV8LohbT4M3B8Rr5N0PnBtRJwu6W3A+yLitSOs8+8l/d/08+pIhr0AOJXkOREzgfsl/SfJ8MmnkzxToBO4V9Kd6bSLgV+KiD3pGPOD6iPiLEkXkow7/yrgD4B/jIgvp0OnlCr79ZgdzkFgVScidkq6luTBJnvLZp0DXJJ+vg4YGhSQDNnxm+lyviupQ9KsClY70qGhb0fEXmCvpDtIHhpyLnB9RPSTDC72feClwK8AX4iIPen6nytbzuBggqtIxq4H+DHwl5IWAzdExGMV1Gl2BB8asmr1SZIxW2bmXMfQMVwmOqbL/vS9n/QPuIj4CnARSdjdnPZgzMbNQWBVKf1r+qscepQhwI9IRiwF+B3grmF+9K50HpLOA7bGkOc7jNPFSp6120EySN696Tp+W8lzl7uAXyYZKO024C2SWtL1zx1hmaTzjwcej4irSEanPPUo6rQa5kNDVs0+Dryz7Pu7SJ7w9X6Sp329ZZif+RBwjaSHgD0cGgp4LOXnCCA5BATJSKF3kJwL+OuIeEbSN0kOUz1I0kP400iGmf4vSacD3ZIOADcDfzHKOn8L+D1JvSRPsfqbCms1O4xHHzXLSHr/wu6IuDLvWsxG40NDZmY1zj0CM7Ma5x6BmVmNcxCYmdU4B4GZWY1zEJiZ1TgHgZlZjfv//EO4nNwrEK4AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "epochs = 1000\n",
    "\n",
    "losses = []\n",
    "\n",
    "for _ in range(epochs):\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    y_pred = model(X)\n",
    "    loss = criterion(y_pred, y)\n",
    "    print(loss.item())\n",
    "    losses.append(loss.item())\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "\n",
    "#plot the training losses\n",
    "\n",
    "plt.plot(losses)\n",
    "plt.xlabel(\"No of Epochs\")\n",
    "plt.ylabel(\"Training Loss\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Evaluation\n",
    "\n",
    "Models will be evaluated using a scheme called **walk-forward validation**. This is where a model is required to make a one week prediction, then the actual data for that week is made available to the model so that it can be used as the basis for making a prediction on the subsequent week.\n",
    "\n",
    "This is both realistic for how the model may be used in practice and beneficial to the models, allowing them to make use of the best available data. \n",
    "\n",
    "For example:\n",
    "\n",
    "`Input -> predict`\n",
    "\n",
    "`[Week1] -> Week2`\n",
    "\n",
    "`[Week1 + Week2]  -> Week3`\n",
    "\n",
    "`[Week1 + Week2 + Week3]  ->  Week4`\n",
    "\n",
    "`...`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "# summarize scores\n",
    "def summarize_scores(name, score, scores):\n",
    "    s_scores = ', '.join(['%.1f' % s for s in scores]) \n",
    "    print('%s: [%.3f] %s' % (name, score, s_scores))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_forecasts(actual, predicted):\n",
    "  \n",
    "  scores = list()\n",
    "  # calculate an RMSE score for each day\n",
    "  for i in range(actual.shape[1]):\n",
    "    # calculate mse\n",
    "    mse = mean_squared_error(actual[:, i], predicted[:, i])\n",
    "    # calculate rmse\n",
    "    rmse = np.sqrt(mse)\n",
    "    # store\n",
    "    scores.append(rmse)\n",
    "  # calculate overall RMSE\n",
    "    s=0\n",
    "    for row in range(actual.shape[0]):\n",
    "        for col in range(actual.shape[1]):\n",
    "            s += (actual[row, col] - predicted[row, col])**2\n",
    "    score = np.sqrt(s / (actual.shape[0] * actual.shape[1]))\n",
    "  return score, scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(159, 7, 7)\n",
      "[2195.452 2094.098 2047.968 2451.11  2211.892 1224.252 1311.806]\n"
     ]
    }
   ],
   "source": [
    "history = [x for x in train]\n",
    "\n",
    "data = array(history)\n",
    "print(data.shape)\n",
    "data = data.reshape((data.shape[0]*data.shape[1], data.shape[2]))\n",
    "input_x = data[-7:, 0]\n",
    "print(input_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1442, 7)"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([2008.4426, 2112.8894, 2104.7432, 2032.2062, 2088.1284, 2155.6680,\n",
       "        2071.2598], grad_fn=<SelectBackward0>)"
      ]
     },
     "execution_count": 210,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[-1]\n",
    "\n",
    "model.forward(X[-1][None])[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2195.452, 2094.098, 2047.968, 2451.11 , 2211.892, 1224.252,\n",
       "        1311.806]], dtype=float32)"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train[-1][:, 0][None]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(46, 7, 7)"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "metadata": {},
   "outputs": [],
   "source": [
    "#evaluation \n",
    "\n",
    "#get the last train \n",
    "n_test = len(test)\n",
    "n_input = 7\n",
    "\n",
    "#extract the last 7 days training data from train\n",
    "history = train[-1][:, 0]\n",
    "#rehsape hist into (B, T, C)\n",
    "B = 1\n",
    "T = 7\n",
    "C = 1\n",
    "\n",
    "predictions = []\n",
    "\n",
    "for i in range(n_test):\n",
    "    n_in = 0\n",
    "    # reshape the data from (B, T, C) -> (B, C, T)\n",
    "    history = torch.tensor(history).view(B, T, C)\n",
    "    # # reshape into [1, n_input, 1]\n",
    "    with torch.no_grad():\n",
    "        yhat = model.forward(history)\n",
    "        # store the predictions\n",
    "        predictions.append(yhat[0].tolist())\n",
    "    history = test[i][n_in:n_in+n_input, 0]\n",
    "    n_in += n_input\n",
    "    \n",
    "predictions = array(predictions)\n",
    "score, scores = evaluate_forecasts(array(test[:, :, 0].tolist()), predictions)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cnn: [430.179] 492.5, 429.3, 349.1, 395.3, 425.0, 325.5, 551.1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAEXCAYAAAAeBxskAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAA8+0lEQVR4nO3dd3wUZf7A8c83jYQEUmlJgNCRIgRDPWxYsCByds9TUe88uz+9Q4+fnuKpP08563lnb1gOu2I71FMUTahSQpGyIRBCzYYESC/P74+Z4BJSNqTM7ub7fr3yyu7slO/szM535nmeeUaMMSillFL+IMjpAJRSSilvadJSSinlNzRpKaWU8huatJRSSvkNTVpKKaX8hiYtpZRSfsPnk5aIHC8iGxxYbraInNpGy3pARPJEZFdbLE8dSUSuF5HdInJQROJ9IJ5nReQv/jZvfyMia0XkpAY+XyAiv2vDeIyI9G+r5fkjn0la9SUJY8xCY8ygVlqmEZEi+0CVKyKPiUhwE+dxkohsb0YMvYA/AkOMMd3rmX+1HeMBEdkgIlfVsR57RCTEY1ioPcx4DBsqIl+KSL6IFIjIchE5q47leP6NP9p1a45a28YtIv8VkYtbaVmhwGPA6caYKGOMuwXmecTBR0Rmicgb3kxvjLnOGHN/c+No7rxb4qAtIpNF5Ht7/90rIt+JyFT7s+n2d3VHrWm21yQT+3szInKRx+ch9rCU5sRmjBlqjFngsRyvtk9dROSyWr+dYjvG45oTY3OJSIodR0jjYzd7Wa2e5H0maTlohDEmCjgF+A3w+zZefi/AbYzZ08A4O+wYOwO3AS+ISO1Evg840+P9mfYwT58AXwHdga7ALcD+2sup9ZfR9FXyXiM/pJptMwh4FXhaRO5thTC6AeHA2qZOKBa/+R019aSsBZZ3AfAuMAdIxvqu7wHO8RgtH7hDRDo1MKt84L62jr8pjDFvev52gBuALOAnh0MLLMYYn/gDsoFT6xh+ErC91nh/AlYDhcDbQLjH51OAlUABkA4c28AyDdDf4/27wNO14wE6AE8AO+y/J+xhkUAJUA0ctP8S61hONNaPdi+wFbgb64Th1FrTv9rY+tvD9gAX1lqPu4F3PYa9B9xlbWIDkGCPF1PPd3HEcrzYXjOBdVjJ8RVvt4M97Z32NiwDQhrbNvawC4BSIN5+fxWwHjiAdXD4g8e4a4BzPN6HAnlAaq15DgSK7OUdBL6xh08Altr72FJggsc0C4AHgR/t7dffy/hnAW94ft9YV9l7gJ3AVR7jvgo8YL9eD0zx+CzE3pdGeey3u+xYvweG1prPM8Dn9nqeWmvescCn9vz22a+T7c8eBKrs7/wgv/w2BmOd/OQDG4CL6tlHBNgGzGhgP5oO/IB1QnWvx/DtwEke39ubwCrgSo/vwAApdczzZCDT4/1XwFKP9wuBaZ6/c+AMoByosNd1lce2vt/e1geAL4EEL38j33quUz3jzLC3/Q7gas/9BjgbWIF1YpkDzPKY7jPg5lrzWg38uo5lpNjzDfHYJ/5pz+MAsBjoV2vfvQXrN5UHzAaCau/Dtedd1/5i7wOPY+3j+4FMYJi3x5k6v7PmTNySfzQtaS0BEoE4rB/0dfZnqfaXMxYIBq60x+9QzzI9d5AhWD/8a2rHA/wVWIR1ddIF6yB8f13x1bOcOcDHQCd7I2/0WE6D03t+jpXopmIludRa6zEM2A3EYB2IdtvDjD2OAJuwDkrTgG4Nfc9ebq81QE97O/zILwfCBreD/XqlPW1EY9vGY1goUAmc6fGj7mev24lAMb8cyO8A3vaY9lw8DmSN/KjjsA7gl2P9GC+139ckywVYB+Oh9uehXsY/i8OTVqW9b4UCZ9nxx9qfv+rxfd4DvOkxn7OB9R7vr8bat2pOrlZ6fPYqVjL7lb3/hNeadzxwPtDRnse7wEce0y8AfufxPhLrAHqVve6pWAe2IXV8B4Pt76FPA/vRdKykNdL+juPs4bWT1htY+36W/X01lLQisA6cCfa4u4Fce/0isE40arZlNr/8zg9tn1rr78I6uYmw3//Ni99Hb6wDeEPrfga//E4jgbc4/Jh0EjDc3m7H2uNOsz+7CFjsMa8RgBsI82L/ftUed4z9Pb4JzK21736L9TvohXW8+l1d31Ed8669v0wGlmMdlwQ4Bujh7XGmrj+/Kdao5SljzA5jTD7WGdpIe/i1wHPGmMXGmCpjzGtYZ/LjGpjXTyKyz57Pi1hXDLVdBvzVGLPHGLMXuA/rgNYouzjjEmCmMeaAMSYbeNTb6W2JIlKA9WP7ELjdGLOi1jil9jpcbP/Ns4cBhzLXyVg/0keBnXY9w4Day6n1F9lAXE8bY3Ls7fAg1sEdvNsOT9nTlnj7JRhjKrAOkHH2+8+MMS5j+Q7rLPh4e/Q3gLNEpLP9/nLgdS8XdTawyRjzujGm0hjzb+BnDi/SetUYs9b+vMLbdailAmu/qjDGfI51dlpX/e1bwFQR6Wi//w3w75oPjTEv2/tWGdZBZYSIRHtM/7Ex5kdjTLUxptRjOMYYtzHmfWNMsTHmANZ2PLGBmKcA2caYV+x1XwG8D1xYx7g1DVp2NjC/mjhWYl0R3dnAOPOwrggbrDOx96mlwAnAcVhXaD9iJe5xWNu2KfWWrxhjNtrzfYdfjjcNuQJYaIzZ0sA4F9nzXmOMKcLadp7rscAYk2lvt9VY27xm28wDBnr8fi/HOkkr93KdPjTGLDHGVGIlrdrr9LAxJt8Ysw3rROhSjk4F1snCYECMMeuNMY3uDw3x16Tl2cquGIiyX/cG/uh50MU6m09sYF6jjDGxxph+xpi7jTHVdYyTiFWsV2NrI/P0VHO2V3v6JC+nB6uuKQarTuspYFI9483B+rFcYb8+jDFmuzHmJmNMP6zvqqjWeDuMMTG1/ooaiCvH47Xnd+LNdvCc1it2g4kuWMVSiMiZIrKopmEJ1tVKgr2uO7AOVOeLSAxWHd+bXi6q9vaGI7dZY/FXYW13T6FYP+IabvugUcNzXz7EGLMZq0ThHDtxTcVKZIhIsIj8TURcIrIf66QE7O+hsVhFpKOIPCciW+3pvwdiGqg76g2MrbVtL8OqJ62tJjH0qG/5tdwDXC8i3RoY526sYu/wRub1HdaVygn26wVYB/wT7fdNUd/xpiFXAK/VvBGRXp6NNOzBiRz5G8JjmrEi8q3deKUQuI5f9u9SrKqR39p1qpfi/UmZN+tU32+7SYwx32AVE/4T2CMiz3ucSB4Vf01a9ckBHqx10O1onyk3xw6sH2uNXvYwsC6NG5KHdaCqPX1uU4Owz6TvBIaLyLQ6RlmIdYDohlXk0tC8crB2pGFNjcNDT4/Xnt+JN9uhse+tLudiFaktEZEOWGf4f8cq6ozBqrcRj/FfA36LdRWQYYzx9juvvb3hyG3WWPzbsIpOPPXhyGTorX9jHZjOBdbZiQysq65zseploj2W6fk9NBTrH7Gu7sYaYzpjHeQ9p689bQ7wXa1tG2WMub6OeW+wxz+/sZUDMMb8DHyAlZTqG+crYDNWI4eG1E5a39F40jqaffIIIvIrrIP8e4dmbMw2c3gjDbCuQGv/hjy9hXVF1dMYEw08y5H792VYjciKTcs2mqrvt12EVZRco/bJyhHfoTHmKWPMcVhVMAOx6vGOmq8lrVARCff4a2oTzReA6+wzFBGRSBE5u5FWSd74N3C3iHQRkQSsM8KaprG7gfhaxTGHGGOqsIoUHhSRTiLSG7jdY/omsS//H7VjqP2ZwSrCmmq/PkREYkXkPhHpLyJB9npcjVVXd7RuFJFkEYnDOtC8bQ9v0e0gInEichlWkn3YLtoJw6rD2QtUisiZwOm1Jv0IGAXcSh1Xng34HKvo5Td20+qLsX5wnzZhHm9j7TPJ9vd9Kta2ea+R6eozF2v9rse+yrJ1wip6dWMdTP6vifPthFXsXGBvx3trfb4b6Ovx/lOs7+ZysW6rCBWR0SJyTO0Z2/vg7cBfROQqEelsfxcTReT5euK5D6u+LKaBmO/CqrNsSDpWMh4DLDHGrMW+SsS6mqzLbiClBVqDXgm8bxe3NuQdYLqIDLGvoGt/952AfGNMqYiMwTpBOcROUtVYx4OmXGV5Y4Z9zOiJ9fup+W2vBE6wrxyjsRpjeTpsf7H3jbF2KUkRVpVFXaVZXvO1pPU51g+o5m9WUyY2xizDarL+NFal7masit7megBYhtU6JxOrCesD9jJ/xkpqWXZxSV2X0TdjbbAsrCugt4CXmxHPy0AvETmn9gd2PUtdTbfLsc7Cv8ZqxbMG62A33WOcRDnyPq2GzpLfwqpHysKqrK75TlpqO6yyi1I2Y9Vj3GaMucdexgGsFk7v2Mv4DdZZ6SF2HcT7WFc4H3i7UDspTsG6CnFjHSCnGGPymhD7X7EOnD/Y8T0CXGaMWdOEeXjGtBPIwGrV+LbHR3Owrt5ysVpyNvUk5AmsBgZ59rT/qfX5k8AFIrJPRJ6yv/fTseppd2AVMz2MdQJRV9zvYdWxXm2PvxtrP/m4nvG3YB2A661LNcb8iNUYq152sfZPwFqPep4MYKup//aSd+3/bhE5qmbqIhKOVVf1WmPjGmO+wPr+v8Hax7+pNcoNwF9F5ADWSeo7dcxmDlZjjaO+v6weH2M1oFiJ1crwJTvmr7D2v9X257VP5A7bX7CqNF7A+g1sxfo9zQYQkf8VkS+aGpjUOiFXyisiko3VSuhrp2NpiIjcAww0xvzW6ViUamkicgVwrTFmYgvO0wADPIqgfUqr3yGtlFPs4q5raFpLTaX8gl2keAPwL6djaUu+VjyoVIsQkd9jNQL4whhTXx2GUn5JRCZj1efu5vA6zoCnxYNKKaX8hl5pKaWU8huatJRSSvkNTVpKKaX8hiYtpZRSfkOTllJKKb+hSUsppZTf0KSllFLKb2jSUkop5Tc0aSmllPIbmrSUUkr5DU1aSiml/IYmLaWUUn5Dk5ZSSim/oUlLKaWU39CkpZRSym9o0lJKKeU3QpwOACAhIcGkpKQ4HYZSSvmV5cuX5xljujgdR1vyiaSVkpLCsmXLnA5DKaX8iohsdTqGtqbFg0oppfyGJi2llFJ+Q5OWUkopv6FJSymllN/QpKWUUspv+ETrQaWUam8+WpHL7Pkb2FFQQmJMBDMmD2JaapLTYfk8r660RCRbRDJFZKWILLOHzRKRXHvYShE5y2P8mSKyWUQ2iMjk1gpeKaX80Ucrcpn5QSa5BSUYILeghJkfZPLRilynQ/N5TbnSOtkYk1dr2OPGmL97DhCRIcAlwFAgEfhaRAYaY6qaF6pSSgWG2fM3UFJx+CGxpKKK2fM36NVWI1qjTutcYK4xpswYswXYDIxpheUopZRf2lFQ0qTh6hfeJi0DfCkiy0XkWo/hN4nIahF5WURi7WFJQI7HONvtYYcRkWtFZJmILNu7d+9RBa+UUv4oMSaiScPVL7xNWhONMaOAM4EbReQE4BmgHzAS2Ak82pQFG2OeN8akGWPSunRpV11nKaXauRmTBxEcJIcNiwgNZsbkQQ5F5D+8SlrGmFz7/x7gQ2CMMWa3MabKGFMNvMAvRYC5QE+PyZPtYUoppYBpqUl0iQqjQ0gQAiTFRPDQecO1PssLjTbEEJFIIMgYc8B+fTrwVxHpYYzZaY/2a2CN/Xoe8JaIPIbVEGMAsKTlQ1dKKf+090AZu/aXcecZg7n+pH5Oh+NXvGk92A34UERqxn/LGPMfEXldREZi1XdlA38AMMasFZF3gHVAJXCjthxUSqlfZGS5AZjQL97hSPxPo0nLGJMFjKhj+OUNTPMg8GDzQlNKqcCU4cqjU3gIQxM7Ox2K39FunJRSqo1luNyM7RNPSLAegptKvzGllGpDuQUlZLuLGa9Fg0dFk5ZSSrWhDJfWZzWHJi2llGpDGS43cZFhDOrWyelQ/JImLaWUaiPGGDJceYzrG0dQrZuLlXc0aSmlVBvZ6i5mR2Ep4/slOB2K39KkpZRSbUTvz2o+TVpKKdVG0l1uunbqQN+ESKdD8VuatJRSqg3U1GdN6BeP3cOQOgqatJRSqg1s2nOQvIPlTND6rGbRpKWUUm2g5v4svam4eTRpKaVUG0h35ZEcG0HPuI5Oh+LXNGkppVQrq642LMrK11aDLUCTllJKtbJ1O/dTWFKh9VktQJOWUkq1Mq3PajmatJRSqpWlu/Lo2yWSbp3DnQ7F72nSUkqpVlRRVc2SLVqf1VI0aSmlVCvKzC2kqLyK8X21PqslaNJSSqlWVFOfNa5vnMORBAZNWkop1YrSXXkM7t6J+KgOTocSEDRpKaVUKymrrGJZ9j5tNdiCNGkppVQrWbGtgLLKar0/qwVp0lJKqVaS7nITJDCmj9ZntRRNWkop1UoWudwMT4omOiLU6VAChiYtpZRqBcXllazI2cc4rc9qUZq0lFKqFSzL3kdFldH6rBamSUsppVpBRpabkCBhdEqs06EEFE1aSinVCtJdbkb2jKFjWIjToQQUTVpKKdXC9pdWkLm9QPsbbAWatJRSqoUtycqn2sB4rc9qcZq0lFKqhWVkuQkLCSK1V4zToQQcTVpKKdXC0l1u0nrHEh4a7HQoAUeTllJKtaD8onLW79yv9VmtRJOWUkq1oMVZ1qNItD6rdWjSUkqpFpTuctMxLJhjk6OdDiUgeZW0RCRbRDJFZKWILLOHxYnIVyKyyf4faw8XEXlKRDaLyGoRGdWaK6CUUr4k3ZXHmD5xhAbrNUFraMq3erIxZqQxJs1+/2fgv8aYAcB/7fcAZwID7L9rgWdaKlillPJle/aX4tpbpPVZrag5pwLnAq/Zr18DpnkMn2Msi4AYEenRjOUopZRfyKipz+qr9VmtxdukZYAvRWS5iFxrD+tmjNlpv94FdLNfJwE5HtNut4cdRkSuFZFlIrJs7969RxG6Ukr5lvTNbjqHhzAksbPToQQsbzvFmmiMyRWRrsBXIvKz54fGGCMipikLNsY8DzwPkJaW1qRplVLKF6Vn5TGubzzBQeJ0KAHLqystY0yu/X8P8CEwBthdU+xn/99jj54L9PSYPNkeppRSASsnv5ic/BLGa31Wq2o0aYlIpIh0qnkNnA6sAeYBV9qjXQl8bL+eB1xhtyIcBxR6FCMqpVRAqqnP0udntS5vige7AR+KSM34bxlj/iMiS4F3ROQaYCtwkT3+58BZwGagGLiqxaMGPlqRy+z5G9hRUEJiTAQzJg9iWuoRVWdKKdUmMlxu4iPDGNgtyulQAlqjScsYkwWMqGO4GziljuEGuLFFoqvHRytymflBJiUVVQDkFpQw84NMAE1cSqk2Z4whw+VmfL947BN81Ur88u632fM3HEpYNUoqqpg9f4NDESml2rMteUXs2l+q9VltwC+T1o6CkiYNV0qp1pTu0vqstuKXSSsxJqLO4QlRHdo4EqWUshph9IgOJyW+o9OhBDy/TFozJg8iotZzagQ4WFbB5j0HnAlKKdUuVVcbFrncjO+r9VltwS+T1rTUJB46bzhJMREIkBQTwV1nH0Nkh1Auf2mJFhMqpdrMxj0HcBeVa31WG/G2RwyfMy016YiWguP7xXPJc4u4/KXFvHvdBOIiwxyKTinVXqRvrnl+liattuCXV1r1GZoYzYtXppGzr4SrXl1KUVml0yEppQJcRpabXnEdSY7V+qy2EFBJC2Bs33ievjSVzO0FXPfGcsorq50OSSkVoKqqDYuy3PookjYUcEkL4PSh3fnb+ceycFMet7+zkupq7Y9XKdXy1u4o5EBppRYNtiG/rdNqzEVpPckvKudvX/xMXGQY900dqi17lFItKsNV8/wsTVptJWCTFsB1J/Yjv6ic57/PIj6yA7eeOsDpkJRSASTd5aZ/1yi6dg53OpR2I6CTFsDMMwfjPljO419vJC4qjMvH9XY6JKVUAKioqmZpdj4XHJfsdCjtSsAnLRHh4fOHU1Bczj0fryG2YyhTjk10OiyllJ9bvb2A4vIqbYTRxgKyIUZtIcFB/POyUaT1juW2t1eycNNep0NSSvm59M1uRGBsH01abaldJC2A8NBgXrxyNP26RPGH15ezKqfA6ZCUUn4s3eXmmO6didVODNpUu0laANERocy5egzxUWFMf2UJm/ccdDokpZQfKq2oYvm2fVo06IB2lbQAunYO5/WrxxIcJFzx0mJ2Fmo/hUqppvlp2z7KK6v1/iwHtLukBZCSEMmrV43hQGkll7+0hH1F5U6HpJTyIxkuN8FBwpg+cU6H0u60y6QFMCwpmuevSGNbfjFXvbqU4nLtp1Ap5Z10l5vhSdF0Cg91OpR2p90mLbB6Zf7Hpams3l7AdW/8pP0UKqUaVVRWyaqcAi0adEi7TloAk4d256HzhvP9xr386d1V2k+hUqpBS7Pzqaw22gjDIQF/c7E3Lh7dC3dROY/8ZwNxkWHce84Q7adQKVWnDJeb0GAhrbfWZzlBk5bt+hP7kX+wnBd/2EJ8ZBg3n6L9FCqljpSR5Sa1VywRYcFOh9IutfviwRoiwv+edQznpSbx6FcbeWPRVqdDUkr5mMLiCtbkFmqv7g7SKy0PQUHCwxccS0FJBX/5eA1xkWGcNbyH02EppXzE4i1uqg1an+UgvdKqJTQ4iH/+ZhTH9Yrlf+au5MfNeU6HpJTyERlZbsJDgxjZK8bpUNotTVp1iAgL5qUrR9MnIZJr5yxj9fYCp0NSSvmADJebtN5xdAjR+iynaNKqR3THUOZcM4bYyDCmv7IU117tp1Cp9izvYBk/7zqg92c5TJNWA7p1Duf1a8YiwBUvLWFXYanTISmlHLIoyw1ofZbTNGk1ok9CJK9dPYbCkgouf2kxBcXaT6FS7VGGy01UhxCGJ0U7HUq7pknLC1Y/hcex1V3M1dpPoVLtUobLzZg+cYQE62HTSfrte2lCvwSeunQkK3MKuOHNn6io0n4KlWovdhWWkpVXpEWDPkCTVhOcMawHD/56OAs27GWG9lOoVLuRkWXd+jJObyp2nN5c3ESXjulFflE5s+dvIDYyjHumaD+FSgW69M1uoiNCGdKjs9OhtHuatI7CDSf1w32wnJd/3EJCVAduPLm/0yEppVpRusvN+L7xBAXpCarTvC4eFJFgEVkhIp/a718VkS0istL+G2kPFxF5SkQ2i8hqERnVSrE7RkS4++xj+HVqErPnb+CtxducDkkp1Upy8ovJLShhQn8tGvQFTbnSuhVYD3heH88wxrxXa7wzgQH231jgGft/QAkKEh654Fj2FZdz90eZxHYM5Uztp1CpgJPusuqztJNc3+DVlZaIJANnAy96Mfq5wBxjWQTEiEhAHs1Dg4P412WjGNkzhlvnriRd+ylUKuCku9wkRHWgf9cop0NReF88+ARwB1C7nfeDdhHg4yLSwR6WBOR4jLPdHnYYEblWRJaJyLK9e/c2MWzf0TEshJenjyYloSO/n7OMzO2FToeklGohxhjSXW4m9IvXBlc+otGkJSJTgD3GmOW1PpoJDAZGA3HAnU1ZsDHmeWNMmjEmrUuXLk2Z1OfEdAxjztVjiekYxvRXlpCl/RQqFRBce4vYe6BM+xv0Id5caf0KmCoi2cBcYJKIvGGM2WkXAZYBrwBj7PFzgZ4e0yfbwwJa9+hwXr9mDAa4/KUl7N6v/RQq5e8y7PosvanYdzSatIwxM40xycaYFOAS4BtjzG9r6qnEumaeBqyxJ5kHXGG3IhwHFBpjdrZK9D6mb5coXrtqDAXF5Vzx0hIKiyucDkkp1QzpLjdJMRH0iuvodCjK1pweMd4UkUwgE0gAHrCHfw5kAZuBF4AbmhWhnxmeHM0LV6SxJa+Iq19bSkl5ldMhKaWOQnW1YVGWm3F9tT7LlzQpaRljFhhjptivJxljhhtjhhljfmuMOWgPN8aYG40x/ezPl7VG4L5sQv8EnrhkJD9t28cNby7XfgqV8kM/7zrAvuIKLRr0Mdr3YCs5a3gPHpg2jG837OWO91ZrP4VK+ZlD92dp0vIp2o1TK7psbG/yD5bz6FcbiYsM4+6zj9FiBqX8xKIsN30SIkmMiXA6FOVBk1Yru2lSf9xF5bz0wxbio8K44STtp1ApX1dZVc3irHymjEh0OhRViyatViYi3DNlCPuKy3nkPxuI6xjGJWN6OR2WUqoBa3bs50BZpdZn+SBNWm0gKEiYfcEICoor+N8PM4npGMYZw7o7HZZSqh419Vn6/Czfow0x2khYSBDP/HYUI3rGcMvcFWS43E6HpJSqR4bLzcBuUXTp1KHxkVWb0qTVhjqGhfDK9NH0jrP6KVyTq/0UKuVryiurWZqdz4R+CU6HouqgSauNxXQMY841Y4iOCGX6K0vIzityOiSllIeVOQWUVlRrU3cfpUnLAT2iI5hzzRiqDVz+8mL2aD+FSvmMDJcbERjXR5OWL9KGGA7p1yWKV6aP5tIXFnHu0z+ACLsKS0mMiWDG5EFMSz3iaS5KqTaQ7spjaGJnojuGOh2KqoNeaTloRM8Yrhyfws79ZewsLMUAuQUlzPwgk49WBHzH+Er5nNKKKlZsK9D6LB+mScth81btOGJYSUUVs+dvcCAapdq35Vv3UV5VzXht6u6zNGk5bEdBSZOGK6VaT7orj+AgYXSfOKdDUfXQOi2HJcZEkFtHgtL+zlRL+WhFLrPnb2BHQYnWmTYi3eVmRHI0UR300Oir9ErLYTMmDyIiNPiwYQLceor2Uaia76MVucz8IJPcghKtM23EwbJKVm8v1PosH6dJy2HTUpN46LzhJMVEIEBCVBgG+GlbgcORqUAwe/4GSioOfxCp1pnWbemWfKqqjd6f5eP0GtgHTEtNOqy45uH//MwzC1ycMLALZw3v4WBkyp/l5BfXWfQMWmdal3RXHmHBQRzXO9bpUFQD9ErLB91+2kBGJEfz5/dX13vQUao+OwtLuOvDTE7++4J6x9E60yOlu9yM6h1DeK3ieuVbNGn5oNDgIJ66NJWqasNtc1dSpU89Vl7Yc6CUWfPWcuLsBbyzLIdLxvRk1tQhR9aZCvzp9IEORembCorLWbdzP+P7an2Wr9PiQR/VOz6S+6cN4/Z3VvH0N5u59dQBToekfJT7YBnPfZ/FnIxsKqoMF4xK5qZJ/ekZ1xGAmIiwQ60HoyNCKSipsFr7qEMWZeVjDEzor/VZvk6Tlg87b1Qy32/cy5P/3civ+seTlqL3jqhfFBZX8PxCF6/+mE1JRRXTRiZxyykDSEmIPGw8zzrT6mrDtH/9yN+++JnTh3QnUpt2A5DhyiMiNJgRyTFOh6IaocWDPu7+acNIio3g1rkrKSypcDoc5QMOlFbw5NebmPjIN/zzWxcnD+7Kl7edwGMXjzwiYdUWFCTce85Qdu8v45/fbm6jiH1fRpabtJRYwkL0kOjrdAv5uE7hoTx5SSq79pdy14eZGKP1W+1VUVkl/1qwmeMf+ZbHv97I+L7xfHHr8Tz9m1H079rJ6/kc1zuWX6cm8eLCLWx166Nx9h4oY+Pug3p/lp/QpOUHRvWK5fbTBvLp6p28u3y70+GoNlZaUcWLC7M44ZFveeQ/G0jtGcMnN03k+SvSOKZH56Oa55/PHExIsPDAZ+tbOFr/k5FlPUV8gt6f5Re0QNtPXHdiPxZu2suseWtJ6x1L3y5RToekWllZZRVzl+Twz283s+dAGRP7J3DbaQNb5D6ibp3DufHk/syev4GFm/Zy/IAuLRCxf8pw5dEpPIShiUd3AqDall5p+YngIOGJi1MJCwni1rkrKa+sdjok1Uoqqqp5a/E2Tp69gHvnrSUlPpK5147jjd+NbdEbX6+Z2IdecR2575N1VFS13/0pw+VmbJ84QoL1cOgPdCv5ke7R4Tx8/rFk5hby9y+1G55AU1lVzbvLcpj06AL+98NMunYO5/VrxvD2H8YxrhUelREeGszdZx/D5j0HeT1ja4vP3x/kFpSQ7S5mvNZn+Q0tHvQzk4d257KxvXj++ywm9k/ghIHtt1gnUFRVGz5dvYMnv95EVl4Rw5I6c9/0oZw8qCsirXtD1WlDunH8gAQe/3oj545MJD6qQ6suz9dkuLQ+y9/olZYfuvvsIQzoGsXt76wi72CZ0+Goo1RdbfgicydnPvk9t85dSWhwEM/+9jg+uWkikwZ3a/WEBSAi3DNlCMXlVfz9y42tvjxfk+FyE9sxlEHdvG99qZylScsPRYQF84/fpLK/tIIZ767SZvB+xhjD1+t2M+UfP3D9mz9RVW34x6WpfHHr8ZwxrHubJCtPA7p14orxvZm7dBtrcgvbdNlOMsaQ4cpjfL94goK0ixB/oUnLTw3u3pm7zjqGbzfs5dX0bKfDUV4wxvDdxr1M+1c6v5uzjKLySh67aARf3nYi54xIdPTA+T+nDiS2Yxj3fbK23ZwEbXUXs6OwVOuz/IwmLT92xfjenDK4Kw99/jPrdux3OhzVgAyXmwufzeDKl5eQd6CMh88fzte3n8h5o5IJ9oGz/OiIUP50+iCWZu9j3qodTofTJmruzxrfCo1cVOvRpOXHRIRHLjiWmI6h3Pzvnygpr2p8ItWmlm/N5zcvLOLSFxaRs6+Y+6cN49s/ncTFo3sR6mNNrC8e3ZOhiZ156POfKS6vdDqcVpfuctO1Uwf6dWm46yvlW3zrV6OaLD6qA49dNJKsvCLu/2yd0+Eo26qcAq58eQnnP5PBxt0H+MuUIXw342QuH9fbZ/u3Cw4SZk0dyq79pTyzwOV0OK2qpj5rQr/4Nq9DVM2jTd4DwMQBCVx7Ql+e+y6LEwYkcMYwfdqxU9bt2M9jX23k6/W7ie0Yyp/PHMwV43vTMcw/fmqjU+KYOiKR577P4qK0nocebxJoNu05SN7BcsZrU3e/4/Upn4gEi8gKEfnUft9HRBaLyGYReVtEwuzhHez3m+3PU1opduXhj6cN4tjkaO58P1Mfpe6ATbsPcMObyznrqYUs2eLmj6cNZOGdk7juxH5+k7BqzDxrMMEiPBjA/RL+cn+WNsLwN00pp7gV8NyLHwYeN8b0B/YB19jDrwH22cMft8dTrSwsJIgnL0mloqqa297Wpx23lay9B7l17gpOf+J7vt+Yxy2T+rPwzkncfMoAovz0WVU9oiO44aR+/GftLtI35zkdTqtId+WRHBsRsFeSgcyrpCUiycDZwIv2ewEmAe/Zo7wGTLNfn2u/x/78FNFC4zbRJyGSv547jMVb8nlmgT4rqTXl5Bfzp3dXcdrj3/Pl2t384YR+LLzjZG4/fRDREaFOh9dsvz+hL8mxEdz3yToqA6xfwupqw6KsfO0Fw095eyr4BHAHUHPbeDxQYIypaWK0HUiyXycBOQDGmEoRKbTHP+yUTUSuBa4F6NWr11GGr2o7f1QS32/cy+Nfb2J8v4QW7WBVwY6CEv7xzWbeXZZDUJAwfUIK153Yjy6dAqv7o5p+Ca974yfeXLyNKyekOB1Si1m3cz+FJRVan+WnGk1aIjIF2GOMWS4iJ7XUgo0xzwPPA6SlpWlZVgsRER749TB+2raPW+eu4PNbj6dzuP+f+TvhoxW5zJ6/gR0FJXTrHM6AblEszsrHYLh0TC9uPLk/3aPDnQ6z1Uwe2p0J/eJ57KuNTB2RSGxkmNMhtYia+qzxfbU+yx95Uzz4K2CqiGQDc7GKBZ8EYkSkJuklA7n261ygJ4D9eTTgbsGYVSM620873llYyt0frmk3PRy0pI9W5DLzg0xyC0owwK79pSzclMeoXjF8+6eTuH/asIBOWGCdAN17zlAOllXy6FeB81SBdFcefbtEBvz2C1SNJi1jzExjTLIxJgW4BPjGGHMZ8C1wgT3alcDH9ut59nvsz78xetRsc8f1juV/ThnAvFU7eP+n3MYnUIeZPX8DJRVH3qyds6+E5Nj2U3k/qHsnfju2F28t3hYQva5UVFWzZEu+9oLhx5pzl+OdwO0ishmrzuole/hLQLw9/Hbgz80LUR2tG07uz9g+cdzz8Rqy84qcDsdvVFUbcuu5baA93k5w22kDiY4IDYh+CTNzCykqr9Km7n6sSUnLGLPAGDPFfp1ljBljjOlvjLnQGFNmDy+13/e3P89qjcBV44KDhMcvHklocBC3zF2hTzv2QmFxBVe/urTezxNjItowGt8Q0zGM208fxOIt+XyeucvpcJqlpj5rXN84hyNRR8s3+5NRLSYxJoKHzx/O6u2FAVUv0Rp+3rWfc57+gXRXHhemJRMRGnzY5xGhwcyYPMih6Jz1mzG9GNy9E//3+Xq/7uMyw+VmcPdO7e5hl4FEk1Y7cMawHlw6phfPfZfFD5sC82bR5vps9U7O+1c6pRVVzL12PLMvGMFD5w0nKSYCAZJiInjovOFMS01qdF6BqKZfwtyCEp773j/7JSyrrGJpdr42dfdz/nnLvmqye6YMYWl2Pre/s5Ivbj1ezzRtVdWG2fM38Ox3Lo7rHcszl42ia2erVdm01KR2m6TqMq5vPGcP78Gz37m4MK0nSX5WVLpiWwFlldVan+Xn9EqrnYgIC+apS1IpKK7gzvdX+32FeksoKC5n+itLePY7F5eN7cW/fz/uUMJSdZt51mCMgf/73P/6JUx3uQkSGNNH67P8mSatdmRIYmdmnjWYr9fvYU7GVqfDcdT6nVb91eKsfP523nAe/PVwn31kiC9Jju3IdSf247PVO1mU5V+3Xy5yuRmWFB0Q3Wy1Z/orbWemT0jh5EFdePDz9azf6f/33RyNT1bt4Lx/pVNeWc3cP4zjkjHajVhTXHdiPxKjw7nvk3V+0zFzcXklK3L2aX1WANCk1c6ICLMvHEHn8FBu+fcKSuu4gTZQVVZV89Dn67n53ysYmtiZT26eyKhe2jdjU0WEBfO/Zx/D+p37+feSbU6H45Vl2fuoqDJanxUANGm1QwlRHXjsohFs2nOQB9rJ0473FZUz/ZWlPPd9FpeP681bvx9H105af3W0zh7eg7F94nj0yw0UFlc4HU6jMrLchAQJadqBtN/TpNVOnTCwC9ee0Jc3Fm1j/lr/vmG0Met27GfqP39gyZZ8Hjn/WO6fNkzrr5qppl/CwpIKHv96o9PhNCrd5WZkzxgi/fQZZ+oX+sttx/50+iCGJXXmzvdXs7MwMLsn+nhlLuc98yMVlYZ3rhvPRaN7Oh1SwBiS2JlLx/Ti9UVb2bDrgNPh1Gt/aQWZ2wv0+VkBQpNWOxYWEsRTl6RSXlnN7W+v8ptKdW9UVlXz4GfruHXuSo5NiuGTmycysmeM02EFnD+ePoioDiH89VPf7Zdw6ZZ8qg2M06QVEDRptXN9u0Qxa+pQMrLcPPudf/Z0UFt+UTlXvrKEFxZu4crxvXnjd2MD7iGNviIuMozbTxvIj5vdzF+72+lw6pTuchMWEqSNbgKEJi3FhcclM+XYHjz21UZWbNvndDjNsnZHIef84weWZu9j9gXHct+5Wn/V2i4b24tB3Trx4OfrfLI1arrLTVrvWMJr9SWp/JP+mhUiwoO/Hk73zuHcMncFB0p9vzVYXT5emcv5z6RTbQzv/mE8F6Zp/VVbCAkO4t5zhpCTX8KLC33roQ75ReWs37lf67MCiCYtBUB0RChPXjKS3H0l/OWjNU6H0ySVVdXc/6ldf5Ucw7ybJjJC66/a1IT+CZwxtDv//NblU416Ftu9duhNxYFDk5Y6JC0ljltPGchHK3fw4YrtTofjFffBMq54eQkv/bCF6RNSeFPrrxxz19nHUGUMf/viZ6dDOSTd5aZjWDDHJsc4HYpqIZq01GFumtSfMSlx3P3hGra6fftpx2tyC5n69I8s27qPv184gllThxIarLu0U3rGdeQPJ/Tl45U7WJad73Q4AKS78hjTJ073iwCiW1IdJjhIePySkQQHCbfMXUlFlW8+7fjDFds5/5l0jDG8d914Ljgu2emQFHD9Sf3oER3OvfPWOn4LxZ79pbj2FjG+rxYNBhJNWuoISTER/O38Y1mVU8BjX/lWbwcVVdX89ZN13Pb2Kkb2jGHezRO16MeHdAwL4c9nDmbtjv28syzH0Vgy7Pos7W8wsGjSUnU6a3gPLhndk2e/c5G+2Teeduw+WMblLy3m5R+3cNWvUnjjd2NJ0IdZ+pypIxIZnRLL7PkbKCxxriVq+mY3ncNDGJLY2bEYVMvTpKXqdc85Q+ibEMlt76wkv6jc0Vgyt1v3X63YVsBjF43g3nO0/spX1fRLuK+4nCe/3uRYHBlZbsb2jSc4SByLQbU8/dWrenUMC+GpS1PZV1TBHe8597Tj95dv5/xn0xER3r9+AueN0vorXzcsKZpLRvdkTkY2m/e0fb+EOfnFbMsv1vuzApAmLdWgoYnR3HnmYL5ev5s3FrXt044rqqqZNW8tf3x3Fcf1imXeTb9iWFJ0m8agjt6fTh9ERFgw932yrs1PeLQ+K3Bp0lKNumpCCicO7MIDn61vs9688w6W8dsXF/NqejbXTOzD69eMIV7rr/xKfFQH/ufUgSzclMfX6/e06bIzXG7iI8MY2C2qTZerWp8mLdWooCDh7xeOoFMbPe149fYCzvnHD6zMKeCJi0fylylDCNH6K790xfje9O8axQOfraOssm36JTTGkOFyM65fPCJanxVo9EigvNKlUwcevWgEG3Yf4P8+X99qy3l3WQ4XPJtBkF1/NS01qdWWpVpfaHAQ90wZwlZ3MS/9sKVNlrklr4hd+0u1PitAadJSXjtxYBd+N7EPczK28tW6ln0MRUVVNfd+vIYZ760mrXcsn9w8UeuvAsQJA7tw6jHdePqbzezeX9rqy0t3aX1WINOkpZpkxhmDGJrYmTveW8WuwpY5AO09UMZlLyzmtYyt/P74Psy5egxxkWEtMm/lG/4y5RgqqwwPt0G/hBlZbrp3DiclvmOrL0u1PU1aqkk6hATz1KWplFZUc/s7K6luZlc9K3Os+qvVuQU8eclI7jpb668CUe/4SK45vg8frMjlp1Z8Zlt1tWGRy80Erc8KWHp0UE3Wr0sUs6YOId3l5rnvj/75Se8sy+Gi5zIICbbqr84dqfVXgezGk/vTtVMH7pu3ttknO/XZuOcA7qJyfRRJANOkpY7KRWk9OXt4Dx79cgMrcwqaNG15ZTV/+WgNd7y3mtEpsXxy00SGJmr9VaCL6mD1S7hqeyHv/dQ6j77JcOnzswKdJi11VESE//v1cLp1DufWuSs4WFbp1XR7DpRy2YuLeH3RVv5wQl9eu2oMsVp/1W5MG5nEqF4xPPKfDa3yhOx0l5tecR1JjtX6rEClSUsdteiOoTxxyUhy8ou55+PGn3a8Yts+zvnHD2TmFvLUpanMPOsYrb9qZ4KChFlTh+IuKuMf32xu0XlXVRsWZbm1qXuA0yOGapbRKXHcPGkAH/yUy8crc+sd7+2l27j4uUWEhQTxwfW/YuqIxDaMUvmSY5NjuPC4ZF75cQtZew+22HzX7ijkQGmlFg0GuEaTloiEi8gSEVklImtF5D57+KsiskVEVtp/I+3hIiJPichmEVktIqNaeR2Uw26e1J+03rHc9eEatrmLD/usvLKauz7M5M73MxnbN455N07UR0UoZkweTHhIMPd/uq7F5nmoPksf+hjQQrwYpwyYZIw5KCKhwA8i8oX92QxjzHu1xj8TGGD/jQWesf+rABUSHMQTl4zkzCcX8tuXFlFZbdhZUEq3zuGEhwaR7S7mDyf2Zcbpg7Q4UAFWDyu3nDKABz9fz7c/7+HkwV2bPc90l5v+XaPo2jm8BSJUvqrRI4ix1FzDh9p/DbVXPReYY0+3CIgRkR7ND1X5suTYjpyXmsS2/BJ2FJRigF37S8l2F3PF+N7MPFPrr9ThrpyQQt8ukdz/6TrKK6ubNa+KqmqWZudrfVY74NVRRESCRWQlsAf4yhiz2P7oQbsI8HERqemCOwnwfM72dntY7XleKyLLRGTZ3r17j34NlM+oryfv/7ZxD9/KP4SFBPGXKUPIyivi1fTm9Uu4ensBxeVVWjTYDniVtIwxVcaYkUAyMEZEhgEzgcHAaCAOuLMpCzbGPG+MSTPGpHXp0qVpUSuftKOgpEnDlTp5UFcmDe7KU//dzJ4DR98tWPpmqz5rnCatgNek8hpjTAHwLXCGMWanXQRYBrwCjLFHywV6ekyWbA9TAS4xJqJJw5UC+MuUIZRVVjH7PxuOeh7pLjdDenTWe/7aAW9aD3YRkRj7dQRwGvBzTT2VWB18TQNqbtSZB1xhtyIcBxQaY3a2QuzKx8yYPIiI0ODDhkWEBjNj8iCHIlL+oE9CJFf/qg/vLt/Oqib2rgJQWlHF8m37tKl7O+HNlVYP4FsRWQ0sxarT+hR4U0QygUwgAXjAHv9zIAvYDLwA3NDiUSufNC01iYfOG05STAQCJMVE8NB5w/WZWKpRN03qT0JUB2Z90vR+CX/ato/yympthNFONNrk3RizGkitY/ikesY3wI3ND035o2mpSZqkVJN1Cg/lzjMGMeO91Xy0MpfzRiV7PW2Gy01wkDCmT1wrRqh8hbZBVkr5hPNHJTOiZwx/++Jnr/uyBKs+a1hSNJ3CQ1sxOuUrNGkppXxCUJAw65wh7DlQxj+/9a5fwqKySlblFGjRYDuiSUsp5TNSe8Vy3qgkXlq4hey8okbHX5qdT2W10aTVjmjSUkr5lD+fMZjQYOGBzxrvlzDD5SY0WEjrrfVZ7YUmLaWUT+naOZybJg3g6/V7+G5jw73lZGS5Se0ZS0RYcIPjqcChSUsp5XOunphCSnxH/vrJWiqq6u6XsLC4gjW5hXp/VjujSUsp5XM6hARz99lDcO0t4rX07DrHWbzFTbVB67PaGU1aSimfdMoxXTlhYBee/HoTeQfLjvg8I8tNh5AgRvaKafvglGM0aSmlfJKIcM+UIZRUVPH3+Uf2S5jhcjM6JY4OIVqf1Z5o0lJK+az+XaOYPiGFt5flkLm98NDwvINl/LzrgNZntUOatJRSPu2WUwcQHxnGfZ+sxeolDhZlWY8i0aTV/mjSUkr5tM7hocyYPIhlW/cxb9UOwCoajOoQwrFJ0Q5Hp9qaJi2llM+78LieDE+K5qHPf6a4vJIMl5sxfeIICdZDWHujW1wp5fOCgoRZU4ewa38paQ98TVZeEcuy8/lohT5ftr3RpKWU8gs5+SUEi1BcXgXA/tJKZn6QqYmrndGkpZTyC7Pnb6DKHP6AyJKKKmbX0RxeBS5NWkopv7CjoKRJw1Vg0qSllPILiTERTRquApMmLaWUX5gxeRARoYf3fhERGsyMyYMcikg5IcTpAJRSyhvTUpMAq25rR0EJiTERzJg86NBw1T5o0lJK+Y1pqUmapNo5LR5USinlNzRpKaWU8huatJRSSvkNTVpKKaX8hiYtpZRSfkNMrW5RHAlCZC+w9SgnTwDyWjAcJ+m6+KZAWZdAWQ/QdanR2xjTpSWD8XU+kbSaQ0SWGWPSnI6jJei6+KZAWZdAWQ/QdWnPtHhQKaWU39CkpZRSym8EQtJ63ukAWpCui28KlHUJlPUAXZd2y+/rtJRSSrUfgXClpZRSqp3QpKWUUspvaNJSR0VEYkTkBqfjcJqInCQinzodRw3P7eJrsTWHiNwiIutF5M1aw9NE5Cmn4moNIjJdRBKdjsNXadJSRysGaPdJywfF0MTtIiLBjY/luBuA04wxl9UMEJEQY8wyY8wtDsbVGqYDmrTq4fNJS0QiReQzEVklImtE5GIRyRaRBPvzNBFZYL+eJSIvi8gCEckSEZ/YmUUkRUR+FpFXRWSjiLwpIqeKyI8isklExohInIh8JCKrRWSRiBxrT+uT6wT8DegnIitFZKnnGb2IPC0i0+3Xx4nIdyKyXETmi0gPpwKuTURm1HyfIvK4iHxjv55kb6PTRSRDRH4SkXdFJMr+/Ax7e/4EnOfgKtTl0HYBZgNRIvKeHe+bIiIA9m/oYXsdLnQw3kaJyLNAX+ALESkUkddF5EfgdX+5mqznOHaP/dtZIyLPi+UCIA140/5tRTgdu88xxvj0H3A+8ILH+2ggG0iw36cBC+zXs4B0oANW1yhuINQH1iEFqASGY50oLAdeBgQ4F/gI+Adwrz3+JGClH6zTGvv1ScCnHp89jXW2GGrH3sUefjHwstOxe8Q5DnjXfr0QWGLHfC9wJ/A9EGl/fidwDxAO5AAD7O33jue6O/1Xx3YpBJLt/S4DmGh/lg3c4XS8TVivbHv/n2X/fiLq2vd89a+e41icx/vXgXPs1wuANKdj9tU/n7/SAjKB0+yzwuONMYWNjP+ZMabMGJMH7AG6tX6IXtlijMk0xlQDa4H/GmsPzcQ60EzE2nExxnwDxItIZ3taX12nxgwChgFf2Wf+d2MdQH3FcuA4+3suwzqopwHHAyXAEOBHO/Yrgd7AYKxtucnefm84EXgTLDHGbLf3u5VY+1qNtx2JqPnmGWNKnA6iieo6jp0sIotFJBPrRHWosyH6hxCnA2iMMWajiIwCzgIeEJH/Yl211CTc8FqTlHm8rsJ31tEzrmqP99VYMVZ4Oa0vrVMNz+0Bv2wTAdYaY8a3fUiNM8ZUiMgWrKvCdGA1cDLQH9gCfGWMudRzGhEZ2cZhNldD+05RG8fSUvwu7nqOYzdiXVHliMgsjjyWqTr4/JWW3Yqm2BjzBlYZ/SisooLj7FHOdyi0lrYQuAysVl9AnjFmv5MBNeIA0Ml+vRUYIiIdRCQGOMUevgHoIiLjAUQkVER87WxyIfAnrKLAhcB1wApgEfArEekPh+okBgI/Ayki0s+e/tIjZ+koz+2ifEQ9xzGAPLuu9AKP0XUbNsDXztjrMhyYLSLVWFcj1wMRwEsicj9W+W8gmAW8LCKrgWKs4iifZYxx2w1J1gBfYNXtrMG6Qllhj1NuVyw/JSLRWPvbE1jFo75iIXAXkGGMKRKRUmChMWav3Zjk3yLSwR73bvuM+VrgMxEptqf3mQNMre1SAux2OiYF1H0cm4b1m9kFLPUY91XgWREpAcb7YVFoq9JunJRSSvkNny8eVEoppWpo0lJKKeU3NGkppZTyG5q0lFJK+Q1NWkoppfyGJi2llFJ+Q5OWUkopv/H/hP9QTEIOnXgAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize scores\n",
    "summarize_scores('cnn', score, scores)\n",
    "# plot scores\n",
    "days = ['sun', 'mon', 'tue', 'wed', 'thr', 'fri', 'sat'] \n",
    "plt.plot(days, scores, marker='o', label='cnn') \n",
    "plt.title(\"Line Plot of RMSE per Day for Univariate CNN with 7-day Inputs. \\n\", loc='left')\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot shows that perhaps Tuesdays and Fridays are easier days to forecast than the other days and that perhaps Saturday at the end of the standard week is the hardest day to forecast."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-channel CNN Model\n",
    "\n",
    "In this section, we will update the CNN developed in the previous section to use each of the eight time series variables to predict the next standard week of daily total power consumption. We will do this by providing each one-dimensional time series to the model as a separate channel of input. The CNN will then use a separate kernel and read each input sequence onto a separate set of filter maps, essentially learning features from each input time series variable. This is helpful for those problems where the output sequence is some function of the observations at prior time steps from multiple different features, not just (or including) the feature being forecasted.\n",
    "\n",
    "It is unclear whether this is the case in the power consumption problem, but we can explore it nonetheless. First, we must update the preparation of the training data to include all of the eight features, not just the one total daily power consumed. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**change supervised data transform function**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 260,
   "metadata": {},
   "outputs": [],
   "source": [
    "def to_supervised(train, n_input, n_out=7):\n",
    "    \"\"\"\n",
    "    \n",
    "    input - a list of weeks (history) and the number of time steps to use as inputs outputs \n",
    "    returns - the data in the overlapping moving window format.\n",
    "    \n",
    "    \"\"\"\n",
    "    #flatten data\n",
    "    data = train.reshape(train.shape[0]*train.shape[1], train.shape[2])\n",
    "    X, y = [], []\n",
    "    in_start = 0\n",
    "\n",
    "    for _ in range(len(data)):\n",
    "\n",
    "        #define the end of the input sequence\n",
    "        in_end = in_start + n_input\n",
    "        out_end = in_end + n_out\n",
    "\n",
    "        #ensure we have all the data for that window\n",
    "        if out_end < len(data):\n",
    "            x_input = data[in_start:in_end, :]\n",
    "            #x_input = x_input.view((len(x_input), 1))\n",
    "            X.append(x_input)\n",
    "            y.append(data[in_end:out_end, 0])\n",
    "        # move along one time step\n",
    "        in_start += 1\n",
    "    return torch.tensor(X), torch.tensor(y)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1092, 14, 7]), torch.Size([1092, 7]))"
      ]
     },
     "execution_count": 296,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X, y = to_supervised(train, 14)\n",
    "X.shape, y.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use 14 days of prior observations across seven of the input variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 297,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_input = 14"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the model used in the previous section does not perform well on this new framing of the problem. The increase in the amount of data requires a larger and more sophisticated model that is trained for longer. \n",
    "\n",
    "**New Model Architecture**\n",
    "\n",
    "Two convolutional layers with `32` filter maps followed by pooling, then another convolutional layer with `16` feature maps and pooling. The fully connected layer that interprets the features is increased to `100` nodes and final layer contains `n_out` nodes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 325,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultiChannelCNNModel(nn.Module):\n",
    "\n",
    "    def __init__(self, n_feature, n_out) -> None:\n",
    "        super().__init__()\n",
    "        #define convnets\n",
    "        self.conv_layer1 = nn.Sequential(\n",
    "            nn.Conv1d(in_channels=n_feature, out_channels=32, kernel_size=2),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool1d(kernel_size=2),\n",
    "        )\n",
    "        self.conv_layer2 = nn.Sequential(\n",
    "            nn.Conv1d(in_channels=32, out_channels=16, kernel_size=2),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool1d(kernel_size=2),\n",
    "        )\n",
    "        self.flt_layer = nn.Flatten()\n",
    "        self.ln_layer = nn.Sequential(\n",
    "            nn.Linear(in_features=32, out_features=100),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(in_features=100, out_features=n_out)\n",
    "        )\n",
    "\n",
    "    def forward(self, X):\n",
    "\n",
    "        #shape of X => (B, T, C) => (B, C, T)\n",
    "        B, T, C = X.shape\n",
    "        # print(B, T, C)\n",
    "        X = X.view(B, C, T)\n",
    "        X = self.conv_layer1(X)\n",
    "        X = self.conv_layer2(X)\n",
    "        X = self.flt_layer(X)\n",
    "        X = self.ln_layer(X)\n",
    "\n",
    "        return X\n",
    "    \n",
    "n_feature = 7\n",
    "n_out = 7     \n",
    "multi_channel_model = MultiChannelCNNModel(n_feature=n_feature, n_out=n_out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method Module.parameters of MultiChannelCNNModel(\n",
       "  (conv_layer1): Sequential(\n",
       "    (0): Conv1d(7, 32, kernel_size=(2,), stride=(1,))\n",
       "    (1): ReLU()\n",
       "    (2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  )\n",
       "  (conv_layer2): Sequential(\n",
       "    (0): Conv1d(32, 16, kernel_size=(2,), stride=(1,))\n",
       "    (1): ReLU()\n",
       "    (2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  )\n",
       "  (flt_layer): Flatten(start_dim=1, end_dim=-1)\n",
       "  (ln_layer): Sequential(\n",
       "    (0): Linear(in_features=32, out_features=100, bias=True)\n",
       "    (1): ReLU()\n",
       "    (2): Linear(in_features=100, out_features=7, bias=True)\n",
       "  )\n",
       ")>"
      ]
     },
     "execution_count": 326,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multi_channel_model.parameters"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Define optimizer and loss function**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 327,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.Adam(multi_channel_model.parameters(), lr=1e-3)\n",
    "\n",
    "criterion = torch.nn.MSELoss()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Models Training**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 328,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "390870464.0\n",
      "237132912.0\n",
      "130711816.0\n",
      "61832276.0\n",
      "23469858.0\n",
      "5587388.0\n",
      "3823495.25\n",
      "10836916.0\n",
      "18979518.0\n",
      "24608476.0\n",
      "26469838.0\n",
      "26073732.0\n",
      "24360348.0\n",
      "21469828.0\n",
      "17906974.0\n",
      "14261224.0\n",
      "10877793.0\n",
      "7866655.5\n",
      "5328859.0\n",
      "3331293.25\n",
      "1994532.0\n",
      "1294689.75\n",
      "1166760.875\n",
      "1464935.375\n",
      "1958473.0\n",
      "2409328.5\n",
      "2755325.75\n",
      "2931585.5\n",
      "2935711.0\n",
      "2793035.75\n",
      "2536169.25\n",
      "2202276.0\n",
      "1829042.375\n",
      "1453139.0\n",
      "1109390.375\n",
      "828338.625\n",
      "631955.0625\n",
      "528976.75\n",
      "513120.84375\n",
      "560076.125\n",
      "631488.25\n",
      "690232.25\n",
      "718660.875\n",
      "717517.1875\n",
      "697101.4375\n",
      "668838.375\n",
      "639529.0625\n",
      "610258.1875\n",
      "578637.1875\n",
      "542385.875\n",
      "502159.96875\n",
      "462073.71875\n",
      "428080.125\n",
      "405193.375\n",
      "395398.125\n",
      "396877.5625\n",
      "404921.34375\n",
      "414070.0\n",
      "420248.34375\n",
      "421961.46875\n",
      "420100.5\n",
      "416702.3125\n",
      "413488.96875\n",
      "410961.03125\n",
      "408400.5625\n",
      "404600.59375\n",
      "398803.9375\n",
      "391275.75\n",
      "383255.78125\n",
      "376368.4375\n",
      "371863.4375\n",
      "370089.0625\n",
      "370460.125\n",
      "371860.4375\n",
      "373236.34375\n",
      "374024.8125\n",
      "374233.59375\n",
      "374195.75\n",
      "374207.25\n",
      "374286.0625\n",
      "374179.4375\n",
      "373563.0\n",
      "372285.3125\n",
      "370493.875\n",
      "368577.34375\n",
      "366972.1875\n",
      "365963.0625\n",
      "365576.96875\n",
      "365628.8125\n",
      "365853.6875\n",
      "366051.5625\n",
      "366154.25\n",
      "366195.9375\n",
      "366232.5625\n",
      "366273.46875\n",
      "366267.53125\n",
      "366143.40625\n",
      "365865.21875\n",
      "365465.53125\n",
      "365030.59375\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Training Loss')"
      ]
     },
     "execution_count": 328,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAERCAYAAAB2CKBkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAi3ElEQVR4nO3de5wddX3/8dd7z57ds9ndEMhuICSBIOAFlYtGBEGhWB6NSKGtUPHXKiI2P1sv0Nqq2D5Q6e/X2qoV0BYbgRLQIhapjYgichH4yW0DIVyVEEECgWwC5L73z++PmU3ObvZyNtnZk915Px+ex56Z+c7MZ5i4n/1+vzPfryICMzPLr5pqB2BmZtXlRGBmlnNOBGZmOedEYGaWc04EZmY550RgZpZzkzIRSLpS0lpJj1ZQ9gBJt0t6SNIKSadMRIxmZpPFpEwEwFXAwgrL/h3w/Yg4CjgL+LesgjIzm4wmZSKIiDuBl8vXSTpY0k8lLZN0l6TX9xcHpqff9wJemMBQzcz2eLXVDmAcLQY+FhFPSXo7yV/+JwFfBH4m6ZNAI/C71QvRzGzPMyUSgaQm4B3Af0nqX12f/vwAcFVEfE3SscA1kt4UEX1VCNXMbI8zJRIBSRPXqxFx5BDbziXtT4iIeySVgBZg7cSFZ2a255qUfQSDRcRG4DeSzgRQ4oh082+Bd6fr3wCUgPaqBGpmtgfSZBx9VNK1wIkkf9m/BHwBuA24DJgNFIHvRcRFkg4Dvg00kXQcfyYiflaNuM3M9kSTMhGYmdn4mRJNQ2ZmtusmXWdxS0tLzJ8/v9phmJlNKsuWLVsXEa1DbZt0iWD+/Pm0tbVVOwwzs0lF0rPDbcu8aUhSIR3n58YhttVLuk7SSkn3SZqfdTxmZjbQRPQRnAc8Mcy2c4FXIuIQ4OvAP01APGZmVibTRCBpLvBe4PJhipwOLEm/Xw+8W2WvBpuZWfayrhFcDHwGGG44hznAcwAR0QNsAGZmHJOZmZXJLBFIOhVYGxHLxuFYiyS1SWprb/dLwWZm4ynLGsFxwGmSngG+B5wk6TuDyjwPzAOQVEsyTPT6wQeKiMURsSAiFrS2Dvn0k5mZ7aLMEkFEXBARcyNiPsmEMLdFxJ8OKrYUODv9fkZaxq86m5lNoAl/s1jSRZJOSxevAGZKWgn8FfC5rM77qxc38bWf/YqXt3RldQozs0lpQl4oi4g7gDvS7xeWre8AzpyIGFa1b+Ybt63kvYfPZp/Guok4pZnZpJCbsYZKdQUAtnX1VjkSM7M9S24SwbRimgi6nQjMzMrlJhE0pDWCDicCM7MB8pMI0hrBVjcNmZkNkJtEUCq6j8DMbCi5SQRuGjIzG1p+EoE7i83MhpSbRLCjaWi48e/MzPIpN4mgUCPqamtcIzAzGyQ3iQCS5iH3EZiZDZS7ROCnhszMBspVIphWV3DTkJnZILlKBKViwS+UmZkNkqtE0FDnPgIzs8HylQiKbhoyMxssV4mg5M5iM7Od5CoRuGnIzGxnmSUCSSVJ90t6WNJjkr40RJkPS2qXtDz9fDSreAAain6hzMxssCynquwEToqIzZKKwN2SfhIR9w4qd11EfCLDOLZzH4GZ2c4ySwQREcDmdLGYfiKr81WiVOc+AjOzwTLtI5BUkLQcWAvcEhH3DVHsfZJWSLpe0rxhjrNIUpuktvb29l2OZ1qxls6ePnr7qpqPzMz2KJkmgojojYgjgbnA0ZLeNKjIj4D5EXE4cAuwZJjjLI6IBRGxoLW1dZfjaahLLtcdxmZmO0zIU0MR8SpwO7Bw0Pr1EdGZLl4OvDXLODwngZnZzrJ8aqhV0oz0ewNwMvDkoDKzyxZPA57IKh7wdJVmZkPJ8qmh2cASSQWShPP9iLhR0kVAW0QsBT4l6TSgB3gZ+HCG8Xi6SjOzIWT51NAK4Kgh1l9Y9v0C4IKsYhjMTUNmZjvL15vFbhoyM9tJrhJBqc41AjOzwXKVCPprBO4jMDPbIZeJwJPTmJntkKtEMM1NQ2ZmO8lVItjeR+AagZnZdrlKBO4jMDPbWa4SQbFQQ22N3DRkZlYmV4kA0jkJuvqqHYaZ2R4jd4mgVOfJaczMyuUuETQUPW+xmVm5XCaCrV091Q7DzGyPkbtEkDQNuY/AzKxf7hJBQ7GGDr9HYGa2Xe4SwbS6WncWm5mVyV0iaCj6qSEzs3JZTlVZknS/pIclPSbpS0OUqZd0naSVku6TND+rePqVigUPMWFmVibLGkEncFJEHAEcCSyUdMygMucCr0TEIcDXgX/KMB4AGupq/PiomVmZzBJBJDani8X0E4OKnQ4sSb9fD7xbkrKKCdw0ZGY2WKZ9BJIKkpYDa4FbIuK+QUXmAM8BREQPsAGYOcRxFklqk9TW3t6+WzH1J4KIwTnJzCyfMk0EEdEbEUcCc4GjJb1pF4+zOCIWRMSC1tbW3YqpVFcgAjp7/C6BmRlM0FNDEfEqcDuwcNCm54F5AJJqgb2A9VnG4gnszcwGyvKpoVZJM9LvDcDJwJODii0Fzk6/nwHcFhm32WxPBO4nMDMDoDbDY88GlkgqkCSc70fEjZIuAtoiYilwBXCNpJXAy8BZGcYDQIOnqzQzGyCzRBARK4Cjhlh/Ydn3DuDMrGIYipuGzMwGyt+bxXWertLMrFz+EoH7CMzMBshdIii5acjMbIDcJQJ3FpuZDZS/ROAagZnZAPlNBK4RmJkBeUwEbhoyMxsgd4mgvrYGCU9XaWaWyl0ikOShqM3MyuQuEYDnJDAzK5fLRJBMV+lhqM3MIKeJoKGuwLbunmqHYWa2R8hnIvAE9mZm2+U3EbiPwMwMyGkiKNUV2NbtPgIzM8hpImgo1vg9AjOz1JgSgaQaSdMrLDtP0u2SHpf0mKTzhihzoqQNkpannwuHOtZ4c9OQmdkOoyYCSf8pabqkRuBR4HFJf1PBsXuAT0fEYcAxwMclHTZEubsi4sj0c9GYot9FDXW1TgRmZqlKagSHRcRG4A+AnwAHAR8cbaeIWBMRD6bfNwFPAHN2PdTx01AsuGnIzCxVSSIoSiqSJIKlEdENxFhOImk+yfzF9w2x+VhJD0v6iaQ3juW4u6qhroat3b1EjOkyzMympEoSwb8DzwCNwJ2SDgQ2VnoCSU3AD4Dz05pFuQeBAyPiCOAbwA+HOcYiSW2S2trb2ys99bAaigV6+4LuXicCM7NRE0FEXBoRcyLilEg8C/xOJQdPaxI/AL4bETcMceyNEbE5/X4TSe2jZYhyiyNiQUQsaG1treTUIyp5TgIzs+0q6Sw+L+0slqQrJD0InFTBfgKuAJ6IiH8Zpsx+aTkkHZ3Gs35MV7AL+uck6HAiMDOjtoIyH4mISyT9HrA3SUfxNcDPRtnvuLTsI5KWp+s+DxwAEBHfAs4A/lxSD7ANOCsmoOHe01Wame1QSSJQ+vMU4JqIeKz/r/iRRMTdZfsOV+abwDcriGFcebpKM7MdKuksXibpZySJ4GZJzcCkHp+h5Okqzcy2q6RGcC5wJLAqIrZKmgmck2lUGWusSy57S6eHojYzGzURRESfpLnA/0pbhH4RET/KPLIMNZeSy97c4URgZlbJU0NfBs4DHk8/n5L0D1kHlqWm+iQRbHIiMDOrqGnoFODIiOgDkLQEeIjkCaBJaXqpCMDGju4qR2JmVn2Vjj46o+z7XhnEMaGa+puG3EdgZlZRjeAfgYck3U7yOOi7gM9lGlXGCjViWl3BTUNmZlTWWXytpDuAt6WrPgscmGVQE6G5VOvOYjMzKqsREBFrgKX9y5LuJ31DeLJqqq9lU6f7CMzMdnWqylHfLN7TNZeKbhoyM2PXE8GkH7+5uVTrRGBmxghNQ5J+xNC/8AXMzCyiCdJcquWFV7dVOwwzs6obqY/gq7u4bVJori/68VEzM0ZIBBHxi4kMZKI1uWnIzAzY9T6CSa+5VMvWrl56eif1QKpmZrstx4kgGWZiS6eHojazfMtvIkgHnvN4Q2aWd6O+UDbM00MbgDbg3yOiY5j95gFXA/um+y+OiEsGlRFwCcnAdluBD0fEg2O9iF3RPxS1+wnMLO8qqRGsAjYD304/G4FNwGvT5eH0AJ+OiMOAY4CPSzpsUJn3AIemn0XAZWOKfjf0Nw35ySEzy7tKhph4R0S8rWz5R5IeiIi3SXpsuJ3SYSnWpN83SXoCmEMyp0G/04Gr0wnr75U0Q9LsdN9MNW2vEbhpyMzyrZIaQZOk7eMKpd+b0sWuSk4iaT5wFHDfoE1zgOfKllen6wbvv0hSm6S29vb2Sk45qmYPRW1mBlRWI/g0cLekp0neKj4I+AtJjcCS0XaW1AT8ADg/IjbuSpARsRhYDLBgwYJxGd5iR2exE4GZ5Vslw1DfJOlQ4PXpql+VdRBfPNK+kookSeC7EXHDEEWeB+aVLc9N12Wuv4/ATUNmlneVPj76VuCNwBHAH0v60Gg7pE8EXQE8ERH/MkyxpcCHlDgG2DAR/QMApWINtTXynARmlnuVPD56DXAwsBzof/sqSB4NHclxwAeBRyQtT9d9nnQeg4j4FnATyaOjK0keHz1nTNHvBkkeZsLMjMr6CBYAh6VP9lQsIu5mlHkL0mN+fCzHHU/NpVp3FptZ7lXSNPQosF/WgVRDc33RfQRmlnuV1AhagMfT6Sk7+1dGxGmZRTVBmkq1fmrIzHKvkkTwxayDqJbppVpeeHXIETLMzHKjksdHp+y8BJ7A3sxs5Kkq746I4yVtYuCgcyLp552eeXQZay4V/fiomeXeSDOUHZ/+bJ64cCZW/wT2EUHy2oOZWf5U0keApALJcNLby0fEb7MKaqI0lWrp6Qs6uvtoqCtUOxwzs6qo5IWyTwJfAF4C+ud1DODwDOOaENuHmejsdiIws9yqpEZwHvC6iFifdTATbXrZ5DSzpmwDmJnZyCp5oew5khnJppymes9SZmZWSY1gFXCHpB8z8IWy4QaSmzS2z1LmRGBmOVZJIvht+qlLP1PGjhqB3yUws/yq5IWyL01EINWwfQJ7DzxnZjk20gtlF0fE+ZJ+xMAXyoCpMdbQ9O2T0zgRmFl+jVQjuCb9+dWJCKQaGuuTR0bdNGRmeTbSm8XL0p9Tdqyh2kIN0+oK7iw2s1wb9fFRSYdKul7S45JW9X8q2O9KSWslPTrM9hMlbZC0PP1cuCsXsLuaPUuZmeVcJe8R/AdwGdAD/A7JFJXfqWC/q4CFo5S5KyKOTD8XVXDMcddU71nKzCzfKkkEDRFxK6CIeDYivgi8d7SdIuJO4OXdjC9zzaUiG91HYGY5Vkki6JRUAzwl6ROS/hBoGqfzHyvpYUk/kfTG4QpJWiSpTVJbe3v7OJ064aYhM8u7ShLBecA04FPAW4E/Bc4eh3M/CBwYEUcA3wB+OFzBiFgcEQsiYkFra+s4nHoHT2BvZnk3YiJIh59+f0RsjojVEXFORLwvIu7d3RNHxMaI2Jx+vwkoSmrZ3eOOlSewN7O8GzYRSKqNiF7g+CxOLGk/pbPBSDo6jWXCRzhtctOQmeXcSC+U3Q+8BXhI0lLgv4At/Rsj4oaRDizpWuBEoEXSapI5DYrpvt8CzgD+XFIPsA04KyJ2eoM5a82lWrZ29dLbFxRqPEuZmeVPJYPOlUj+Uj+JZKgJpT9HTAQR8YFRtn8T+GZlYWanfATSvaYVqxyNmdnEGykRzJL0V8Cj7EgA/Sb8L/esNPePQNrZ7URgZrk0UiIokDwmOlR7ydRJBCVPTmNm+TZSIlhTrbd9J1KTE4GZ5dxIj4/moud0+wT2foTUzHJqpETw7gmLoor2TvsFXtnqRGBm+TRsIoiIPX6coPEws6kegPWbO0cpaWY2NVUyxMSU1lhXoFSsYZ0TgZnlVO4TgSRmNtazfnNXtUMxM6uK3CcCgJbmetpdIzCznHIiAFqb6ljnGoGZ5ZQTAaRNQ64RmFk+OREALc11rN/SRV/flHlh2sysYk4EJDWC3r5gwza/S2Bm+eNEQNJZDPgRUjPLJScCoKWpDsAdxmaWS04EQEuTawRmll+ZJQJJV0paK+nRYbZL0qWSVkpaIektWcUyGicCM8uzLGsEVwELR9j+HuDQ9LMIuCzDWEY0o6FIoUZ+u9jMcimzRBARdwIjDVx3OnB1JO4FZkianVU8I6mpEfs01rlGYGa5VM0+gjnAc2XLq9N1O5G0SFKbpLb29vZMgpnZ6LeLzSyfJkVncUQsjogFEbGgtbU1k3O0Nte7RmBmuVTNRPA8MK9seW66ripamupZv8WJwMzyp5qJYCnwofTpoWOADRGxplrBzGysY90mNw2ZWf6MNHn9bpF0LXAi0CJpNfAFoAgQEd8CbgJOAVYCW4FzsoqlEi3N9Wzr7mVLZw+N9Zn9ZzEz2+Nk9hsvIj4wyvYAPp7V+cdqZmPydvH6zV1OBGaWK5Ois3gi9I835AlqzCxvnAhSLY2exN7M8smJINXS7IHnzCyfnAhSM10jMLOcciJI1dXWML1U65fKzCx3nAjKtDTXs26Lm4bMLF+cCMq0NNazbpNrBGaWL04EZVqaPQKpmeWPE0GZmY31rHfTkJnljBNBmZamel7d2k13b1+1QzEzmzBOBGX63yV42bUCM8sRJ4Iy/e8StLvD2MxyxImgTGtaI3A/gZnliRNBGdcIzCyPnAjKzJ5RolAjnl2/pdqhmJlNGCeCMvW1BQ7YZxpPt2+udihmZhPGiWCQg1ubWLnWicDM8iPTRCBpoaRfSVop6XNDbP+wpHZJy9PPR7OMpxIHz2rkmXVb6fG7BGaWE1nOWVwA/hU4GVgNPCBpaUQ8PqjodRHxiaziGKtDWpvo6u3juVe2cVBLY7XDMTPLXJY1gqOBlRGxKiK6gO8Bp2d4vnFx8KwmAJ5285CZ5USWiWAO8FzZ8up03WDvk7RC0vWS5g11IEmLJLVJamtvb88i1u0Obk0SwUp3GJtZTlS7s/hHwPyIOBy4BVgyVKGIWBwRCyJiQWtra6YB7dVQpLW53jUCM8uNLBPB80D5X/hz03XbRcT6iOh/e+ty4K0ZxlOxQ1qbXCMws9zIMhE8ABwq6SBJdcBZwNLyApJmly2eBjyRYTwVO3hWI0+v3UxEVDsUM7PMZfbUUET0SPoEcDNQAK6MiMckXQS0RcRS4FOSTgN6gJeBD2cVz1gc0trExo4e2jd3Mqu5VO1wzMwylVkiAIiIm4CbBq27sOz7BcAFWcawKw6Z1QzA02u3OBGY2ZRX7c7iPdLBs5L3B9xPYGZ54EQwhP2ml2isK/jJITPLBSeCIUji4FlNHnzOzHLBiWAYh7Q2uUZgZrngRDCMg2c18cKGDrZ09uz2sSKCzp7ecYjKzGz8ZfrU0GTWP9TEqvYtvHnuXmPev7u3j3tXreeWx1/i54+/xAsbOqivrWF6Q5HZe5U4/pAWTnhtK285cG+KBedjM6seJ4JhHLL9yaFNY04EL2/p4pyrHuDh516lVKzhnYe2ctbRB7Cls4cN27pZ1b6Ff79zFf92x9PsPa3I2e+Yz4ffMZ8Z0+qyuBQzsxE5EQzjwJmNTKsrcP9vXuYPj5pb8X5rNmzjg1fcz29f3spXzjicUw/fn4a6wk7lNnZ088uV67l+2Wou/vlTfPvOVfzJMQfy8RMPYa9pxfG8FDOzEblNYhjFQg2/+4Z9+cmjL9Jd4SQ1v1m3hTMuu4cXN3Rw9UeO5swF84ZMAgDTS0UWvmk/Lj97ATef/y5OPmxfLr9rFSd+9Xb+877f0tvn4S3MbGI4EYzg1MNn8+rWbv7fynWjlt2wrZsPXnEfHd29fG/RMRzzmpkVn+d1+zVz8VlHceMn38mh+zbz+f9+hNO+eTfLnn1ld8I3M6uIE8EITnhdK831tdy4Ys2I5SKCz16/ghc3dPDtsxfwpjlj71wGOGz/6Vy36Bgu/cBRrN/cxfsu+yWfuf5h1m/uHH1nM7Nd5EQwgvraAie/cV9ufuzFER//vPqeZ/npYy/ymYWv4y0H7L1b55TEaUfsz62fPoH/fcJruOHB5/mdr97B5XetoqvH8yib2fhzIhjF7x++P5s6erjr10M3Dz36/Ab+74+f4KTXz+Kjx79m3M7bWF/LBe95Az89/50cMW8G/+fHT3Dy13/BTY+s8fDYZjaunAhGcdwhLezVUOTGFS/stK19Uycf+84yZjbV8bUzj6CmRuN+/kNmNXPNuW9nyUeOplRb4C+++yC//827ueXxl5wQzGxcOBGMoq62hoVv3I9bHn+Jju4dzUPbunr56NVtrNvcybf+9K3s3ZjtOwAnvLaVH3/qeP75jMPZ1NHDn13dxqnfuJsfPvS831o2s93iRFCBU4+YzZauXpb88hm2dPbQ2xecf91DrFj9KpeedRRHzJsxIXHUFmr44wXzuPWvTuBrZx7Btq5ezr9uOcd9+Ta+cvOT/GbdlgmJw8ymFmXZvCBpIXAJyQxll0fElwdtrweuJpmreD3w/oh4ZqRjLliwINra2rIJeBg9vX2855K7eGrtZuoKNRzU0sivXtrEF37/MM457qAJjaVcX19w98p1XH3Ps9z25Ev0Bbx+v2be++bZvOu1rbxh9nTqascn1/f1BVu7e+ns7qWrt4/uniDY8W+nWKihWKihrlBDqS75KY1/U5mZ7RpJyyJiwZDbskoEkgrAr4GTgdUkcxh/ICIeLyvzF8DhEfExSWcBfxgR7x/puNVIBABdPX20PfMytz25lrueWsdJb5jFZxe+fsLjGM6aDdu46ZEX+ckja2hL3z+oq63hzXP24rX7NrP/XiX2n9HAjGlFioUaaguirw82d/awtauHjdu6eXlLF+u3dPHK1i5e2dLNK1u72LCtm80dPWzu6mEs/1QKNaKhWKCpvpamUi3NpVqaS0WaS7VMLxWZ3tD/s8j0Um1SLi3bWFfLtPoCDcUC9bUFigXtVlKJCCJguPAFSDhx2ZRWrURwLPDFiPi9dPkCgIj4x7IyN6dl7pFUC7wItMYIQVUrEUwmazd20PbsKzz47Cs89NyrPLNuC+u3dI26nwQzGors3VjHPtPqmDGtjhnTitt/iTfWFSgVC9TVJn/99/eNR0BPXx9dvUFXTx8d3b1s7epha1cvWzp72NzZw6aOHjZ29LBpWzcbO7rZuK2Hrgrf2K5RUuOorRG16Xkl0f9ruy+CvkhqLX0R9EbQ10fyM00CYyH1Jwcl5yJZ0f9dgpr0/P0JpKZ/nQbuV1OWYJJtO47Rf45y5f/0Y8D6sV1D//mG3VbR/jtKDSivIb+OKZFO1IMOu5Pc+/es9iMZ5Vfw/rfN46Pv3LWnE0dKBFmONTQHeK5seTXw9uHKpJPdbwBmAgOe1ZS0CFgEcMABB2QV75Qxa3qJU948m1PePHv7uo7uXtZs6GDjtu7kl3ZPUKPkMdWm+uQv9hnT6ihk8OTTcDq6e9nY0c2mjh42dyTJYnNnD9u6e9jcmTRDdaaJpaunj56+oDf99DdLRSS1j5r0//CFGm1fLtRAQUltovwXcLmkplBWY4jkyP3r+2JQmTSx9A1a1zegfH+Zgfun/0u2s+OXezD4F2rZ9wHrx+cXbSW/2Mp3H5iQhk5Uu/TbMut/arvxGzwG7azMg60sjpam+kzOMykGnYuIxcBiSGoEVQ5nUioVCxzU0ljtMAYoFZMaxqzmakdilm9ZPjX0PDCvbHluum7IMmnT0F4kncZmZjZBskwEDwCHSjpIUh1wFrB0UJmlwNnp9zOA20bqHzAzs/GXWdNQ2ub/CeBmksdHr4yIxyRdBLRFxFLgCuAaSSuBl0mShZmZTaBM+wgi4ibgpkHrLiz73gGcmWUMZmY2Mr9ZbGaWc04EZmY550RgZpZzTgRmZjmX6aBzWZDUDjy7i7u3MOit5ZzI43Xn8Zohn9edx2uGsV/3gRHROtSGSZcIdoektuHG2pjK8njdebxmyOd15/GaYXyv201DZmY550RgZpZzeUsEi6sdQJXk8brzeM2Qz+vO4zXDOF53rvoIzMxsZ3mrEZiZ2SBOBGZmOZebRCBpoaRfSVop6XPVjicLkuZJul3S45Iek3Reun4fSbdIeir9uXe1Y82CpIKkhyTdmC4fJOm+9J5flw6HPmVImiHpeklPSnpC0rF5uNeS/jL99/2opGsllabivZZ0paS1kh4tWzfk/VXi0vT6V0h6y1jOlYtEIKkA/CvwHuAw4AOSDqtuVJnoAT4dEYcBxwAfT6/zc8CtEXEocGu6PBWdBzxRtvxPwNcj4hDgFeDcqkSVnUuAn0bE64EjSK59St9rSXOATwELIuJNJEPcn8XUvNdXAQsHrRvu/r4HODT9LAIuG8uJcpEIgKOBlRGxKiK6gO8Bp1c5pnEXEWsi4sH0+yaSXwxzSK51SVpsCfAHVQkwQ5LmAu8FLk+XBZwEXJ8WmVLXLWkv4F0kc3oQEV0R8So5uNckw+c3pLMaTgPWMAXvdUTcSTJPS7nh7u/pwNWRuBeYIWk2FcpLIpgDPFe2vDpdN2VJmg8cBdwH7BsRa9JNLwL7ViuuDF0MfAboS5dnAq9GRE+6PNXu+UFAO/AfaXPY5ZIameL3OiKeB74K/JYkAWwAljG173W54e7vbv2Oy0siyBVJTcAPgPMjYmP5tnQq0Cn1zLCkU4G1EbGs2rFMoFrgLcBlEXEUsIVBzUBT9F7vTfLX70HA/kAjOzef5MJ43t+8JILngXlly3PTdVOOpCJJEvhuRNyQrn6pv5qY/lxbrfgychxwmqRnSJr9TiJpP5+RNh/A1Lvnq4HVEXFfunw9SWKY6vf6d4HfRER7RHQDN5Dc/6l8r8sNd39363dcXhLBA8Ch6ZMFdSSdS0urHNO4S9vFrwCeiIh/Kdu0FDg7/X428D8THVuWIuKCiJgbEfNJ7u1tEfEnwO3AGWmxKXXdEfEi8Jyk16Wr3g08zhS/1yRNQsdImpb+e++/7il7rwcZ7v4uBT6UPj10DLChrAlpdBGRiw9wCvBr4Gngb6sdT0bXeDxJVXEFsDz9nELSXn4r8BTwc2Cfasea4X+DE4Eb0++vAe4HVgL/BdRXO75xvtYjgbb0fv8Q2DsP9xr4EvAk8ChwDVA/Fe81cC1JP0g3SQ3w3OHuLyCSJyOfBh4heaqq4nN5iAkzs5zLS9OQmZkNw4nAzCznnAjMzHLOicDMLOecCMzMcs6JwKYUSSHpa2XLfy3pi+Nw3HpJP5e0XNL7B227StJv0m3LJf1yd8836Ph3SMrd5Ow2cWpHL2I2qXQCfyTpHyNi3Tge9yiAiDhymO1/ExHXD7PNbI/mGoFNNT0kc7n+5eANkuZLui0dr/1WSQcMUWYfST9My9wr6XBJs4DvAG9L/+I/uJJAJH1R0jWS7knHj/+zdL0kfSUdT/+R8hqGpM+m6x6W9OWyw50p6X5Jv5b0zrTsG9N1y9N4Dx3TfymzlGsENhX9K7BC0j8PWv8NYElELJH0EeBSdh6u+EvAQxHxB5JOIhna90hJHwX+OiJOHeacX5H0d+n3xyIZ4gLgcJK5IRqBhyT9GDiW5K3gI4AW4AFJd6brTgfeHhFbJe1TdvzaiDha0inAF0jG3PkYcElEfDcdOqVQ2X8es4GcCGzKiYiNkq4mmcBkW9mmY4E/Sr9fAwxOFJAM0/G+9Di3SZopaXoFpx2uaeh/ImIbsE3S7SRzYxwPXBsRvSSDiP0CeBtwAvAfEbE1PX/5WPT9AwguA+an3+8B/jadi+GGiHiqgjjNduKmIZuqLiYZm6WxynEMHsNlV8d06Ux/9pL+ARcR/wmcRpLsbkprMGZj5kRgU1L61/T3GThl4S9JRicF+BPgriF2vSvdhqQTgXUxaE6HMTpdyZy6M0kGxHsgPcf7lcyx3Eoy09j9wC3AOZKmpeffZ5hjkm5/DbAqIi4lGYXy8N2I03LMTUM2lX0N+ETZ8idJZvT6G5LZvc4ZYp8vAldKWgFsZceQv6Mp7yOApAkIkpFBbyfpC/j7iHhB0n+TNFM9TFJD+Ewkw0r/VNKRQJukLuAm4PMjnPOPgQ9K6iaZreofKozVbACPPmqWkfT9hc0R8dVqx2I2EjcNmZnlnGsEZmY55xqBmVnOORGYmeWcE4GZWc45EZiZ5ZwTgZlZzv1/JW/hTYiFi5sAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "epochs = 100\n",
    "\n",
    "losses = []\n",
    "\n",
    "for _ in range(epochs):\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    y_pred = multi_channel_model(X)\n",
    "    loss = criterion(y_pred, y)\n",
    "    print(loss.item())\n",
    "    losses.append(loss.item())\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "\n",
    "#plot the training losses\n",
    "\n",
    "plt.plot(losses)\n",
    "plt.xlabel(\"No of Epochs\")\n",
    "plt.ylabel(\"Training Loss\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Model Evaluation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_forecasts(actual, predicted):\n",
    "  \n",
    "  scores = list()\n",
    "  # calculate an RMSE score for each day\n",
    "  for i in range(actual.shape[1]):\n",
    "    # calculate mse\n",
    "    mse = mean_squared_error(actual[:, i], predicted[:, i])\n",
    "    # calculate rmse\n",
    "    rmse = np.sqrt(mse)\n",
    "    # store\n",
    "    scores.append(rmse)\n",
    "  # calculate overall RMSE\n",
    "    s=0\n",
    "    for row in range(actual.shape[0]):\n",
    "        for col in range(actual.shape[1]):\n",
    "            s += (actual[row, col] - predicted[row, col])**2\n",
    "    score = np.sqrt(s / (actual.shape[0] * actual.shape[1]))\n",
    "  return score, scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 324,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(159, 7, 7)"
      ]
     },
     "execution_count": 324,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 331,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7, 7)"
      ]
     },
     "execution_count": 331,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train[-2][:, :].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 329,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'numpy.ndarray' object has no attribute 'extend'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb Cell 82\u001b[0m in \u001b[0;36m<cell line: 8>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb#Y316sZmlsZQ%3D%3D?line=4'>5</a>\u001b[0m n_input \u001b[39m=\u001b[39m \u001b[39m14\u001b[39m\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb#Y316sZmlsZQ%3D%3D?line=6'>7</a>\u001b[0m \u001b[39m#extract the last 7 days training data from train\u001b[39;00m\n\u001b[0;32m----> <a href='vscode-notebook-cell:/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb#Y316sZmlsZQ%3D%3D?line=7'>8</a>\u001b[0m history \u001b[39m=\u001b[39m train[\u001b[39m-\u001b[39;49m\u001b[39m2\u001b[39;49m][:, :]\u001b[39m.\u001b[39;49mextend(train[\u001b[39m-\u001b[39m\u001b[39m1\u001b[39m][:, :])\n\u001b[1;32m      <a href='vscode-notebook-cell:/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb#Y316sZmlsZQ%3D%3D?line=8'>9</a>\u001b[0m \u001b[39m#rehsape hist into (B, T, C)\u001b[39;00m\n\u001b[1;32m     <a href='vscode-notebook-cell:/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb#Y316sZmlsZQ%3D%3D?line=9'>10</a>\u001b[0m B \u001b[39m=\u001b[39m \u001b[39m1\u001b[39m\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'numpy.ndarray' object has no attribute 'extend'"
     ]
    }
   ],
   "source": [
    "#evaluation \n",
    "\n",
    "#get the last train \n",
    "n_test = len(test)\n",
    "n_input = 14\n",
    "\n",
    "#extract the last 7 days training data from train\n",
    "history = train[-2][:, :].extend(train[-1][:, :])\n",
    "#rehsape hist into (B, T, C)\n",
    "B = 1\n",
    "T = 14\n",
    "C = 7\n",
    "\n",
    "predictions = []\n",
    "\n",
    "for i in range(n_test):\n",
    "    n_in = 0\n",
    "    # reshape the data from (B, T, C) -> (B, C, T)\n",
    "    history = torch.tensor(history[-T]).view(B, T, C)\n",
    "    # # reshape into [1, n_input, 1]\n",
    "    with torch.no_grad():\n",
    "        yhat = multi_channel_model.forward(history)\n",
    "        # store the predictions\n",
    "        predictions.append(yhat[0].tolist())\n",
    "    history = history.extend(test[i][n_in:n_in+n_input, :])\n",
    "    n_in += n_input\n",
    "    \n",
    "predictions = array(predictions)\n",
    "score, scores = evaluate_forecasts(array(test[:, :, 0].tolist()), predictions)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 322,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cnn: [485.897] 449.7, 544.5, 431.6, 508.7, 440.1, 397.2, 597.9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAEXCAYAAAAeBxskAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAABEkklEQVR4nO3deXwU9f348debJBDucIQzQAin3GBEDsWzKoqAR613ra1oa+1lseXXfq09bLXU1lpPtNbW+6hFWxVEEUQSUJBTIcCGcIQr2QABkpDr/ftjJriEkGzIZmd3834+Hnlkd8737MzOe+bz+cxnRVUxxhhjokEzrwMwxhhjgmVJyxhjTNSwpGWMMSZqWNIyxhgTNSxpGWOMiRqWtIwxxkSNiE9aInK2iGR5sN4cEbkwTOv6nYjki8iecKzPnEhEvisie0XksIh0ioB4nhSR/4u2ZUcbEflCRM6tZfwiEflOGONREekfrvVFo4hJWidLEqq6RFUHNdI6VUSOuCeqXBH5s4jE1XMZ54rIzgbE0Bu4Gxiiqt1OsvxKN8ZDIpIlIt+qYTv2iUh8wLAEd5gGDBsqIu+LSIGIHBCRlSJyaQ3rCfwbf6rb1hDV9o1fRD4UkW800roSgD8DF6lqG1X1h2CZJ5x8ROQ+EXkhmPlV9Q5V/W1D42joskNx0haRi0XkY/f4zRORxSIy1R13i/tZ3VNtnp1VycT93FRErgkYH+8OS21IbKo6VFUXBawnqP1TExG5odp3p8iN8fSGxNhQIpLqxhFf99QNXlejJ/mISVoeGqmqbYALgOuB28K8/t6AX1X31TLNLjfGdsCPgadFpHoi3w9MDng/2R0W6L/AAqAb0AX4AVBYfT3V/jLrv0nBq+OLVLVvBgHPAY+KyK8aIYyuQCLwRX1nFEfUfI/qe1EWgvVdDbwO/AtIwfms7wUuD5isALhHRNrWsqgC4Nfhjr8+VPXFwO8O8D0gG/jc49Bii6pGxB+QA1xYw/BzgZ3VpvspsBY4CLwKJAaMnwKsBg4AGcCIWtapQP+A968Dj1aPB2gBPAzscv8edoe1BoqBSuCw+9ejhvW0x/nS5gHbgF/iXDBcWG3+5+rafnfYPuDr1bbjl8DrAcPeAH7h7GIF6OxOl3SSz+KE9QSxv2YBX+Ikx38Eux/ceX/m7sOjQHxd+8YddjVQAnRy338L2AAcwjk53B4w7Xrg8oD3CUA+MLraMgcCR9z1HQYWusMnAJ+5x9hnwISAeRYB9wNL3f3XP8j47wNeCPy8ce6y9wG7gW8FTPsc8Dv39QZgSsC4ePdYGhNw3O5xY/0YGFptOU8A77rbeWG1ZXcA/ucub7/7OsUddz9Q4X7mh/nquzEY5+KnAMgCrjnJMSLAdmBmLcfRLcAnOBdUvwoYvhM4N+BzexFYA3wz4DNQILWGZZ4HrAt4vwD4LOD9EmB64PccuAQoBcrcbV0TsK9/6+7rQ8D7QOcgvyMfBW7TSaaZ6e77XcCtgccNcBmwCufCcgdwX8B87wB3VVvWWuCKGtaR6i43PuCYeMxdxiFgOdCv2rH7A5zvVD4wG2hW/Riuvuyajhf3GPgLzjFeCKwDhgV7nqnxM2vIzKH8o35J61OgB9AR5wt9hztutPvhnAnEAd90p29xknUGHiBDcL74364eD/AbYBnO3Ukyzkn4tzXFd5L1/At4C2jr7uRNAeupdf7A8TiJbipOkhtdbTuGAXuBJJwT0V53mLrTCLAZ56Q0Heha2+cc5P5aD/Ry98NSvjoR1rof3Ner3Xlb1rVvAoYlAOXA5IAvdT93284BivjqRH4P8GrAvNMIOJHV8aXuiHMCvwnny3id+74qWS7CORkPdccnBBn/fRyftMrdYysBuNSNv4M7/rmAz/Ne4MWA5VwGbAh4fyvOsVV1cbU6YNxzOMlsonv8JFZbdifgKqCVu4zXgbkB8y8CvhPwvjXOCfRb7raPxjmxDanhMxjsfg59azmObsFJWqPcz7ijO7x60noB59jPdj+v2pJWS5wTZ2d32r1Arrt9LXEuNKr2ZQ5ffc+P7Z9q2+/Dubhp6b5/IIjvRx+cE3ht234JX31PWwMvcfw56VxguLvfRrjTTnfHXQMsD1jWSMAPNA/i+H7OnXas+zm+CLxS7dj9COd70BvnfPWdmj6jGpZd/Xi5GFiJc14S4DSge7DnmZr+oqZYo5pHVHWXqhbgXKGNcofPAJ5S1eWqWqGq/8S5kh9Xy7I+F5H97nKewbljqO4G4Dequk9V84Bf45zQ6uQWZ1wLzFLVQ6qaAzwU7PyuHiJyAOfL9h/gJ6q6qto0Je42fMP9e9sdBhzLXOfhfEkfAna79QwDqq+n2l/rWuJ6VFV3uPvhfpyTOwS3Hx5x5y0O9kNQ1TKcE2RH9/07qupTx2Kcq+Cz3clfAC4VkXbu+5uA54Nc1WXAZlV9XlXLVfVlYCPHF2k9p6pfuOPLgt2GaspwjqsyVX0X5+q0pvrbl4CpItLKfX898HLVSFV91j22juKcVEaKSPuA+d9S1aWqWqmqJQHDUVW/qv5bVYtU9RDOfjynlpinADmq+g9321cB/wa+XsO0VQ1adteyvKo4VuPcEf2slmnexrkjrLXOxD2mPgMmAafj3KEtxUnc43D2bX3qLf+hqpvc5b7GV+eb2twMLFHVrbVMc4277PWqegRn3wVuxyJVXefut7U4+7xq37wNDAz4/t6Ec5FWGuQ2/UdVP1XVcpykVX2bHlTVAlXdjnMhdB2npgznYmEwIKq6QVXrPB5qE61JK7CVXRHQxn3dB7g78KSLczXfo5ZljVHVDqraT1V/qaqVNUzTA6dYr8q2OpYZqOpqr/r8PYOcH5y6piScOq1HgPNPMt2/cL4sN7uvj6OqO1X1+6raD+ezOlJtul2qmlTt70gtce0IeB34mQSzHwLnDYrbYCIZp1gKEZksIsuqGpbg3K10drd1F86J6ioRScKp43sxyFVV399w4j6rK/4KnP0eKAHnS1zF7540qgQey8eo6hacEoXL3cQ1FSeRISJxIvKAiPhEpBDnogTcz6GuWEWklYg8JSLb3Pk/BpJqqTvqA5xZbd/egFNPWl1VYuh+svVXcy/wXRHpWss0v8Qp9k6sY1mLce5UJrmvF+Gc8M9x39fHyc43tbkZ+GfVGxHpHdhIwx3cgxO/QwTMc6aIfOQ2XjkI3MFXx3cJTtXIjW6d6nUEf1EWzDad7LtdL6q6EKeY8DFgn4jMCbiQPCXRmrROZgdwf7WTbiv3SrkhduF8Wav0doeBc2tcm3ycE1X1+XPrG4R7Jf0zYLiITK9hkiU4J4iuOEUutS1rB86BNKy+cQToFfA68DMJZj/U9bnVZBpOkdqnItIC5wr/TzhFnUk49TYSMP0/gRtx7gIyVTXYz7z6/oYT91ld8W/HKToJ1JcTk2GwXsY5MU0DvnQTGTh3XdNw6mXaB6wz8HOoLda7ce7uzlTVdjgn+cD5q8+7A1hcbd+2UdXv1rDsLHf6q+raOABV3Qi8iZOUTjbNAmALTiOH2lRPWoupO2mdyjF5AhGZiHOSf+PYglW36/GNNMC5A63+HQr0Es4dVS9VbQ88yYnH9w04jciKNLSNpk723T6CU5RcpfrFygmfoao+oqqn41TBDMSpxztlkZa0EkQkMeCvvk00nwbucK9QRERai8hldbRKCsbLwC9FJFlEOuNcEVY1jd0LdKpWHHOMqlbgFCncLyJtRaQP8JOA+evFvf1/yI2h+jjFKcKa6r4+RkQ6iMivRaS/iDRzt+NWnLq6U3WniKSISEecE82r7vCQ7gcR6SgiN+Ak2Qfdop3mOHU4eUC5iEwGLqo261xgDPBDarjzrMW7OEUv17tNq7+B84X7Xz2W8SrOMZPift4X4uybN+qY72Rewdm+7+LeZbna4hS9+nFOJr+v53Lb4hQ7H3D346+qjd8LpAW8/x/OZ3OTOI9VJIjIGSJyWvUFu8fgT4D/E5FviUg797M4S0TmnCSeX+PUlyXVEvMvcOosa5OBk4zHAp+q6he4d4k4d5M12QukhqA16DeBf7vFrbV5DbhFRIa4d9DVP/u2QIGqlojIWJwLlGPcJFWJcz6oz11WMGa654xeON+fqu/2amCSe+fYHqcxVqDjjhf32DjTLSU5glNlUVNpVtAiLWm9i/MFqvq7rz4zq+oKnCbrj+JU6m7BqehtqN8BK3Ba56zDacL6O3edG3GSWrZbXFLTbfRdODssG+cO6CXg2QbE8yzQW0Qurz7CrWepqel2Kc5V+Ac4rXjW45zsbgmYpoec+JxWbVfJL+HUI2XjVFZXfSah2g9r3KKULTj1GD9W1XvddRzCaeH0mruO63GuSo9x6yD+jXOH82awK3WT4hScuxA/zglyiqrm1yP23+CcOD9x4/sjcIOqrq/HMgJj2g1k4rRqfDVg1L9w7t5ycVpy1vci5GGcBgb57rzzqo3/K3C1iOwXkUfcz/0inHraXTjFTA/iXEDUFPcbOHWst7rT78U5Tt46yfRbcU7AJ61LVdWlOI2xTsot1v4c+CKgnicT2KYnf7zkdfe/X0ROqZm6iCTi1FX9s65pVfU9nM9/Ic4xvrDaJN8DfiMih3AuUl+rYTH/wmmsccrPl53EWzgNKFbjtDL8uxvzApzjb607vvqF3HHHC06VxtM434FtON+n2QAi8v9E5L36BibVLsiNCYqI5OC0EvrA61hqIyL3AgNV9UavYzEm1ETkZmCGqp4VwmUqMCCgCDqiNPoT0sZ4xS3u+jb1a6lpTFRwixS/BzzudSzhFGnFg8aEhIjchtMI4D1VPVkdhjFRSUQuxqnP3cvxdZwxz4oHjTHGRA270zLGGBM1LGkZY4yJGpa0jDHGRA1LWsYYY6KGJS1jjDFRw5KWMcaYqGFJyxhjTNSwpGWMMSZqWNIyxhgTNSxpGWOMiRqWtIwxxkQNS1rGGGOihiUtY4wxUcOSljHGmKhhScsYY0zUsKRljDEmasR7HQBA586dNTU11eswjDEmqqxcuTJfVZO9jiOcIiJppaamsmLFCq/DMMaYqCIi27yOIdyseNAYY0zUsKRljDEmaljSMsYYEzWCSloikiQib4jIRhHZICLjRaSjiCwQkc3u/w7utCIij4jIFhFZKyJjGncTjDHGNBXB3mn9FZinqoOBkcAG4OfAh6o6APjQfQ8wGRjg/s0AnghpxMYYY5qsOlsPikh7YBJwC4CqlgKlIjINONed7J/AIuBnwDTgX6qqwDL3Lq27qu4OefTGGBOl5q7KZfb8LHYdKKZHUktmXjyI6aN7eh1WxAvmTqsvkAf8Q0RWicgzItIa6BqQiPYAXd3XPYEdAfPvdIcdR0RmiMgKEVmRl5d36ltgjDFRZu6qXGa9uY7cA8UokHugmFlvrmPuqlyvQ4t4wSSteGAM8ISqjgaO8FVRIADuXZXWZ8WqOkdV01U1PTm5ST0bZ4xp4mbPz6K4rOK4YcVlFcyen+VRRNEjmKS1E9ipqsvd92/gJLG9ItIdwP2/zx2fC/QKmD/FHWaMMQbYdaC4XsPNV+pMWqq6B9ghIoPcQRcAXwJvA990h30TeMt9/TZws9uKcBxw0OqzjDHmKz2SWtZruPlKsN043QW8KCLNgWzgWzgJ7zUR+TawDbjGnfZd4FJgC1DkTmuMMcY18+JB/PT1NZRXflWr0jIhjpkXD6plLgNBJi1VXQ2k1zDqghqmVeDOhoVljDGxa/ronsz52MemvYepqFRrPVgPEdFhrjHGNCVlFZVs8xfxjTN6cf8Vw70OJ6pYN07GGBNm63IPcqS0ggn9OnsdStSxpGWMMWGW6fMDMC6to8eRRB9LWsYYE2aZPj+Du7WlU5sWXocSdSxpGWNMGB0tr+CznALG9+vkdShRyZKWMcaE0artBzhaXmn1WafIkpYxxoRRps9PM4Gxfa0+61RY0jLGmDDK9PkZ1rM97VsmeB1KVLKkZYwxYVJcWsGqHfutPqsBLGkZY0yYrNhWQFmFWn1WA1jSMsaYMMnw+YlvJqT36eB1KFHLkpYxxoRJhs/PqF5JtG5hPeidKktaxhgTBoUlZazbeYAJVp/VIJa0jDEmDD7bWkClwnirz2oQS1rGGBMGGT4/zeObMbp3ktehRDVLWsYYEwYZPj/pfTqQmBDndShRLajaQBHJAQ4BFUC5qqaLyKtA1c9sJgEHVHWUiKQCG4Asd9wyVb0jlEEbY0w02X+klA27C/npRQO9DiXq1acJy3mqml/1RlW/UfVaRB4CDgZM61PVUQ0Pzxhjot+ybOenSOyh4oZrcLtLERHgGuD8hodjjDGxJ8Pnp1XzOEakJHkdStQLtk5LgfdFZKWIzKg27mxgr6puDhjWV0RWichiETm7pgWKyAwRWSEiK/Ly8k4hdGOMiQ6Z2X7G9u1IQpw1I2ioYD/Bs1R1DDAZuFNEJgWMuw54OeD9bqC3qo4GfgK8JCLtqi9QVeeoarqqpicnJ59i+MYYE9n2FZawZd9hxqdZ0WAoBJW0VDXX/b8P+A8wFkBE4oErgVcDpj2qqn739UrAB1jtozGmScp067Osv8HQqDNpiUhrEWlb9Rq4CFjvjr4Q2KiqOwOmTxaROPd1GjAAyA514MYYEw0yfX7aJcYzpMcJBU7mFATTEKMr8B+nvQXxwEuqOs8ddy3HFw0CTAJ+IyJlQCVwh6oWhCheY4yJKhk+P2emdSKumXgdSkyoM2mpajYw8iTjbqlh2L+Bfzc4MmOMiXI79xexvaCIb01M9TqUmGFNWYwxppFk+qw+K9QsaRljTCPJ9Pnp1Lo5A7u28TqUmGFJyxhjGoGqkuHzM65fJ9w2ASYELGkZY0wj2Jp/hD2FJfb7WSFmScsYYxqBPZ/VOCxpGWNMI8jw+enWLpHUTq28DiWmWNIyxpgQU1WW+fxMsPqskLOkZYwxIbZp72H8R0rtp0gagSUtY4wJsQyf89ODlrRCz5KWMcaEWIbPT++OrUjpYPVZoWZJyxhjQqiiUlme7bem7o3EkpYxxoTQl7sKKSwpt6LBRmJJyxhjQuhYfZb96GOjsKRljDEhlJntp3+XNnRpl+h1KDHJkpYxxoRIWUUln24tsPqsRhRU0hKRHBFZJyKrRWSFO+w+Ecl1h60WkUsDpp8lIltEJEtELm6s4I0xJpKs3XmAotIKKxpsRMH8cnGV81Q1v9qwv6jqnwIHiMgQnF80Hgr0AD4QkYGqWtGwUI0xJrJlbHH6GxxnSavRNEbx4DTgFVU9qqpbgS3A2EZYjzHGRJTMbD9DurejQ+vmXocSs4JNWgq8LyIrRWRGwPDvi8haEXlWRDq4w3oCOwKm2ekOO46IzBCRFSKyIi8v75SCN8aYSFFSVsGKbfutqXsjCzZpnaWqY4DJwJ0iMgl4AugHjAJ2Aw/VZ8WqOkdV01U1PTk5uT6zGmNMxPl8+35KyyutEUYjCyppqWqu+38f8B9grKruVdUKVa0EnuarIsBcoFfA7CnuMGOMiVnLfH7imglj+3b0OpSYVmfSEpHWItK26jVwEbBeRLoHTHYFsN59/TZwrYi0EJG+wADg09CGHVvmrspl4gML6fvzd5j4wELmrrIcb0y0yfD5Gd6zPW0TE7wOJaYF03qwK/Af9zdh4oGXVHWeiDwvIqNw6rtygNsBVPULEXkN+BIoB+60loMnN3dVLrPeXEdxmfMR5R4oZtab6wCYPvqEqkBjTAQ6crSc1TsOcNukNK9DiXl1Ji1VzQZG1jD8plrmuR+4v2GhNQ2z52cdS1hVissqmD0/y5KWMVFixbb9lFeq1WeFgfWI4bFdB4rrNdwYE3kyfPkkxAnpfaw+q7FZ0vJYj6SW9RpujIk8mT4/o3t1oGXzOK9DiXmWtDx2+zl9TxjWPE6YefEgD6IxxtTXweIy1ucetOezwsSSlsey9hxGgK7tWiBAXDOhW/tEpo3q4XVoxpggfLq1gErF6rPCxJKWh3x5h3nlsx3cPL4Py//fhWx94DJ+dfkQthcUs2Lbfq/DM8YEIcOXT4v4ZozqneR1KE2CJS0PzZ6XRWJ8M+66YMCxYV8/vRcdWiXw1OJsDyMzxgQr0+fnjNSOtIi3+qxwsKTlkZXb9jPviz3cfk4/OrdpcWx4y+Zx3DQ+lQ827GXLvsMeRmiMqYv/8FE27jlk9VlhZEnLA6rKA+9tILltC75z9okNMW4e34cW8c14ZondbRkTyZZlFwBY0gojS1oe+GDDPj7L2c+PLhxAq+YnPt/duU0Lrjo9hTc/z2XfoRIPIjTGBCPDl0+bFvGM6Nne61CaDEtaYVZeUcmD8zaSltyab6T3Oul0t52dRlllJf/K2BbG6Iwx9ZGZ7Wds347Ex9mpNFzskw6z11fuZMu+w9xz8eBaD/S+nVtz0ZCuPL9sG0eOlocxQmNMMPYcLCE774g1dQ8zS1phVFRazl8WbOL0Ph24eGjXOqefMakfB4vLeG3FjjqnNcaEV2Z2PgDj0ixphZMlrTB69pOt7Dt0lP936WDcXvNrdXqfDpzepwN//2Qr5RWVYYjQGBOsTJ+f9i0TGNK9ndehNCmWtMLEf/goTy7O5qIhXTm9Hp1qzpiUxs79xby7fk8jRmeMqa8Mn5/xaZ1o1qzuC1ATOpa0wuRvC7dQXFbBPZcMrtd8XzutK2mdWzPnYx+q2kjRGWPqY0dBETv3F1tTdw9Y0gqD7f4iXly+jWvSe9G/S5t6zdusmfCds9NYn1tIZra/kSI0xtRHhs+pz7JGGOEXVNISkRwRWSciq0VkhTtstohsFJG1IvIfEUlyh6eKSLE77WoRebIR448Ks9/PIr5ZM3584YC6J67BlWN60rlNc+Z8bA8bGxMJMn1+OrdpUe+LUNNw9bnTOk9VR6lquvt+ATBMVUcAm4BZAdP63GlHqeodoQo2Gq3deYD/rtnFd87uS5d2iae0jMSEOL45PpVFWXlk7TkU4giNMfWhqk59Vr9OQTWoMqF1ysWDqvq+qlY9QLQMSAlNSLHD6a5pIx1bN2fGpLQGLevGcX1omRBnd1vGeMyXd4R9h45a0aBHgk1aCrwvIitFZEYN428F3gt431dEVonIYhE5u6YFisgMEVkhIivy8vLqGXZ0WLwpjwyfnx+c35+2iQkNWlaH1s25Jj2Ft9fksuegde1kjFeq6pYtaXkj2KR1lqqOASYDd4rIpKoRIvILoBx40R20G+itqqOBnwAvicgJDzKo6hxVTVfV9OTk5AZtRCSqqHTusvp0asX1Z/YJyTK/c3YaFZXKPzK2hmR5xpj6y/Tl0zOpJb07tvI6lCYpqKSlqrnu/33Af4CxACJyCzAFuEHd9tiqelRV/e7rlYAPGBjyyCPcf1blsnHPIX560SCax4emkWavjq2YPLw7Ly3bzqGSspAs0xgTvMpKJdPnZ1ya1Wd5pc6zqYi0FpG2Va+Bi4D1InIJcA8wVVWLAqZPFpE493UaMABoUhUxJWUV/Pn9LEamtOey4d1DuuzbJ6Vx6Gg5r3xqXTsZE25Zew+xv6jMigY9FMwtQFfgExFZA3wKvKOq84BHgbbAgmpN2ycBa0VkNfAGcIeqFoQ+9Mj1z4wcdh0s4eeTTwv50/IjUpI4s29Hnl26lTLr2smYsMrwOfVZ9lCxd078MadqVDUbGFnD8P4nmf7fwL8bHlp0OlBUymMfbeG8QcmNdmDffk4atz63gv+t3cUVo63RpjHhkunLJ7VTK3oktfQ6lCbLesQIsccX+Th0tJyfTa5fd031ce7ALgzo0oanFmdb107GhEl5RSXLswsY36+z16E0aZa0Qij3QDHPZeRw1ZgUBndrvJ6fmzUTbpuUxsY9h1iyOb/R1mOM+coXuwo5dLTc6rM8ZkkrhB56PwsBfvK1xm8sOW1UD7q0bWEPGxsTJlX1Wfb7Wd6ypBUiX+4q5D+rcrllYmpYyrtbxMfxrYl9+WRLPutzDzb6+oxp6jJ8+Qzs2obkti28DqVJs6QVIg/O20i7xAS+d06N7VMaxfVn9qZ18zieWWJ3W8Y0ptLySlbk7GeC1Wd5zpJWCCzdks/iTXl8/7z+tG/VsO6a6qN9ywSuHdub/67dTe6B4rCt15imZs3OAxSXVVjRYASwpNVAlZXKH97bQM+kltw0PjTdNdXHrWf1BeDZT6xrJ2MaS8YWPyIwLi34Xx03jcOSVgP9d+0u1ucWcvdFA0lMiAv7+nsmteTyEd155dPtHCy2rp1Cae6qXCY+sJC+P3+HiQ8sZO6qXK9DMh7JzM5naI92JLVq7nUoTZ4lrQY4Wl7Bn97P4rTu7Zg+qqdncdw2KY0jpRW8uHybZzHEmrmrcpn15jpyDxSjOI8zzHpznSWuJqikrILPtx2w+qwIYUmrAV5ctp0dBcXMmjw45N011cfQHu05q39n/rE0h6PlFZ7FEUtmz8+iuOz4z7K4rILZ87M8ish4ZeW2/ZRWVDLe6rMigiWtU1RYUsbfFm7mrP6dmTTQ+59WmTEpjbxDR3lr9S6vQ4kJu07SsOVkw03syvT5iWsmnNHX6rMigSWtU/TUYh/7i8r4eSN211QfZw/ozGnd2/H0x9lUVlrXTg3VpV3Nz+JYn3NNT4Yvn5Ep7WnTos6uWk0YWNI6BXsOlvD3T7YybVQPhvVs73U4AIgIMyb1ZfO+wyzatM/rcKJaRaXSqoZGNfHNhJkXD/IgIuOVw0fLWbPzoPXqHkEsaZ2Chz/YRGUl/PSiyDqBTRnRg+7tE3lqsT1s3BD/WLqVrf4irh/bi55JLREgMaEZFZVKr452p9WUfJZTQEWlWiOMCGL3u/W0ee8hXluxg1sm9KVXhP3cdkJcM26d2Jf7393Amh0HGNkryeuQoo4v7zCz52dxweAu3H/F8GO/TltYUsaURz7hrpdW8c4PzqZDa2v63BRk+vw0j2vG6X06eB2KcQV1pyUiOSKyzv2xxxXusI4iskBENrv/O7jDRUQeEZEtIrJWRMY05gaE24PzNtK6eTzfPz983TXVx7Vje9G2RTxzrGunequoVGa+vobEhDh+f+Xw435OvV1iAo9dP4b8w6Xc/foaqzdsIjJ8+YzuneTJM5imZvUpHjxPVUeparr7/ufAh6o6APjQfQ8wGRjg/s0AnghVsF77dGsBH2zYxx3n9qNjhF5pt01M4PpxvXlv3W62+4u8DieqPPvJVj7ffoD7pg6ha7vEE8YPT2nPL6ecxsKN+3jaLgpi3oGiUr7YVWhFgxGmIXVa04B/uq//CUwPGP4vdSwDkkSkewPWExFUne6aurVL5NaJfb0Op1a3TuxLXDPh75/YiTVYvrzD/On9LC48rWutD4rfNK4Plw3vzh/nZ7EipyCMEZpwW761AFWY0N8aYUSSYJOWAu+LyEoRmeEO66qqu93Xe4Cu7uuewI6AeXe6w6LavPV7WLX9AD/+2gBaNo/sooKu7RKZNqonr63Yyf4jpV6HE/EqKpWfVhULXjHsuGLB6kSEP1w1nJQOLbnr5VUU2OcbszJ9flomxDEyJcnrUEyAYJPWWao6Bqfo704RmRQ4Up3ffK9XIb+IzBCRFSKyIi8vrz6zhl1ZRSV/nJ/FwK5tuGpMitfhBGXGpDSKyyp4fpl17VSXZ5Zks2r7AX49dShdaigWrK6qfst/uJS7X1tt9VsxKsOXT3pqB5rHWyPrSBLU3lDVXPf/PuA/wFhgb1Wxn/u/6uGgXKBXwOwp7rDqy5yjqumqmp6c7H2PErV55bMdbM0/ws8uGUx8XHQcwAO7tuXcQcn8MyOHkjLr2ulktuw7xEMLNvG1IV2ZNqpH0PMN69me/5tyGh9l5fGU/Xp0zMk7dJRNew9bfVYEqvMMLCKtRaRt1WvgImA98DbwTXeybwJvua/fBm52WxGOAw4GFCNGnSNHy/nrB5sZ27cj5w/u4nU49TJjUhr+I6W8+bl18loTp1hwLa2ax3F/HcWCNblxXB8uG9GdP72fxWdWvxVTlmX7AZhgDxVHnGBuG7oCn4jIGuBT4B1VnQc8AHxNRDYDF7rvAd4FsoEtwNPA90IedRg9vSSb/MNHmTV5cL1Pal4bn9aJ4T3b88wS69qpJk8vyWb1DrdYsG3dxYLViQgPXOnWb71k9VuxJMPnp22LeIb2aOd1KKaaOpOWqmar6kj3b6iq3u8O96vqBao6QFUvVNUCd7iq6p2q2k9Vh6vqisbeiMaSd+gocz7O5tLh3RjdO/oeLnS6dkojO/8ICzbs9TqciLJl3yH+vGATFw/tytSRwRcLVtfWrd8qKCrlx69a/VasWJbt58y0jlFTHdCU2B6pxSMfbqa0vJKZF0dGp7inYvKwbqR0aMkcq3c5pryikrtfX0vr5nH8bvrwBt9BD+vZnnunDGHxpjye/NgXoiiNV3YdKGZr/hHGW31WRLKkdRLZeYd56dPtXDe2N307t/Y6nFMWH9eMb5/Vl5Xb9rNym9W7AMxZks2aHQf49bRhJLetuTf3+rrhzN5MGdGdh97fxKdb7XOOZpk+pz7Lfj8rMlnSOonZ87NIjG/GDy4Y4HUoDXZNei/at0ywuy1g095DPLxgM5OHdePyEaF75l1E+MOVw+ndsRV3vfw5/sNHQ7ZsE16Z2X46tEpgcLe2XodiamBJqwafb9/Pe+v3cNuktJBdiXupdYt4bhrXh/e/3Et23mGvw/FMeUUlP319DW0S4/nt9Pq3FqxL28QEHr1+NPuLyvjxa9Y/YTRSVTJ9fsb36+Tpr5Gbk7OkVY2q8sC7G+ncpgW3nZ3mdTgh880JqSTENeOZT7Z6HYpnnvo4m7U7D/KbaUPp3KZxLkaG9mjPry4fwseb8nhisdVvRZvtBUXkHii2osEIZkmrmg837OPTnAJ+eOEAWsfQL5Umt23BVWN68sbKneQ3waKrrD2H+OsHm7l0eDemjDj11oLBuH5sb6aO7MFD72ex3H3ex0SHjKr6LGuEEbEsaQUor6jkwXkbSevcmmvP6FX3DFHmO2enUVpeyb8ym1bXTmUBxYK/mTas0dcnIvz+yuGkdmrNXS+vapIXCdEq0+enS9sW9EuO3sZXsc6SVoB/f76TzfsOc88lg0iIwecz+iW34cLTuvJ8Zg7FpU2na6enFvtYl3uQ304b1mjFgtW1aRHPYzeM4WBxmT2/FSVUlQyfnwn9OkVdRwJNSeydmU9RcWkFf16wiTG9k7h4aDevw2k0t5+Txv6iMl5fuaPuiWPAxj2F/PXDzVw2ojuXhbC1YDBO696O+6YOZcnmfB5ftCWs6zb1t2XfYfIPH2W8dd0U0SxpuZ5dupW9hUeZdelpMX2Vld6nA6N7J/HMkq1UxPjVf1WxYLvEBH4zdagnMVx7Ri+mjerBnxdsOvb8j4lMmcf6G7T6rEhmSQsoOFLKk4t8XHhaV85I7eh1OI1KRLh9UhrbC4qYt36P1+E0qicX+VifW8jvpg+jU5iKBasTEX5/xXBSO7fmB6+sIu+Q1W9FqowtflI6tKRXx1Zeh2JqYUkLeHThFo6UlvOzSwZ5HUpYfG1IN1I7tWLOxz6cn0KLPRt2F/LIws1MGdGdycO9/eHs1i3ieez6MRS69VuxfocbjSorlWVb/dbUPQo0+aS13V/E88tyuCa9FwO6No0n4OOaCd8+O401Ow/GZJdDVcWC7VsmhKW1YDBO696OX08dyidb8nnsI6vfijQb9hRyoKiMCf0taUW6Jp+0/vR+FnHNhB9/baDXoYTV109PoWPr5jHZtdPjH/n4Ylchv5s+nI6tm3sdzjHfOKMXV4zuycMfbCLDl+91OCbAV/0NWn1WpGvSSWvdzoO8vWYX3z6rL12D+Jn1WJKYEMfN4/vw4cZ9bN57yOtwQubLXYX8beFmpo7swSXDIqsVqIjwu+nD6Nu5NT98ZbXVb0WQDJ+ftM6t6da+aZ0HolGTTVqqygPzNtChVQK3n9PP63A8cfP4VBITmvH0kti426oqFkxq1Zxfe9RasC6tW8Tz+A2nc6ikjB+9usrqtyJAeUUln24tsKbuUSLopCUicSKySkT+575fIiKr3b9dIjLXHX6uiBwMGHdvI8XeIB9vzmfpFj93nT+AdokJXofjiY6tm/P103sxd9Uu9hWWeB1Ogz320Ra+3F3I/VcMo0MEFQtWN6hbW34zdRhLt/j528LNXofT5K3LPcjho+XW1D1K1OdO64fAhqo3qnq2qo5S1VFAJvBmwLRLqsap6m9CE2roVFYqD7y3kV4dW3LDuN5eh+Op75zdl7LKSp7LyPE6lAb5YtdBHl24hWmjekTFw+FfT0/hytE9+euHm8nYYvVbXqrqb3BcWmw/7hIrgkpaIpICXAY8U8O4dsD5wNyQRtaI5q7OZcPuQmZePJgW8XFeh+OpPp1ac8nQbrywbBuHj5Z7Hc4pKS2v5KevryWpVXPuuzwyiwWrExF+d8Uw+iW34QevrGbfoei/041WmT4/g7u19exZPlM/wd5pPQzcA1TWMG468KGqFgYMGy8ia0TkPRGp8SwiIjNEZIWIrMjLy6tPzA1SUlbBQ+9vYnjP9kzx+PmdSDFjUhqFJeW8+ll0du306Edb2LC7kN9HeLFgda2aO89vHT5axg9ftue3vHC0vIIV26w+K5rUmbREZAqwT1VXnmSS64CXA95/DvRR1ZHA3zjJHZiqzlHVdFVNT05Orl/UDfB85jZyDxQza/Jg+5E31+jeHRib2pFnP9lKWUVN1yWRa33uQR7/aAtXjO7JRVFQLFjdoG5t+e20YWRm+3nkQ6vfCrfV2w9QUlZp9VlRJJg7rYnAVBHJAV4BzheRFwBEpDMwFninamJVLVTVw+7rd4EEdzrPHSwq49GPtnDOwGQm9I+IkCLGjElp5B4o5t11u70OJWhOseAaOrRuzq8uH+J1OKfs6+m9uGpMCo8s3Mwnm61+K5wyfH6aCYzta/VZ0aLOpKWqs1Q1RVVTgWuBhap6ozv6auB/qnqsQF5Euonb46yIjHXXERE9hT6+aAuFJWX8fPJgr0OJOOcP7kK/5NbM+Tg7arp2enThZjbuOcQfrhhOUqvoKRasyW+nD6V/cht+9OqqmGjJGS0ys/0M69me9i2bZgviaNTQ57Su5fiiQXAS2XoRWQM8AlyrEXAWzD1QzD8ycrhidE9O697O63AiTrNmwm1np/HFrsJjraki2frcgzy2yMeVo3ty4ZCuXofTYK2ax/P4DWM4crSCH7xiz2+FQ3FpBau277f6rChTr6SlqotUdUrA+3NVdV61aR5V1aGqOlJVx6lqRqiCbYg/v78JgLsvahqd4p6K6aN70rlNC56K8K6djpZX8NPX19CpdXN+FSWtBYMxoGtbfjt9GMuyC/jrB5u8DifmrdhWQFmFWie5UaZJ9IixYXchb67ayS0TUumZ1NLrcCJWYkIc35qYyseb8tiwu7DuGTzytw+3sHHPIR64ajjtW8VWsc7Vp6fw9dNT+NtHW1iyOXytapuiTJ+f+GYS8z9HFGuaRNJ6cN5G2raI53vnNs3umurjxjP70Kp5XMR27bR25wGeWOzjqjEpnD84+osFa/KbacMY0KUNP3plNXutfqvRZPj8jOqVROsW8V6HYuoh5pNWhi+fRVl53Hle/6ivrA+H9q0S+MYZvXh79S52Hyz2OpzjVBULdm7TnHujuLVgXVo2j+Ox68dQVFrBD15eRXmUPYYQDQpLyli784DVZ0WhmE5aVd019WifyDcnpHodTtS4dWJfFPjH0hyvQznOIx9uZtPewzxw5YiYb+01oGtbfjd9GMu3FvBXe34r5D7bWkClYkkrCsV00npn3W7W7jzI3RcNIjGhaXfXVB+9Orbi0uHdeWn5dgpLyrwOB3CKBZ9cnM3Vp6dw3uAuXocTFlednsI16Sk8+tEWPt5k9VuhlOnz0zy+GWN6d/A6FFNPMZu0SssrmT0/i8Hd2jJ9dE+vw4k6t09K4/DRcl5evt3rUDhaXsHdr60huU0L/m9K7BYL1uTXU4cxsEtbfvTqavYctPqtUMnw+Unv08EuZqNQzCatl5ZvY3tBET+fPJg4666p3ob1bM+Efp34x9IcSsu9rVN5+IPNbN53mD9cNTzmiwWra9k8jsduGENJmfP8ltVvNdz+I6V8ubvQmrpHqZhMWodKynhk4RYm9OvEOQPD169hrJkxKY09hSX8d80uz2JYveMATy32cU16CucNahrFgtX179KG+68YxqdbC/iLPb/VYMu3Og/PT+hvSSsaxWTSempxNgVHSpk1+TTcHqXMKThnYDKDurbl6SXedO1UUua0FuzaLpFfNrFiwequGJ3CtWf04rGPfCy2+q0GyfD5adU8jhEpSV6HYk5BzCWtvYUlPPNJNpeP7MHwlPZehxPVRITbJqWxcc8hT06UD3+wmS37DvPAVSOa7K9LB7pv6lAGd2vLj19dHXGPI0STDJ+fM1I7khAXc6e/JiHm9trDH2yiolKZad01hcTUkT3o1i6ROWHu2mnV9v3M+djHtWf0siJeV2JCHI9e79Zv2fNbp2TfoRK27DvMBGvqHrViKmlt2XeIVz/bwQ1n9qF3p1ZehxMTmsc341sTU8nw+VmfezAs66wqFuzWLpFfXHZaWNYZLfp3acPvrxjOZzn7+fMCq9+qr0y3M2j7/azoFVNJ68F5WbRqHs9d5/f3OpSYct2ZvWnTIj5sd1t/WbAJX94RHrhqBG2tWPAE00f35LqxvXh8kY+PsvZ5HU5UyfT5aZcYz5Ae9ksP0SpmktaKnAIWfLmXO85Jo1ObFl6HE1PaJSZw/Zm9eWfdbnbuL2rUdX2+fT9PL8nmurG9mGTFgif1q8ud+q2fWP1WvWRm+zkzrZM9BhPFYiJpqSq/f3cDXdq24Naz+nodTky6ZUIqAvz9k62Nto6qYsHu7Vvy/y61YsHaJCY4z2+Vlldy10urKLP6rTrt3F/ENn+R1WdFuZhIWvO/2Mvn2w/wk68NpFVz67G5MfRIasnUkT149bMdHCxqnK6d/rxgE9l5R3jgquFWLBiEfslt+P2Vw1mxbT8PvW/1W3Wx+qzYEHTSEpE4EVklIv9z3z8nIltFZLX7N8odLiLyiIhsEZG1IjKmkWJn7qpcJjzwIXe8sJL4ZkLzOLvlb0y3TUqjqLSCF5ZvC/myV24r4Okl2Vx/Zm/OHmDFgsGaNqon15/ZmycX+/hoo9Vv1SbT56dT6+YM7NrG61BMA9TnTuuHwIZqw2aq6ij3b7U7bDIwwP2bATzR4ChrMHdVLrPeXMeuA05/bOWVyi/mfsHcVbmNsToDnNa9HZMGJvOPpTkcLa8I2XJLyiqY+fpaelix4Cm5d8oQTuvejh+/tppdB6x+qyaqSma2n3H9OlmHA1EuqKQlIinAZcAzQUw+DfiXOpYBSSLSvQEx1mj2/CyKy44/cRaXVTB7flaoV2UC3D4pjfzDR0N6cfCn+Vlk5x/hj1ePoI39IF+9JSbE8dj1oykrr+Sul61+qyY5/iJ2Hyyx+qwYEOyd1sPAPUD1b8P9bhHgX0SkqsleT2BHwDQ73WHHEZEZIrJCRFbk5dW/t4WTXVHalWbjmtCvE0N7tGPOx9lUVja8a6cVOQX8felWbjizNxP7W13DqUpLbsMfrhrBym37+ZNduJ0gw5cPYJ3kxoA6k5aITAH2qerKaqNmAYOBM4COwM/qs2JVnaOq6aqanpxc/zqMHkkt6zXchIaIMGNSGr68IyxsYB1KcWkFM99wigVnWbFgg00d2YMbzuzNUx9n8+GGvV6HE1EyfX66tUukb+fWXodiGiiYO62JwFQRyQFeAc4XkRdUdbdbBHgU+Acw1p0+F+gVMH+KOyykZl48iJbVfgunZUIcMy+27psa26XDu9MzqWWDHzb+0/tZbM0/wmwrFgyZ/5syhCHd23H362vItVIHwK3P8vmZYPVZMaHOpKWqs1Q1RVVTgWuBhap6Y1U9lThHwXRgvTvL28DNbivCccBBVd0d6sCnj+7JH64cTs+klgjQM6klf7hyuP3gYxgkxDXj1rP68mlOAau27z+lZXyWU8CzS7dy07g+TLBiwZCpen6rvEK566XPrX4L2LT3MP4jpYyz+qyY0JDL2xdFJBkQYDVwhzv8XeBSYAtQBHyrIQHWZvronpakPHLtGb346webeHpJNo/fcHq95i0urWDm62tI6dCSn08e3EgRNl19O7fmgauG8/2XVjF7flaTb5GZ6dZnWSOM2FCvpKWqi4BF7uvzTzKNAnc2NDAT2Vq3iOfGcX14crGPbf4j9OkUfF3BH+dvJMdfxMu3jaO1FQs2iikjerA8u4A5H2czNrUjFw7p6nVInsnw+endsRUpHawT7VgQEz1iGG/cMiGV+GbNeGZJ8F07fbq1gOcycrh5fB/G25Vvo/rFZacxtIdTv9XYfUZGqopKZVm23+6yYoglLXPKurRLZProHry+cgcFR0rrnL6otJyZbzjFgj+7xIoFG1tiQhyP3zCGktJyzvvTIvr+/B0mPrCwST2Av2F3IYUl5XaBFEMsaZkGmTEpjZKySp7PrLtrpz/Oy2Kbv4jZV4+0YsEwWbX9AJVAWYWiQO6BYma9ua7JJC57Piv2WNIyDdK/S1suGNyFf2XmUFJ28q6dlmX7eS4jh1smpDLOTiBhM3t+FmUVxz8E3pR6jsnw+enfpQ1d2iV6HYoJEUtapsFmTErDf6SUN1burHF8UWk597yxlj6dWnHPJfYcXTg15Z5jyioq+XRrgd1lxRhLWqbBxvbtyMheSTyzJJuKGrp2evC9jWwvKOKPV42wn44Js5P1EBMfJ+wrLAlzNOG1dudBikorrBFGjLGkZRpMRJhxdho5/iIWfLnnuHGZPj//zNzGLRNSOdOueMOupp5jqn7CZ+qjS1mfe9CLsMKi6vksK46OLZa0TEhcMqwbvTu2Oq5rpyNHy7nn32tItWJBz9TUc8wfrx7JW3eeRTOBq5/M4L11Ie+wJiJk+Pyc1r0dHVo39zoUE0JWVmNCIq6Z8J2z+3LvW1+wIqeA9NSOPDhvIzv3F/Pa7eOtWNBDJ+s5Zu73J3L78yv57oufc/fXBvL98/vHTN98JWUVrNy2nxvH9fE6FBNidiYxIfP103vxwLsbuPGZ5ZSUO33enTOgM2ekdvQ4MlOTLm0Tefm2ccx6cx0PLdjE5n2H+ePVI0isVpwYjVZtP8DR8kqrz4pBVjxoQmb+F3sordBjCQtgeU5Bk3kmKBolJsTx52tGMvPiQby9ZhffmLMsJhpoZPryaSZwRl+7YIo1lrRMyMyen0V5tdaDJWWVTeaZoGglItx5Xn+evPF0Nu05xLTHor+BRma2n+EpSbRLTPA6FBNilrRMyDTlZ4JiwSXDuvHGd8cjwNefzGTe+uhsoFFUWs6q7QesaDBGWdIyIWO/Jh39hvZoz9zvT2RQt7bc8cLnPLpwM84PN0SPz3L2U16plrRilCUtEzL2a9KxoUvbRF6ZMY7po3rwp/c38aNXV9faRVekyfT5SYgT0vtYfVYsCjppiUiciKwSkf+5718UkSwRWS8iz4pIgjv8XBE5KCKr3b97Gyt4E1ns16RjR2JCHH/5xihmXjyIt1bv4to5y9h3KDoaaGT68hndqwMtm0d/K0hzovo0ef8hsAFo575/EbjRff0S8B3gCff9ElWdEpIITVSxX5OOHVUNNPolt+HHr65m2qNLefrmdIb1bO91aCd1sLiMdbkHuev8AV6HYhpJUHdaIpICXAY8UzVMVd9VF/ApkNI4IRpjvHTJsG68fsd4IPIbaHy6tYBKxX4/K4YFWzz4MHAPUFl9hFsseBMwL2DweBFZIyLvicjQmhYoIjNEZIWIrMjLy6tn2MaYcBrWsz1vRUEDjUyfnxbxzRjdO8nrUEwjqTNpicgUYJ+qrjzJJI8DH6vqEvf950AfVR0J/A2YW9NMqjpHVdNVNT05Obn+kRtjwqqqgcY0t4HGjyOwgUaGL58zUjvSIt7qs2JVMHdaE4GpIpIDvAKcLyIvAIjIr4Bk4CdVE6tqoaoedl+/CySISOdQB26MCb/EhDgedhtozI2wBhr+w0fZuOeQFQ3GuDqTlqrOUtUUVU0FrgUWquqNIvId4GLgOlU9VmwoIt3E7XVTRMa66/A3SvTGmLD7qgeNMWTtOcT0R5fyxS7ve9BYvrUAsPqsWNeQ57SeBLoCmdWatl8NrBeRNcAjwLUaiYXfxpgGuWRYd16/YzwKXP1EJvPW76lznsaU4cunTYt4RkRw60bTcBIJ+SQ9PV1XrFjhdRjGmFOwr7CEGc+vZPWOA8y8eBDfO7efJz9xcv5Di0jt1Jpnbzkj7Ov2ioisVNV0r+MIJ+sRwxjTIF3aOQ00po7swez5WZ400NhbWEJ23hHG268Uxzz7PS1jTIMlJsTx12tHMbBrG/70/ia2FRTx1E2n06VtYljWn+lzqs2tPiv22Z2WMSYkRITvnz+AJ24Yw8bd4W2gkeHLp33LBIZ0b1f3xCaqWdIyxoTU5OFOA41KdRpozP+i8RtoZGb7GZfWkWbNwl+XZsLLkpYxJuSG9WzP29+fyMBubbn9+ZU89tGWRutBY0dBETsKipnQzx4HbQosaRljGkWXdom8OmMcl7sNNH7y2ppGaaBRVZ9lv5/VNFhDDGNMo0lMiOORa0cxsEsbHlqwiRz/EebclE5y2xYhW0eGL5/ObVrQv0ubkC3TRC670zLGNCoR4a4LnAYaG3YXMu3RT/hyV2FIlq2qZGb7Gd+vkyfPhpnws6RljAmLycO788YdE5wGGk9mhKSBRnb+EfYWHrWiwSbEkpYxJmyqGmgM6NKGO15YyeOLGtZAI6Pq+Sx7qLjJsKRljAmrLu0SefX28UwZ0YM/zsvi7gY00Fjm89OjfSJ9OrUKcZQmUllDDGNM2FU10BjQpQ1/dhtoPFXPBhqVlU591nmDulh9VhNid1rGGE+ICD+4YACP3zCGL3cXMv2xpfVqoJG19xAFR0qtPquJsaRljPHUpcO78/rtE6ioVK5+MoP3g2ygYf0NNk2WtIwxnhue0p633AYat7+wkicW+epsoJHh85PaqRU9klqGKUoTCSxpGWMiQle3gcZlw7vz4LyN3P36Go6W19xAo6JSWb7Vz3jruqnJCTppiUiciKwSkf+57/uKyHIR2SIir4pIc3d4C/f9Fnd8aiPFboyJMYkJcfztutH8+MKBvPl5Ltc/vZz8w0dPmO6LXQc5VFJuRYNNUH3utH4IbAh4/yDwF1XtD+wHvu0O/zaw3x3+F3c6Y4wJiojwwwsH8Nj1Y/hi10GmPbqUDbuPb6Bhz2c1XUElLRFJAS4DnnHfC3A+8IY7yT+B6e7rae573PEXiLVHNcbU02UjnAYa5ZWVXPVEBgu+3HtsXIbPz8CubULah6GJDsHeaT0M3ANUuu87AQdUtdx9vxPo6b7uCewAcMcfdKc/jojMEJEVIrIiLy/v1KI3xsS04Sntefv7Z9G/SxtmPL+Cu176nAl/+JCPN+WRu7+YuatyvQ7RhFmdSUtEpgD7VHVlKFesqnNUNV1V05OTk0O5aGNMDOnaLpFXZ4xnZEp7/rt2N7sOlgBwpLSCWW+us8TVxARzpzURmCoiOcArOMWCfwWSRKSqR40UoOrIyQV6Abjj2wP+EMZsjGliWjaPI+/QiQ0yissqmD0/y4OIjFfqTFqqOktVU1Q1FbgWWKiqNwAfAVe7k30TeMt9/bb7Hnf8Qm2snyw1xjQZuw6UnGR4cZgjMV5qyHNaPwN+IiJbcOqs/u4O/zvQyR3+E+DnDQvRGGM46UPE9nBx01KvDnNVdRGwyH2dDYytYZoS4OshiM0YY46ZefEgZr25juKAHuFbJsQx8+JBHkZlws16eTfGRIXpo50GyrPnZ7HrQDE9kloy8+JBx4abpsGSljEmakwf3dOSVBNnfQ8aY4yJGpa0jDHGRA1LWsYYY6KGJS1jjDFRw5KWMcaYqCGR0FmFiOQB2xqwiM5AfojC8VKsbAfYtkSiWNkOsG2p0kdVm1TnrRGRtBpKRFaoarrXcTRUrGwH2LZEoljZDrBtacqseNAYY0zUsKRljDEmasRK0prjdQAhEivbAbYtkShWtgNsW5qsmKjTMsYY0zTEyp2WMcaYJsCSljHGmKhhScucEhFJEpHveR2H10TkXBH5n9dxVAncL5EWW0OIyA9EZIOIvFhteLqIPOJVXI1BRG4RkR5exxGpLGmZU5UENPmkFYGSqOd+EZG4xgklpL4HfE1Vb6gaICLxqrpCVX/gYVyN4RbAktZJRHzSEpHWIvKOiKwRkfUi8g0RyRGRzu74dBFZ5L6+T0SeFZFFIpItIhFxMItIqohsFJHnRGSTiLwoIheKyFIR2SwiY0Wko4jMFZG1IrJMREa480bkNgEPAP1EZLWIfBZ4RS8ij4rILe7r00VksYisFJH5ItLdq4CrE5GZVZ+niPxFRBa6r89399FFIpIpIp+LyOsi0sYdf4m7Pz8HrvRwE2pybL8As4E2IvKGG++LIiIA7nfoQXcbIvqXxkXkSSANeE9EDorI8yKyFHg+Wu4mT3Ieu9f97qwXkTniuBpIB150v1stvY494qhqRP8BVwFPB7xvD+QAnd336cAi9/V9QAbQAqdrFD+QEAHbkAqUA8NxLhRWAs8CAkwD5gJ/A37lTn8+sDoKtmm9+/pc4H8B4x7FuVpMcGNPdod/A3jW69gD4hwHvO6+XgJ86sb8K+BnwMdAa3f8z4B7gURgBzDA3X+vBW6713817JeDQIp73GUCZ7njcoB7vI63HtuV4x7/97nfn5Y1HXuR+neS81jHgPfPA5e7rxcB6V7HHKl/EX+nBawDvuZeFZ6tqgfrmP4dVT2qqvnAPqBr44cYlK2quk5VK4EvgA/VOULX4ZxozsI5cFHVhUAnEWnnzhup21SXQcAwYIF75f9LnBNopFgJnO5+zkdxTurpwNlAMTAEWOrG/k2gDzAYZ19udvffC14EXg+fqupO97hbjXOsVXnVk4ga7m1VLfY6iHqq6Tx2nogsF5F1OBeqQ70NMTrEex1AXVR1k4iMAS4FficiH+LctVQl3MRqsxwNeF1B5GxjYFyVAe8rcWIsC3LeSNqmKoH7A77aJwJ8oarjwx9S3VS1TES24twVZgBrgfOA/sBWYIGqXhc4j4iMCnOYDVXbsXMkzLGEStTFfZLz2J04d1Q7ROQ+TjyXmRpE/J2W24qmSFVfwCmjH4NTVHC6O8lVHoUWakuAG8Bp9QXkq2qhlwHV4RDQ1n29DRgiIi1EJAm4wB2eBSSLyHgAEUkQkUi7mlwC/BSnKHAJcAewClgGTBSR/nCsTmIgsBFIFZF+7vzXnbhITwXuFxMhTnIeA8h360qvDpjc9mEtIu2KvSbDgdkiUolzN/JdoCXwdxH5LU75byy4D3hWRNYCRTjFURFLVf1uQ5L1wHs4dTvrce5QVrnTlLoVy4+ISHuc4+1hnOLRSLEE+AWQqapHRKQEWKKqeW5jkpdFpIU77S/dK+YZwDsiUuTOHzEnmGr7pRjY63VMBqj5PDYd5zuzB/gsYNrngCdFpBgYH4VFoY3KunEyxhgTNSK+eNAYY4ypYknLGGNM1LCkZYwxJmpY0jLGGBM1LGkZY4yJGpa0jDHGRA1LWsYYY6LG/wfLXNa8cRDQEgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize scores\n",
    "summarize_scores('cnn', score, scores)\n",
    "# plot scores\n",
    "days = ['sun', 'mon', 'tue', 'wed', 'thr', 'fri', 'sat'] \n",
    "plt.plot(days, scores, marker='o', label='cnn') \n",
    "plt.title(\"Line Plot of RMSE per Day for Univariate CNN with 7-day Inputs. \\n\", loc='left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-headed CNN Model\n",
    "\n",
    "We can further extend the CNN model to have a separate sub-CNN model or head for each input variable, which we can refer to as a multi-headed CNN model.\n",
    "\n",
    "This requires a modification to the preparation of the model, and in turn, modification to the preparation of the training and test datasets.\n",
    "\n",
    "Starting with the model, we must define a separate CNN model for each of the eight input variables. The configuration of the model, including the number of layers and their hyperparameters, were also modified to better suit the new approach. \n",
    "\n",
    "We can loop over each variable and create a submodel that takes a one-dimensional sequence of 14 days of data and outputs a flat vector containing a summary of the learned features from the sequence. Each of these vectors can be merged via concatenation to make one very long vector that is then interpreted by some fully connected layers before a prediction is made.\n",
    "\n",
    "As we build up the submodels, we keep track of the input layers and flatten layers in lists. This is so that we can specify the inputs in the definition of the model object and use the list of flatten layers in the merge layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 333,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optimizer\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 334,
   "metadata": {},
   "outputs": [],
   "source": [
    "def split_dataset(data):\n",
    "  # split into standard weeks\n",
    "  train, test = data[1:-328], data[-328:-6]\n",
    "  # restructure into windows of weekly data\n",
    "  train = array(split(train, len(train)/7))\n",
    "  test = array(split(test, len(test)/7))\n",
    "  return train, test\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {},
   "outputs": [],
   "source": [
    "# summarize scores\n",
    "def summarize_scores(name, score, scores):\n",
    "    s_scores = ', '.join(['%.1f' % s for s in scores])\n",
    "    print('%s: [%.3f] %s' % (name, score, s_scores))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert history into inputs and outputs\n",
    "def to_supervised(train, n_input, n_out=7):\n",
    "  # flatten data\n",
    "  data = train.reshape((train.shape[0]*train.shape[1], train.shape[2]))\n",
    "  X, y = list(), list()\n",
    "  in_start = 0\n",
    "  # step over the entire history one time step at a time\n",
    "  for _ in range(len(data)):\n",
    "    # define the end of the input sequence\n",
    "    in_end = in_start + n_input\n",
    "    out_end = in_end + n_out\n",
    "    # ensure we have enough data for this instance\n",
    "    if out_end < len(data):\n",
    "     X.append(data[in_start:in_end, :])\n",
    "     y.append(data[in_end:out_end, 0])\n",
    "    # move along one time step\n",
    "    in_start += 1\n",
    "  return array(X), array(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 345,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot training history\n",
    "def plot_history(history):\n",
    "    # plot loss\n",
    "    plt.subplot(2, 1, 1) \n",
    "    plt.plot(history.history['loss'], label='train') \n",
    "    plt.plot(history.history['val_loss'], label='test') \n",
    "    plt.title('loss', y=0, loc='center') \n",
    "    plt.legend()\n",
    "    # plot rmse\n",
    "    plt.subplot(2, 1, 2) \n",
    "    plt.plot(history.history['rmse'], label='train') \n",
    "    plt.plot(history.history['val_rmse'], label='test') \n",
    "    plt.title('rmse', y=0, loc='center') \n",
    "    plt.legend()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train shape: (159, 7, 7), Test shape: (46, 7, 7)\n"
     ]
    }
   ],
   "source": [
    "train, test = split_dataset(dataset.values)\n",
    "\n",
    "print(f\"Train shape: {train.shape}, Test shape: {test.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 350,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X : torch.Size([1092, 14, 7]), Y : torch.Size([1092, 7])\n"
     ]
    }
   ],
   "source": [
    "train_x, train_y = to_supervised(train, 14)\n",
    "\n",
    "print(f\"X : {train_x.shape}, Y : {train_y.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 439,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "\n",
    "# Assuming your data (train_x, train_y) is already loaded and preprocessed\n",
    "\n",
    "class MultiHeadedCNN(nn.Module):\n",
    "    def __init__(self, n_timesteps, n_features, n_outputs):\n",
    "        super(MultiHeadedCNN, self).__init__()\n",
    "        self.heads = nn.ModuleList([nn.Sequential(\n",
    "            nn.Conv1d(in_channels=1, out_channels=32, kernel_size=3),\n",
    "            nn.ReLU(),\n",
    "            nn.Conv1d(in_channels=32, out_channels=32, kernel_size=3),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool1d(kernel_size=2),\n",
    "            nn.Flatten()\n",
    "        ) for _ in range(n_features)])\n",
    "\n",
    "        self.dense1 = nn.Linear(32 * n_features, 200)\n",
    "        self.dense2 = nn.Linear(200, 100)\n",
    "        self.out = nn.Linear(100, n_outputs)\n",
    "\n",
    "    def forward(self, x):\n",
    "        outputs = [head(x[:, i, :].unsqueeze(1)) for i, head in enumerate(self.heads)]\n",
    "        merged = torch.cat(outputs, dim=1)\n",
    "        # print(merged.shape)\n",
    "        out = torch.relu(self.dense1(merged))\n",
    "        out = torch.relu(self.dense2(out))\n",
    "        out = self.out(out)\n",
    "        return out\n",
    "\n",
    "# Model initialization\n",
    "n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]\n",
    "model = MultiHeadedCNN(n_timesteps, n_features, n_outputs)\n",
    "\n",
    "# Loss and optimizer\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 440,
   "metadata": {},
   "outputs": [],
   "source": [
    "# evaluate model and get scores\n",
    "n_input = 14"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 441,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/xf/gcbfh98s6n53wh5f2hpb84nh0000gn/T/ipykernel_5929/1520043488.py:2: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  train_x_tensor = torch.tensor(train_x, dtype=torch.float32)\n",
      "/var/folders/xf/gcbfh98s6n53wh5f2hpb84nh0000gn/T/ipykernel_5929/1520043488.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  train_y_tensor = torch.tensor(train_y, dtype=torch.float32)\n"
     ]
    }
   ],
   "source": [
    "# Converting data to PyTorch tensors\n",
    "train_x_tensor = torch.tensor(train_x, dtype=torch.float32)\n",
    "train_y_tensor = torch.tensor(train_y, dtype=torch.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 442,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1092, 14, 7])"
      ]
     },
     "execution_count": 442,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_x_tensor.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 443,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [1/1000], Loss: 11158590.0000\n",
      "Epoch [6/1000], Loss: 933244.5000\n",
      "Epoch [11/1000], Loss: 790186.8750\n",
      "Epoch [16/1000], Loss: 433752.8125\n",
      "Epoch [21/1000], Loss: 421936.4688\n",
      "Epoch [26/1000], Loss: 419620.8125\n",
      "Epoch [31/1000], Loss: 391790.5312\n",
      "Epoch [36/1000], Loss: 369922.5312\n",
      "Epoch [41/1000], Loss: 371029.5000\n",
      "Epoch [46/1000], Loss: 369502.1875\n",
      "Epoch [51/1000], Loss: 369686.8125\n",
      "Epoch [56/1000], Loss: 367986.3750\n",
      "Epoch [61/1000], Loss: 364748.9062\n",
      "Epoch [66/1000], Loss: 362624.4062\n",
      "Epoch [71/1000], Loss: 362029.3438\n",
      "Epoch [76/1000], Loss: 360486.3438\n",
      "Epoch [81/1000], Loss: 359203.4688\n",
      "Epoch [86/1000], Loss: 358289.0938\n",
      "Epoch [91/1000], Loss: 357216.0625\n",
      "Epoch [96/1000], Loss: 355425.3125\n",
      "Epoch [101/1000], Loss: 354042.3750\n",
      "Epoch [106/1000], Loss: 352853.2188\n",
      "Epoch [111/1000], Loss: 351562.0625\n",
      "Epoch [116/1000], Loss: 350249.8750\n",
      "Epoch [121/1000], Loss: 348866.9688\n",
      "Epoch [126/1000], Loss: 347324.5312\n",
      "Epoch [131/1000], Loss: 345752.9688\n",
      "Epoch [136/1000], Loss: 344086.7188\n",
      "Epoch [141/1000], Loss: 342328.4688\n",
      "Epoch [146/1000], Loss: 340371.7500\n",
      "Epoch [151/1000], Loss: 338024.7500\n",
      "Epoch [156/1000], Loss: 335666.5938\n",
      "Epoch [161/1000], Loss: 333123.6875\n",
      "Epoch [166/1000], Loss: 330359.1250\n",
      "Epoch [171/1000], Loss: 337288.0312\n",
      "Epoch [176/1000], Loss: 325871.3750\n",
      "Epoch [181/1000], Loss: 321723.5312\n",
      "Epoch [186/1000], Loss: 318240.9062\n",
      "Epoch [191/1000], Loss: 315093.3438\n",
      "Epoch [196/1000], Loss: 310939.0625\n",
      "Epoch [201/1000], Loss: 325091.2188\n",
      "Epoch [206/1000], Loss: 310105.5625\n",
      "Epoch [211/1000], Loss: 300564.5312\n",
      "Epoch [216/1000], Loss: 298611.3750\n",
      "Epoch [221/1000], Loss: 295557.2188\n",
      "Epoch [226/1000], Loss: 294690.5000\n",
      "Epoch [231/1000], Loss: 291256.4688\n",
      "Epoch [236/1000], Loss: 290177.5938\n",
      "Epoch [241/1000], Loss: 397628.7500\n",
      "Epoch [246/1000], Loss: 351149.9688\n",
      "Epoch [251/1000], Loss: 295486.0625\n",
      "Epoch [256/1000], Loss: 291269.7812\n",
      "Epoch [261/1000], Loss: 295923.7500\n",
      "Epoch [266/1000], Loss: 286381.3438\n",
      "Epoch [271/1000], Loss: 285858.1562\n",
      "Epoch [276/1000], Loss: 284137.9688\n",
      "Epoch [281/1000], Loss: 283586.1250\n",
      "Epoch [286/1000], Loss: 281724.5625\n",
      "Epoch [291/1000], Loss: 280991.4375\n",
      "Epoch [296/1000], Loss: 280349.0938\n",
      "Epoch [301/1000], Loss: 279669.5625\n",
      "Epoch [306/1000], Loss: 279207.3750\n",
      "Epoch [311/1000], Loss: 279316.8125\n",
      "Epoch [316/1000], Loss: 278470.4375\n",
      "Epoch [321/1000], Loss: 281410.3750\n",
      "Epoch [326/1000], Loss: 336236.2812\n",
      "Epoch [331/1000], Loss: 284821.2812\n",
      "Epoch [336/1000], Loss: 306312.9062\n",
      "Epoch [341/1000], Loss: 276921.5312\n",
      "Epoch [346/1000], Loss: 284108.5312\n",
      "Epoch [351/1000], Loss: 279153.1875\n",
      "Epoch [356/1000], Loss: 275600.1562\n",
      "Epoch [361/1000], Loss: 275717.2812\n",
      "Epoch [366/1000], Loss: 275738.4688\n",
      "Epoch [371/1000], Loss: 275116.4375\n",
      "Epoch [376/1000], Loss: 274305.6875\n",
      "Epoch [381/1000], Loss: 273371.6562\n",
      "Epoch [386/1000], Loss: 273031.3125\n",
      "Epoch [391/1000], Loss: 272559.4375\n",
      "Epoch [396/1000], Loss: 272616.5312\n",
      "Epoch [401/1000], Loss: 272631.4062\n",
      "Epoch [406/1000], Loss: 320329.0312\n",
      "Epoch [411/1000], Loss: 282510.6250\n",
      "Epoch [416/1000], Loss: 302596.1875\n",
      "Epoch [421/1000], Loss: 288593.3125\n",
      "Epoch [426/1000], Loss: 274093.3438\n",
      "Epoch [431/1000], Loss: 280288.5312\n",
      "Epoch [436/1000], Loss: 272121.5938\n",
      "Epoch [441/1000], Loss: 272258.1875\n",
      "Epoch [446/1000], Loss: 274044.4375\n",
      "Epoch [451/1000], Loss: 272972.3125\n",
      "Epoch [456/1000], Loss: 271271.1250\n",
      "Epoch [461/1000], Loss: 270570.2812\n",
      "Epoch [466/1000], Loss: 269716.5938\n",
      "Epoch [471/1000], Loss: 269962.6250\n",
      "Epoch [476/1000], Loss: 270033.2500\n",
      "Epoch [481/1000], Loss: 278218.3438\n",
      "Epoch [486/1000], Loss: 273158.2188\n",
      "Epoch [491/1000], Loss: 270332.9375\n",
      "Epoch [496/1000], Loss: 270364.5312\n",
      "Epoch [501/1000], Loss: 269725.4062\n",
      "Epoch [506/1000], Loss: 268178.2812\n",
      "Epoch [511/1000], Loss: 267547.5000\n",
      "Epoch [516/1000], Loss: 267745.9688\n",
      "Epoch [521/1000], Loss: 268940.1875\n",
      "Epoch [526/1000], Loss: 333943.5312\n",
      "Epoch [531/1000], Loss: 298136.4375\n",
      "Epoch [536/1000], Loss: 276093.2188\n",
      "Epoch [541/1000], Loss: 274249.8750\n",
      "Epoch [546/1000], Loss: 269486.8438\n",
      "Epoch [551/1000], Loss: 269145.6250\n",
      "Epoch [556/1000], Loss: 269112.5000\n",
      "Epoch [561/1000], Loss: 267361.4688\n",
      "Epoch [566/1000], Loss: 266604.9062\n",
      "Epoch [571/1000], Loss: 266268.5938\n",
      "Epoch [576/1000], Loss: 265858.1250\n",
      "Epoch [581/1000], Loss: 265862.5625\n",
      "Epoch [586/1000], Loss: 265491.5625\n",
      "Epoch [591/1000], Loss: 265448.2812\n",
      "Epoch [596/1000], Loss: 265457.8750\n",
      "Epoch [601/1000], Loss: 266015.8125\n",
      "Epoch [606/1000], Loss: 268062.8125\n",
      "Epoch [611/1000], Loss: 272818.1562\n",
      "Epoch [616/1000], Loss: 266102.7500\n",
      "Epoch [621/1000], Loss: 267014.8750\n",
      "Epoch [626/1000], Loss: 265478.4688\n",
      "Epoch [631/1000], Loss: 263761.1562\n",
      "Epoch [636/1000], Loss: 263542.0938\n",
      "Epoch [641/1000], Loss: 266513.0312\n",
      "Epoch [646/1000], Loss: 347493.7500\n",
      "Epoch [651/1000], Loss: 291419.8438\n",
      "Epoch [656/1000], Loss: 280848.0938\n",
      "Epoch [661/1000], Loss: 266446.1250\n",
      "Epoch [666/1000], Loss: 270958.2812\n",
      "Epoch [671/1000], Loss: 266473.3438\n",
      "Epoch [676/1000], Loss: 265435.9062\n",
      "Epoch [681/1000], Loss: 264878.2188\n",
      "Epoch [686/1000], Loss: 263921.8438\n",
      "Epoch [691/1000], Loss: 263141.5625\n",
      "Epoch [696/1000], Loss: 262775.5000\n",
      "Epoch [701/1000], Loss: 262356.9375\n",
      "Epoch [706/1000], Loss: 261368.3125\n",
      "Epoch [711/1000], Loss: 261566.4531\n",
      "Epoch [716/1000], Loss: 261523.2031\n",
      "Epoch [721/1000], Loss: 260790.1875\n",
      "Epoch [726/1000], Loss: 260518.6094\n",
      "Epoch [731/1000], Loss: 260413.7500\n",
      "Epoch [736/1000], Loss: 260912.9531\n",
      "Epoch [741/1000], Loss: 273645.2188\n",
      "Epoch [746/1000], Loss: 269628.6875\n",
      "Epoch [751/1000], Loss: 261284.3750\n",
      "Epoch [756/1000], Loss: 263112.9375\n",
      "Epoch [761/1000], Loss: 264070.4688\n",
      "Epoch [766/1000], Loss: 262795.3750\n",
      "Epoch [771/1000], Loss: 260013.0781\n",
      "Epoch [776/1000], Loss: 261692.4219\n",
      "Epoch [781/1000], Loss: 262811.3125\n",
      "Epoch [786/1000], Loss: 266461.6875\n",
      "Epoch [791/1000], Loss: 260064.8750\n",
      "Epoch [796/1000], Loss: 261462.1562\n",
      "Epoch [801/1000], Loss: 258749.6406\n",
      "Epoch [806/1000], Loss: 261056.3281\n",
      "Epoch [811/1000], Loss: 259911.5156\n",
      "Epoch [816/1000], Loss: 259677.9688\n",
      "Epoch [821/1000], Loss: 274021.4375\n",
      "Epoch [826/1000], Loss: 273172.1562\n",
      "Epoch [831/1000], Loss: 260001.2500\n",
      "Epoch [836/1000], Loss: 263692.4062\n",
      "Epoch [841/1000], Loss: 261463.4844\n",
      "Epoch [846/1000], Loss: 260783.4219\n",
      "Epoch [851/1000], Loss: 259142.9062\n",
      "Epoch [856/1000], Loss: 257565.0312\n",
      "Epoch [861/1000], Loss: 258894.7969\n",
      "Epoch [866/1000], Loss: 259116.8125\n",
      "Epoch [871/1000], Loss: 256984.5938\n",
      "Epoch [876/1000], Loss: 257699.5000\n",
      "Epoch [881/1000], Loss: 257951.6875\n",
      "Epoch [886/1000], Loss: 258632.2500\n",
      "Epoch [891/1000], Loss: 262222.8438\n",
      "Epoch [896/1000], Loss: 257889.7656\n",
      "Epoch [901/1000], Loss: 258726.4062\n",
      "Epoch [906/1000], Loss: 256586.5625\n",
      "Epoch [911/1000], Loss: 255840.2188\n",
      "Epoch [916/1000], Loss: 256403.4688\n",
      "Epoch [921/1000], Loss: 255505.1250\n",
      "Epoch [926/1000], Loss: 255176.4219\n",
      "Epoch [931/1000], Loss: 255144.1406\n",
      "Epoch [936/1000], Loss: 257729.4844\n",
      "Epoch [941/1000], Loss: 276489.3125\n",
      "Epoch [946/1000], Loss: 261006.5000\n",
      "Epoch [951/1000], Loss: 255909.0469\n",
      "Epoch [956/1000], Loss: 265125.0625\n",
      "Epoch [961/1000], Loss: 254489.4062\n",
      "Epoch [966/1000], Loss: 257437.2500\n",
      "Epoch [971/1000], Loss: 253733.3906\n",
      "Epoch [976/1000], Loss: 261098.2031\n",
      "Epoch [981/1000], Loss: 278652.7812\n",
      "Epoch [986/1000], Loss: 262957.8125\n",
      "Epoch [991/1000], Loss: 259413.4844\n",
      "Epoch [996/1000], Loss: 263587.3750\n"
     ]
    }
   ],
   "source": [
    "# Training loop\n",
    "epochs = 1000\n",
    "train_losses = []\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    optimizer.zero_grad()\n",
    "    outputs = model(train_x_tensor)\n",
    "    loss = criterion(outputs, train_y_tensor)\n",
    "    train_losses.append(loss.item())\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "\n",
    "    if epoch % 5 == 0:\n",
    "        print(f'Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 444,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAERCAYAAAB2CKBkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAdYUlEQVR4nO3df5RcZZ3n8fenqro7SScSkvQgJoEEjD8ARTAHRWZdRtRBnYE5M+NKlBlHOcPRFcXVMyMed9XRmXPGcVdXRmTEGXRkHVAZdTOYBV3AHzPrDzojIAkiIUESRNNAEkJ+dHdVffePe6u6qvp2p5P0TXX3/bzO6VO3bt2u+t7cTn3qqefe51FEYGZmxVXqdgFmZtZdDgIzs4JzEJiZFZyDwMys4BwEZmYF5yAwMyu4WRkEkq6XtFPSfVPY9pOS7k5/fi5p9zEo0cxs1tBsvI5A0suBp4EvRsQZh/F77wTOioi35lacmdksMytbBBHxPeDJ1nWSTpV0q6SNkr4v6XkZv7oOuPGYFGlmNktUul3ANLoOeFtEPCjpJcBngFc0HpR0MrAauKNL9ZmZzUhzIggkLQReBnxVUmN1X8dmlwA3R0TtWNZmZjbTzYkgIPmKa3dEvGiSbS4B3nFsyjEzmz1mZR9Bp4h4Ctgm6fUASpzZeDztLzge+EGXSjQzm7FmZRBIupHkTf25knZIugx4E3CZpHuATcDFLb9yCXBTzMZTpMzMcjYrTx81M7PpMytbBGZmNn1mXWfxsmXLYtWqVd0uw8xsVtm4cePjETGQ9disC4JVq1YxODjY7TLMzGYVSb+Y6DF/NWRmVnAOAjOzgnMQmJkVnIPAzKzgHARmZgXnIDAzKzgHgZlZwRUmCO56+Ek+8a0HGKnWu12KmdmMUpgg2PiLXVx9xxaqdQeBmVmrwgRBY7oaj7FnZtauOEGQJoFzwMysXXGCIG0TeNhtM7N2xQkCtwjMzDIVJgga3CAwM2tXmCCQmwRmZpmKEwTpbTgJzMzaFCcIGg0C54CZWZviBEF66xwwM2tXnCCQTx81M8tSoCBIbh0DZmbtihME6a0bBGZm7QoTBI0mgc8aMjNrV5ggaLQInANmZu2KEwTuIzAzy1ScIGgOOtflQszMZpjiBEGzReAkMDNrlVsQSLpe0k5J903wuCRdLWmLpHslnZ1XLeCzhszMJpJni+ALwIWTPP4aYE36czlwbY61uI/AzGwCuQVBRHwPeHKSTS4GvhiJHwKLJZ2YVz2emMbMLFs3+wiWA9tb7u9I140j6XJJg5IGh4aGjuzVPOicmVmmWdFZHBHXRcTaiFg7MDBwRM+hQ29iZlZI3QyCR4GVLfdXpOtyMTboXF6vYGY2O3UzCNYDf5yePfRSYE9EPJbXi3liGjOzbJW8nljSjcD5wDJJO4APAT0AEfF3wAbgtcAWYD/wlrxqSepJbt0iMDNrl1sQRMS6QzwewDvyev1OPn3UzCzbrOgsng4+fdTMLFtxgsAtAjOzTIUJggY3CMzM2hUmCBqnj7pNYGbWrjhBkN66RWBm1q44QeA+AjOzTMUJAk9MY2aWqThB4IlpzMwyFScI0lu3CMzM2hUnCDzEhJlZpsIEQaNN4K+GzMzaFSYI3CIwM8tWnCDodgFmZjNUcYLAE9OYmWUqThCkt+4jMDNrV5wgcB+BmVmm4gVBd8swM5txihMEnpjGzCxTYYIAtwjMzDIVJgg8xISZWbbiBIEnpjEzy1ScIEhv3SIwM2tXnCBwH4GZWabiBIEnpjEzy1ScIGheUOYkMDNrVZwgSG8dA2Zm7QoTBHiICTOzTLkGgaQLJT0gaYukqzIeP0nSnZJ+IuleSa/NrRZPTGNmlim3IJBUBq4BXgOcBqyTdFrHZv8V+EpEnAVcAnwmv3rSBeeAmVmbPFsE5wBbImJrRIwANwEXd2wTwDPS5eOAX+ZVjHPAzCxbnkGwHNjecn9Huq7Vh4FLJe0ANgDvzHoiSZdLGpQ0ODQ0dETFeGIaM7Ns3e4sXgd8ISJWAK8FbpA0rqaIuC4i1kbE2oGBgSN6obELypwEZmat8gyCR4GVLfdXpOtaXQZ8BSAifgDMA5blUYyHmDAzy5ZnENwFrJG0WlIvSWfw+o5tHgEuAJD0fJIgOLLvfg7BQ0yYmWXLLQgiogpcAdwG3E9ydtAmSR+RdFG62XuBP5V0D3Aj8CeR26W/npjGzCxLJc8nj4gNJJ3Ares+2LK8GTgvzxoa3CIwM8vW7c7iY6bRR+AkMDNrV5wgkK8sNjPLUpwgSG/dRWBm1q44QeBB58zMMhUnCJqDzpmZWaviBIEnpjEzy1SYIGhwDJiZtStMELiPwMwsW3GCwANRm5llKk4QuEVgZpapMEFgZmbZChMEHmvIzCxbcYIAz1BmZpalOEHgGcrMzDIVJwjSW7cIzMzaHTIIJJ0nqT9dvlTSJySdnH9p08t9BGZm2abSIrgW2C/pTJIZxR4CvphrVbnwDGVmZlmmEgTVdPrIi4FPR8Q1wKJ8y5p+0qG3MTMroqlMVblX0vuBS4GXSyoBPfmWNf3cR2Bmlm0qLYI3AMPAZRHxK2AF8PFcq8qBZygzM8s2pRYB8KmIqEl6DvA84MZ8y5p+bhGYmWWbSovge0CfpOXAt4A/Ar6QZ1F58FhDZmbZphIEioj9wO8Dn4mI1wNn5FvW9PMMZWZm2aYUBJLOBd4EfPMwfm9G8QxlZmbZpvKG/m7g/cDXI2KTpFOAO3OtKkeOATOzdofsLI6I7wLflbRQ0sKI2Aq8K//Sppc8L42ZWaapDDHxAkk/ATYBmyVtlHR6/qVNL58+amaWbSpfDX0WeE9EnBwRJ5EMM/G5fMuafj591Mws21SCoD8imn0CEfEdoH8qTy7pQkkPSNoi6aoJtvlPkjZL2iTpn6ZU9RHwoHNmZtmmckHZVkn/DbghvX8psPVQvySpDFwDvArYAdwlaX1EbG7ZZg1JR/R5EbFL0m8c7g5MlSemMTPLNpUWwVuBAeBrwD8Dy4C3TOH3zgG2RMTWiBgBbiIZuK7VnwLXRMQugIjYOdXCD5cnpjEzyzaVs4Z20XGWkKQvk4xBNJnlwPaW+zuAl3Rs85z0+f4NKAMfjohbO59I0uXA5QAnnXTSoUrO5D4CM7NsR3ph2LnT9PoVYA1wPrAO+JykxZ0bRcR1EbE2ItYODAwc2Su5j8DMLFOeVwg/Cqxsub8iXddqB7A+IkYjYhvwc5JgmHbCgw2ZmWWZ8KshSWdP9BBTm4/gLmCNpNUkAXAJ8MaObb5B0hL4vKRlJF8VHbIj+kj4rCEzs2yT9RH8j0ke+9mhnjgiqpKuAG4j+f7/+nSIio8AgxGxPn3s1ZI2AzXgzyLiiamXP3XuIzAzyzZhEETEbx3tk0fEBmBDx7oPtiwH8J70J1fNK4udBGZmbWbdKKJHykMNmZllK04QuK/YzCxTcYLAE9OYmWWaMAgkXdqyfF7HY1fkWVQuPDGNmVmmyVoErR24f9vx2FtzqCVXzfkIzMyszWRBoAmWs+7PeD591Mws22RBEBMsZ92f8TwxjZlZtskuKHuepHtJPkyfmi6T3j8l98qmmVsEZmbZJguC5x+zKo4BDzFhZpZtsiuLf9F6X9JS4OXAIxGxMe/CppsnpjEzyzbZ6aO3SDojXT4RuI/kbKEbJL372JQ3fTwxjZlZtsk6i1dHxH3p8luAb0fE75JMLjPrTh9tcIvAzKzdZEEw2rJ8AengcRGxF6jnWVQefB2BmVm2yTqLt0t6J8nkMWcDtwJIms/U5iOYUcb6CNwkMDNrNVmL4DLgdOBPgDdExO50/UuBz+db1vTzoHNmZtkmO2toJ/C2jPV3AnfmWVQePAy1mVm2yaaqXD/ZL0bERdNfTn7GJqbpciFmZjPMZH0E5wLbgRuBHzELxxdqNdYicBKYmbWaLAieCbyKZHL5NwLfBG6MiE3HorDp5j4CM7NsE3YWR0QtIm6NiDeTdBBvAb4zK+cioHXQOTMzazVZiwBJfcDrSFoFq4Crga/nX1aO3CQwM2szWWfxF4EzSC4k+4uWq4xnLcktAjOzTpO1CC4F9gFXAu/S2KW5AiIinpFzbdMuKbzbVZiZzSyTXUcw5ya2l+SzhszMOsy5N/vJuEVgZjZesYLAfQRmZuMUKwiQWwRmZh1yDQJJF0p6QNIWSVdNst0fSApJa/OsB/nKYjOzTrkFgaQycA3wGuA0YJ2k0zK2W0RyZtKP8qql+Vrg74bMzDrk2SI4B9gSEVsjYgS4Cbg4Y7uPAh8DDuZYC+A+AjOzLHkGwXKSQesadqTrmiSdDayMiG9O9kSSLpc0KGlwaGjoiAtK+ggcBWZmrbrWWSypBHwCeO+hto2I6yJibUSsHRgYOIrX9OmjZmad8gyCR4GVLfdXpOsaFpEMYfEdSQ+TDGy3Ps8OY+GvhszMOuUZBHcBayStltQLXAI0J7uJiD0RsSwiVkXEKuCHwEURMZhXQZJPHzUz65RbEEREFbgCuA24H/hKRGyS9BFJXZndLGkROAnMzFpNOgz10YqIDSSjl7au++AE256fZy1Ach2Bc8DMrE3Briw2M7NOxQoC+fRRM7NOBQsCnzVkZtapWEGA+wjMzDoVKwg8MY2Z2TjFCgLcIjAz61SsIHAfgZnZOIUKAjwxjZnZOIUKAnlCAjOzcYoVBLiPwMysU7GCwENMmJmNU6gg+PVTw3x5cPuhNzQzK5BCBYGZmY3nIDAzK7hCBoEHnjMzG1PIIKjVHQRmZg2FDIKqg8DMrMlBYGZWcIUMglrNQWBm1lDIIKjW690uwcxsxihkELiz2MxsTCGDwH0EZmZjChUEV16wBnCLwMysVaGCYPWyfsAtAjOzVoUKgnJJANTcWWxm1lSoIKikQTDq00fNzJoKFQRjLQIHgZlZQ65BIOlCSQ9I2iLpqozH3yNps6R7Jd0u6eQ866mUkyBwH4GZ2ZjcgkBSGbgGeA1wGrBO0mkdm/0EWBsRLwRuBv4mr3oAKqVkd91HYGY2Js8WwTnAlojYGhEjwE3Axa0bRMSdEbE/vftDYEWO9TT7CKruIzAza8ozCJYDrfNC7kjXTeQy4P9kPSDpckmDkgaHhoaOuCD3EZiZjTcjOoslXQqsBT6e9XhEXBcRayNi7cDAwBG/jvsIzMzGq+T43I8CK1vur0jXtZH0SuADwH+MiOEc66Hc7CNwEJiZNeTZIrgLWCNptaRe4BJgfesGks4CPgtcFBE7c6wFaL2OwJ3FZmYNuQVBRFSBK4DbgPuBr0TEJkkfkXRRutnHgYXAVyXdLWn9BE83LdxHYGY2Xp5fDRERG4ANHes+2LL8yjxfv1PzrCEHgZlZ04zoLD5W3CIwMxuvUEHQU0521y0CM7MxhQoCjz5qZjZeoYLAfQRmZuMVKgjcR2BmNl6hgqAx6JznIzAzG1OoICiX3UdgZtapUEHgPgIzs/EKFQTNPgJ/NWRm1lSoIHCLwMxsvEIFgSTKJfmsITOzFoUKAki+HnKLwMxsTOGCYKRa5++++xD7hqvdLsXMbEYoXBA0fP/Bx7tdgpnZjFDYIEj7jc3MCq9wQfCXv3cGAJffsJF/f2RXl6sxM+u+wgXBH754RXP5jvtznx3TzGzGK1wQzOspN5ef3D/SxUrMzGaGwgUBwO+88EQAHt11oMuVmJl1X65zFs9Un37j2YzWBtk6tK/bpZiZdV0hWwQAz1o8nx27DvAP/7qNB3+9t9vlmJl1TWGD4LknLOLAaI2P3rKZ//ylf+92OWZmXVPYIHj5cwaayw/ufJrzP34nr7v6+3x1cDv3P/YUQ3uHqdaKNW/BcLXGv9zzS4artW6XctQigoOjM2c/bvrxI3xr06+6XYZZJkXMrnF31q5dG4ODg9PyXPds3021XucDX7+PnXuHeXJf+1lEPWUhiUV9Feb1lJGgJFESlEqip1SiUhaVcomeUjKgXaUsShKVkiiXSsltWZSb65Lte8uip1yip1KiJ71fKY8t96TLPZWO++USvRURAQ8NPc2yhX2ctGQBS/p76a2UWNhXQUqulhup1hmt1envm1pX0Oe+t5W/2nA/q5f18/bzT+W3T38mx83vmZZ/61ZPHRzlfTffy8tOXcofnbtq2p8f4NrvPMS139nC99/3ilz24XDsPTjKCz78LQAe/uvXdbWWVtf/6zYOjNZ4x289u9ul2DEgaWNErM18rMhB0Onu7bvZvX+EzY89hRB7DoxSj2DvwSqjtTr1CCKgHkG1HlRrdaq1YLQe1OrJcq2ePFaPaLlfp1YPahHUasFILVk3Wq0zWgtGprHlUVJyiuy8njL7R6ocHK2zpL+Xpf29PPO4efSWS5RL4vgFvSjddn5vmZLg5o07+PVTw23Pt+L4+bxwxXEsXzyfgUV9aRCOhV4SXKVmSAbJp/HGv1PzNl1fq8O/3PNLfrD1ieZrrFwyn/+wZoCVxy9gSX8PA4v6AOgplyilodZ4/nIpCWKldZSa4Syq9Toj1TpP7hvhypvu5kBLi2BJfy9nn7SY55ywiOed+AyW9vfSVylRKqlZf60e9FZKlCVKpSS8gbbj+Ysn9jFSq7N6WT8LeitEBPN6ytQjqJRLiKQeCSIgCD73vW1c/2/bmrW8/sUr+M01y1ja38dx83solaCvktU4H3/5uzKuiM+6SF4dG3Zuc8+O3Vx5090AXHTms3jJKUs4/VnHMb+nzMJ5yX6VJIb2DnPqbyykVgv6+8rUI/kbK5c07jUgOcb1SP8GGPs3aLzNbB3ax4+3PcE5q5dy0tIFzKuU0mM5vubG8zX+75SU/L1NpF4Pdh8YZcNPH2P/SJXfO2s5S/v7mvOQdMNorc7Pf723+fdyKEN7h/nxtid5ySlLWDy/h0q6v40Rk49mXxwEM1ykwTJaqzNaTYJhtOVnpBot95PHR6p1Fs2rsHPvMBHBk/tGODhaZ99wleFqjQOjNSqlEgdHazyxb4QDIzX2HBilmobW7v2jBDA8mmxbDzhhUR8XPP8EbvjhL3Ld3wW9ZX73hc/iy4Pbc3uNhX0VnjGvwi/3HMztNQ7HmSsXc8/23d0uo01vuXRUH0Ia70mNN/zp0FMWlVKJWiQftDoHCl7QW27OK1IPmh+yqvXIrKGkdK5yJfOR9JRL1JN0mpjGglNpqKuxTGM4+7EPII3Xb3wYTJaD0Xq9WdOC3jKL5lWo1ZOpcqv1oLdcYuG8CvUIhKjW6jz21MHm7/SWS5RKsKC3wt6Do/RVynz4otPbLoo9HJMFQSFPH51pJNGTfrqmt9vVwEfTYTgaGkE1XE1bRfXkP2C1Hs2vn4LkE5k09h+m8R8lWTf2SXnx/F7m95b58wuf2/z/WK8HTx2scnC0xnC1ln7CD+rpO0Hj+etpC6PR2qiln9brkfxH702/ajt1oJ/eSol9IzV27RuhHsHB0Xp6W6NWD0olQcBIrU5EUC6VGK3VW54zeY1yKXlzKpfguPm9BEkr8eBoDUkMj9aabwjBWI0i+aRbKZd41Wkn0FMW2x7fx3C1zu79I1Rr0XyjG+14Q856n5rqh7bOzSLj2cqlEuc/d4Cf7tjDY3sOsn+kyki1zryectIaSOftOH5BL9t37ae3XGL/SI2SGm/AY8em8UZJy3Efe/Mc+6QvwbKFfbxo5WLu+NlORqpjb5SNecRHakkAVMql5lepjZ9aPdhzYLTt03GllLTeGtvO6ylz3qnLeHT3fjY/trfZ39X4W6mlf6MAymhLtbZeGv/mnS2beiR/i0HSwi2XaKu3J21196RfDy8/fj737tjNSLXeVutwNfng1trqPXlJP6sH+rlr25PN0BqpJR/6njpQZfWy/in9DRyuXFsEki4EPgWUgb+PiL/ueLwP+CLwYuAJ4A0R8fBkzzkXWwRmZnmbrEWQ21lDksrANcBrgNOAdZJO69jsMmBXRDwb+CTwsbzqMTOzbHmePnoOsCUitkbECHATcHHHNhcD/5gu3wxcoKweIzMzy02eQbAcaO0N3JGuy9wmIqrAHmBpjjWZmVmHWXFBmaTLJQ1KGhwaGup2OWZmc0qeQfAosLLl/op0XeY2kirAcSSdxm0i4rqIWBsRawcGBjofNjOzo5BnENwFrJG0WlIvcAmwvmOb9cCb0+U/BO6I2XZhg5nZLJfbdQQRUZV0BXAbyemj10fEJkkfAQYjYj3wD8ANkrYAT5KEhZmZHUO5XlAWERuADR3rPtiyfBB4fZ41mJnZ5GbdEBOShoAjHQNhGfD4NJYzG3ifi8H7XAxHs88nR0RmJ+usC4KjIWlwoivr5irvczF4n4shr32eFaePmplZfhwEZmYFV7QguK7bBXSB97kYvM/FkMs+F6qPwMzMxitai8DMzDo4CMzMCq4wQSDpQkkPSNoi6apu1zNdJK2UdKekzZI2SboyXb9E0rclPZjeHp+ul6Sr03+HeyWd3d09ODKSypJ+IumW9P5qST9K9+vL6bAmSOpL729JH1/V1cKPkKTFkm6W9DNJ90s6twDH+L+kf9P3SbpR0ry5eJwlXS9pp6T7WtYd9rGV9OZ0+wclvTnrtSZSiCCY4iQ5s1UVeG9EnAa8FHhHum9XAbdHxBrg9vQ+JP8Ga9Kfy4Frj33J0+JK4P6W+x8DPplOcrSLZNIjmDuTH30KuDUingecSbLvc/YYS1oOvAtYGxFnkAxTcwlz8zh/AbiwY91hHVtJS4APAS8hmQvmQ43wmJKImPM/wLnAbS333w+8v9t15bSv/xt4FfAAcGK67kTggXT5s8C6lu2b282WH5KRbG8HXgHcQjI97uNApfN4k4x1dW66XEm3U7f34TD39zhgW2fdc/wYN+YqWZIet1uA356rxxlYBdx3pMcWWAd8tmV923aH+ilEi4CpTZIz66XN4bOAHwEnRMRj6UO/Ak5Il+fCv8X/BP4caMz4vhTYHcnkRtC+T3Nh8qPVwBDw+fTrsL+X1M8cPsYR8Sjw34FHgMdIjttG5vZxbnW4x/aojnlRgmDOk7QQ+Gfg3RHxVOtjkXxEmBPnCUv6HWBnRGzsdi3HUAU4G7g2Is4C9jH2VQEwt44xQPq1xsUkIfgsoJ/xX58UwrE4tkUJgqlMkjNrSeohCYEvRcTX0tW/lnRi+viJwM50/Wz/tzgPuEjSwyTzYL+C5PvzxenkRtC+T1Oa/GiG2wHsiIgfpfdvJgmGuXqMAV4JbIuIoYgYBb5Gcuzn8nFudbjH9qiOeVGCYCqT5MxKkkQyr8P9EfGJlodaJ/15M0nfQWP9H6dnH7wU2NPSBJ3xIuL9EbEiIlaRHMc7IuJNwJ0kkxvB+P2d1ZMfRcSvgO2SnpuuugDYzBw9xqlHgJdKWpD+jTf2ec4e5w6He2xvA14t6fi0NfXqdN3UdLuT5Bh2xrwW+DnwEPCBbtczjfv1myTNxnuBu9Of15J8P3o78CDwf4El6fYiOYPqIeCnJGdldH0/jnDfzwduSZdPAX4MbAG+CvSl6+el97ekj5/S7bqPcF9fBAymx/kbwPFz/RgDfwH8DLgPuAHom4vHGbiRpB9klKT1d9mRHFvgren+bwHecjg1eIgJM7OCK8pXQ2ZmNgEHgZlZwTkIzMwKzkFgZlZwDgIzs4JzEJilJNUk3d3yM22j1Epa1Tq6pNlMUjn0JmaFcSAiXtTtIsyONbcIzA5B0sOS/kbSTyX9WNKz0/WrJN2Rjgt/u6ST0vUnSPq6pHvSn5elT1WW9Ll0jP1vSZqfbv8uJfNJ3Cvppi7tphWYg8BszPyOr4be0PLYnoh4AfBpktFPAf4W+MeIeCHwJeDqdP3VwHcj4kySMYE2pevXANdExOnAbuAP0vVXAWelz/O2fHbNbGK+stgsJenpiFiYsf5h4BURsTUd4O9XEbFU0uMkY8aPpusfi4hlkoaAFREx3PIcq4BvRzLRCJLeB/RExF9KuhV4mmToiG9ExNM576pZG7cIzKYmJlg+HMMtyzXG+uheRzJ+zNnAXS2ja5odEw4Cs6l5Q8vtD9Ll/0cyAirAm4Dvp8u3A2+H5tzKx030pJJKwMqIuBN4H8nwyeNaJWZ58icPszHzJd3dcv/WiGicQnq8pHtJPtWvS9e9k2TWsD8jmUHsLen6K4HrJF1G8sn/7SSjS2YpA/8rDQsBV0fE7mnaH7MpcR+B2SGkfQRrI+Lxbtdilgd/NWRmVnBuEZiZFZxbBGZmBecgMDMrOAeBmVnBOQjMzArOQWBmVnD/HzT1qVD1nIndAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plot the training loss\n",
    "plt.plot(train_losses)\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"MSE Loss\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 445,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1092, 14, 7])"
      ]
     },
     "execution_count": 445,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 446,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make a forecast\n",
    "def forecast(model, history, n_input):\n",
    "  # flatten data\n",
    "  data = array(history)\n",
    "  data = data.reshape((data.shape[0]*data.shape[1], data.shape[2]))\n",
    "  # retrieve last observations for input data\n",
    "  input_x = data[-n_input:, :]\n",
    "  # print(input_x.shape)\n",
    "  # reshape into n input arrays\n",
    "  input_x = torch.tensor(input_x, dtype=torch.float32).view(1, n_input, 7)\n",
    "  # print(input_x.shape)\n",
    "  # forecast the next week\n",
    "  yhat = model.forward(input_x)\n",
    "  # we only want the vector forecast\n",
    "  yhat = yhat[0]\n",
    "  return yhat\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 447,
   "metadata": {},
   "outputs": [],
   "source": [
    "# evaluate one or more weekly forecasts against expected values\n",
    "def evaluate_forecasts(actual, predicted):\n",
    "  scores = list()\n",
    "  # calculate an RMSE score for each day\n",
    "  for i in range(actual.shape[1]):\n",
    "    # calculate mse\n",
    "    mse = mean_squared_error(actual[:, i], predicted[:, i])\n",
    "    # calculate rmse\n",
    "    rmse = np.sqrt(mse)\n",
    "    # store\n",
    "    scores.append(rmse)\n",
    "    # calculate overall RMSE\n",
    "    s=0\n",
    "    for row in range(actual.shape[0]):\n",
    "        for col in range(actual.shape[1]):\n",
    "            s += (actual[row, col] - predicted[row, col])**2\n",
    "            score = np.sqrt(s / (actual.shape[0] * actual.shape[1]))\n",
    "  return score, scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 448,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Model Evaluation\n",
    "\n",
    "# evaluate a single model\n",
    "def evaluate_model(model, train, test, n_input):\n",
    " \n",
    "  # history is a list of weekly data\n",
    "  history = [x for x in train]\n",
    "  # walk-forward validation over each week\n",
    "  predictions = list()\n",
    "  for i in range(len(test)):\n",
    "    # predict the week\n",
    "    yhat_sequence = forecast(model, history, n_input)\n",
    "    # store the predictions\n",
    "    predictions.append(yhat_sequence.tolist())\n",
    "    # get real observation and add to history for predicting the next week\n",
    "    history.append(test[i, :])\n",
    "  # evaluate predictions days for each week\n",
    "  predictions = array(predictions)\n",
    "  # print(predictions)\n",
    "  score, scores = evaluate_forecasts(test[:, :, 0], predictions)\n",
    "  return score, scores\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 449,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2083.454  , 1604.132  , 2219.78   , 1777.18   , 1769.442  ,\n",
       "        1797.208  , 2336.9    ],\n",
       "       [2508.466  , 1518.684  , 2190.558  , 2390.4    , 2514.208  ,\n",
       "        2196.76   , 2150.112  ],\n",
       "       [1906.556  , 1881.098  , 2433.508  , 2351.728  , 2137.856  ,\n",
       "        1790.626  , 2611.502  ],\n",
       "       [2269.42   , 1930.51   , 2067.564  , 2760.902  , 2110.234  ,\n",
       "        2092.99   , 2842.728  ],\n",
       "       [1799.196  , 1693.778  , 2157.432  , 2903.014  , 2316.27   ,\n",
       "        2098.892  , 2712.986  ],\n",
       "       [1625.708  , 2582.238  , 2420.932  , 1948.196  , 1369.768  ,\n",
       "        1892.998  , 2351.556  ],\n",
       "       [1688.35   , 2097.132  , 1797.756  , 2054.56   , 1761.156  ,\n",
       "        1953.572  , 1983.728  ],\n",
       "       [2740.314  , 1474.036  , 1461.28   , 1914.278  , 1880.204  ,\n",
       "        2120.156  , 1765.056  ],\n",
       "       [ 711.666  , 1298.728  ,  766.906  ,  935.834  ,  682.88   ,\n",
       "        1133.726  ,  794.712  ],\n",
       "       [1372.472  , 2557.098  , 1760.112  , 1822.632  , 1951.09   ,\n",
       "        1645.424  , 2383.01   ],\n",
       "       [1566.714  , 1791.85   , 1542.65   , 2009.13   , 1562.682  ,\n",
       "        1663.384  ,  641.508  ],\n",
       "       [1262.9    , 1618.59   , 1822.162  , 1994.752  , 1679.884  ,\n",
       "        1651.284  , 1847.97   ],\n",
       "       [1798.214  , 1622.738  , 1906.454  , 1923.802  , 1687.44   ,\n",
       "        1648.622  , 1805.278  ],\n",
       "       [1690.388  , 1118.82   , 1825.852  , 1596.982  , 1649.22   ,\n",
       "        1233.35   , 1190.208  ],\n",
       "       [1603.152  , 1439.426  , 1512.704  , 1660.364  , 1872.178  ,\n",
       "        1562.062  , 1557.712  ],\n",
       "       [1476.106  , 1708.594  , 1833.628  , 1579.914  , 1460.904  ,\n",
       "        1180.536  , 1518.418  ],\n",
       "       [1542.742  , 1082.716  , 1250.94   ,  867.858  , 1103.724  ,\n",
       "        1118.628  , 1320.158  ],\n",
       "       [2187.91   , 1411.302  , 1516.96   , 1805.308  , 1299.25   ,\n",
       "        1644.004  , 1533.422  ],\n",
       "       [1566.51   , 1467.892  , 1571.368  , 2035.418  , 1554.266  ,\n",
       "        1722.146  , 1944.772  ],\n",
       "       [2100.426  , 1628.182  , 1421.184  , 1426.752  , 1153.898  ,\n",
       "        1294.396  , 2095.772  ],\n",
       "       [1487.556  , 1341.332  , 1198.626  , 1387.884  , 1323.342  ,\n",
       "        1398.164  , 2049.808  ],\n",
       "       [1665.7    , 1340.462  , 1268.964  ,  980.898  , 1394.186  ,\n",
       "        1420.554  , 2009.644  ],\n",
       "       [1746.792  , 1012.082  , 1362.034  , 1534.682  , 1215.19   ,\n",
       "        1470.672  , 1974.442  ],\n",
       "       [1674.356  , 1217.984  , 1223.966  , 1768.092  , 1028.018  ,\n",
       "        1332.316  , 1416.22   ],\n",
       "       [1672.55   , 1660.89   , 1520.588  , 1442.064  , 1095.78   ,\n",
       "        1099.438  ,  981.62   ],\n",
       "       [1139.964  , 1471.34   , 1368.008  , 1384.136  , 1349.002  ,\n",
       "        1140.928  , 1120.074  ],\n",
       "       [1538.088  , 1113.99   , 1005.934  ,  834.788  , 1212.652  ,\n",
       "        1083.478  , 1154.504  ],\n",
       "       [1193.572  , 1109.574  , 1382.114  , 1409.472  , 1635.816  ,\n",
       "         820.056  ,  806.352  ],\n",
       "       [ 812.038  ,  801.158  ,  855.824  , 1064.166  , 1164.446  ,\n",
       "        1176.158  ,  921.668  ],\n",
       "       [1130.41   ,  884.15   ,  952.36   ,  862.654  ,  549.104  ,\n",
       "         551.26   ,  554.616  ],\n",
       "       [ 555.664  ,  549.482  ,  615.212  ,  543.356  ,  533.476  ,\n",
       "         524.746  ,  527.476  ],\n",
       "       [ 528.46   ,  535.064  ,  544.758  ,  542.002  ,  529.698  ,\n",
       "         534.294  ,  554.27   ],\n",
       "       [1055.986  , 1129.73   , 1139.47   , 1304.64   , 1304.64   ,\n",
       "        1304.64   , 1304.64   ],\n",
       "       [1260.266  ,  765.658  ,  722.948  ,  980.408  ,  757.252  ,\n",
       "        2030.65   , 1442.856  ],\n",
       "       [1668.48   , 1494.802  , 1365.106  , 1593.318  , 1392.492  ,\n",
       "        1346.414  , 1743.84   ],\n",
       "       [1323.018  , 1422.876  , 1451.854  , 1303.812  , 1238.714  ,\n",
       "        1328.108  , 1463.31   ],\n",
       "       [1612.092  , 1076.32   , 1430.678  ,  986.094  , 1339.738  ,\n",
       "        1497.608  , 1247.946  ],\n",
       "       [ 919.6    , 1682.832  , 1111.376  , 1510.394  , 1449.862  ,\n",
       "        1327.992  ,  281.79   ],\n",
       "       [ 210.23999,  210.23999,  638.764  , 1684.426  , 1248.278  ,\n",
       "        1504.82   , 1923.254  ],\n",
       "       [1830.684  , 1775.96   , 1854.624  , 1315.456  , 1809.954  ,\n",
       "        1510.456  , 1611.78   ],\n",
       "       [1998.904  , 1622.59   , 1579.692  , 2073.886  , 2135.122  ,\n",
       "        1596.324  , 1000.014  ],\n",
       "       [1047.382  , 2713.884  , 1925.978  , 1727.964  , 1355.198  ,\n",
       "        1668.892  , 2293.236  ],\n",
       "       [2392.992  , 1163.292  ,  945.904  , 1260.278  , 1463.326  ,\n",
       "        1204.422  , 2152.7    ],\n",
       "       [1476.43   , 1383.18   , 1191.082  , 1293.298  , 2555.146  ,\n",
       "        1736.088  , 1805.408  ],\n",
       "       [1673.816  , 1825.54   , 2022.098  , 1946.534  , 1773.592  ,\n",
       "        1836.822  , 2023.174  ],\n",
       "       [2038.392  , 1747.382  , 1509.748  , 1582.032  , 1652.152  ,\n",
       "        1570.4    , 2197.006  ]], dtype=float32)"
      ]
     },
     "execution_count": 449,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test[:, :, 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 450,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cnn: [423.335] 450.7, 431.4, 384.1, 428.7, 397.0, 336.7, 512.9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAsWUlEQVR4nO3deXwU9f3H8dcnd7gSEsKVAOES5A6GcGlVrIJXpRZbb0QpPaz1VysqtT+1tra2tNV6tBbxFg9qKbZ4UBX5Fc0GCBAOOZQN4QhXsiHhyJ18f39kli4xkQ3Z3dnj83w89sHuzOzOZ9jNO5PPfmdGjDEopZQKL1F2F6CUUsr3NNyVUioMabgrpVQY0nBXSqkwpOGulFJhKMbuAgC6detmMjMz7S5DKaVCyrp160qNMWktzQuKcM/MzCQ/P9/uMpRSKqSIyO7W5mlbRimlwpCGu1JKhSENd6WUCkMa7kopFYY03JVSKgwFxWgZpZSKNEs3FDN/+Q72l1fROzmRuVOHMD0r3Wevr+GulFIBtnRDMfOWbKaqrgGA4vIq5i3ZDOCzgNe2jFJKBdj85TtOBrtbVV0D85fv8Nk6NNyVUirA9pdXtWn6mdBwV0qpAOudnNim6WdCw10ppQJs7tQhREfJKdMSY6OZO3WIz9ah4a6UUgE2PSudtE5xxMdEIUB6ciK/uXqkjpZRSqlQVnKshoNHa7h32lB+cMFAv6xD99yVUirAHIUuACYNTPXbOjTclVIqwBzOUjonxDC8dxe/rUPDXSmlAszhdDG+fyox0f6LYA13pZQKoOLyKopclUz0Y0sGNNyVUiqgHE7/99tBw10ppQLK4XSR0jGOIT06+3U9Gu5KKRUgxhgczlImDEghqtlBTL7mVbiLSJGIbBaRAhHJt6aliMgHIvKF9W9Xa7qIyBMislNENonIWH9ugFJKhYrdrkr2V1QzcWA3v6+rLXvuFxpjxhhjsq3H9wEfGWMGAx9ZjwEuBQZbtznAX3xVrFJKhbJAjG93a09b5irgJev+S8B0j+kvmyZ5QLKI9GrHepRSKizkOl107xzPgG4d/b4ub8PdAP8WkXUiMsea1sMYc8C6fxDoYd1PB/Z6PHefNe0UIjJHRPJFJL+kpOQMSldKqdDh7rdPGpiKiH/77eD9uWXONcYUi0h34AMR2e450xhjRMS0ZcXGmAXAAoDs7Ow2PVcppULNF4ePU3q8lkkB6LeDl3vuxphi69/DwD+AHOCQu91i/XvYWrwY6OPx9AxrmlJKRSz3+HZ/H7zkdtpwF5GOItLZfR+4BNgC/BOYaS02E3jbuv9P4GZr1MwEoMKjfaOUUhEp11lKRtdE+qR0CMj6vGnL9AD+YfWIYoDXjDHvi8haYLGI3AbsBr5tLf8ucBmwE6gEZvm8aqWUCiGNjYa8wjKmDu9x+oV95LThbowpBEa3MN0FXNTCdAPc7pPqlFIqDGw9cJSKqrqA9dtBj1BVSim/C3S/HTTclVLK73KdpQxI60iPLgkBW6eGu1JK+VFdQyNrdpUF5KhUTxruSinlR5uLKzhR28DEAYHrt4OGu1JK+ZW73z5hQEpA16vhrpRSfpTrLGVoz86kdooP6Ho13JVSyk9q6hvILzoS0FEybhruSinlJxv2lFNT3xjQ8e1uGu5KKeUnuU4XUQI5/QPbbwcNd6WU8ps8p4uR6UkkJcYGfN0a7kop5QeVtfVs2HuECTb020HDXSml/CK/6Ah1DcaWfjtouCullF84Cl3ERAnjMrvasn4Nd6WU8oNcp4sxfZLpEOftBe98S8NdKaV87Gh1HZv3lQf8fDKeNNyVUsrH1hSW0Whgok39dtBwV0opn3MUuoiLiSKrb7JtNWi4K6WUj+U6XWT360pCbLRtNWi4K6WUD5WdqGXbgaO29tuhDeEuItEiskFEllmPV4lIgXXbLyJLrekXiEiFx7wH/FS7UkoFndWF7kvq2ddvBy8ukO3hTmAb0AXAGHOee4aI/B1422PZVcaYK3xSoVJKhZBcp4sOcdGMykiytQ6v9txFJAO4HFjYwrwuwBRgqU8rU0qpEJTrLCWnfwqx0fZ2vb1d++PAPUBjC/OmAx8ZY456TJsoIhtF5D0RGd7SC4rIHBHJF5H8kpKSttSslFJB6fDRapwlJ2zvt4MX4S4iVwCHjTHrWlnkOuB1j8frgX7GmNHAk7SyR2+MWWCMyTbGZKelpbWtaqWUCkIOd789wNdLbYk3e+6TgW+ISBHwBjBFRF4FEJFuQA7wjnthY8xRY8xx6/67QKy1nFJKhbXcnS66JMQwrHcXu0s5fbgbY+YZYzKMMZnAtcAKY8yN1uwZwDJjTLV7eRHpKSJi3c+x1uHyeeVKKRVkcgtLmTAglegosbuUdo9zv5ZTWzLQFPhbRGQj8ARwrTHGtHM9SikV1PaWVbK3rMqW66W2pE2nKzPGrARWejy+oIVlngKeamddSikVUtz9drvO396cHqGqlFI+4HC6SO0Yx1k9OtldCqDhrpRS7WaMweF0MXFgKtZXjrbTcFdKqXbaVXqCg0erg6bfDhruSinVbrnO4Oq3g4a7Ukq1m6PQRa+kBDJTO9hdykka7kop1Q6NjYY8p4uJA4Kn3w4a7kop1S6fHz6G60RtUPXbQcNdKaXaJXen+/ztGu5KKRU2HIUu+qZ0IKNr8PTbQcNdKaXOWEOjIa/QFRSn+G1Ow10ppc7QZ/srOFZdH3QtGdBwV0qpM+Zwus/fruGulFJhI9fpYlD3TnTvkmB3KV+i4a6UUmegrqGRtUVlQdlvBw13pZQ6I5v2lVNZ26DhrpRS4SR3pwsRGN9fw10ppcJGrtPF2T270LVjnN2ltEjDXSml2qi6roF1e44EbUsG2niZvWCydEMx85fvYH95Fb2TE5k7dQjTs9LtLkspFQHW7zlCbX1jUI5vd/N6z11EokVkg4gssx6/KCK7RKTAuo2xpouIPCEiO0Vkk4iM9XXRSzcUM2/JZorLqzBAcXkV85ZsZumGYl+vSimlvsThdBEdJeT0T7G7lFa1pS1zJ7Ct2bS5xpgx1q3AmnYpMNi6zQH+0u4qm5m/fAdVdQ2nTKuqa2D+8h2+XpVSSn1JrtPFyPQkOifE2l1Kq7wKdxHJAC4HFnqx+FXAy6ZJHpAsIr3aUeOX7C+vanF6cXkV5ZW1vlyVUkqd4kRNPRv3lgd1Swa833N/HLgHaGw2/RGr9fKYiMRb09KBvR7L7LOmnUJE5ohIvojkl5SUtKno3smJrc6b8JuPmLdkE9sPHm3TayqllDfWFpVR32iC+stU8CLcReQK4LAxZl2zWfOAocA4IAW4ty0rNsYsMMZkG2Oy09LS2vJU5k4dQmJs9CnTEmOjm75UHZPOkvXFTHt8FdcucPD+lgPUNzT/naSUUmfG4XQRGy1k9wvefjt4N1pmMvANEbkMSAC6iMirxpgbrfk1IvICcLf1uBjo4/H8DGuaz7hHxbQ2WubeaUN5M38vrzh28/1X15OenMiNE/px7bg+QTsmVSkVGhyFLrL6diUxLvr0C9tIjDHeLyxyAXC3MeYKEelljDkgTRcNfAyoNsbcJyKXAz8CLgPGA08YY3K+6nWzs7NNfn7+mW5DqxoaDR9uO8SLnxbhKHQRHxPFVWN6M3NSJsN7J/l8fUqp8FZRWUfWL//NHVMG85OLz7K7HERknTEmu6V57RnnvkhE0gABCoDvW9PfpSnYdwKVwKx2rKNdoqOEqcN7MnV4T3YcPMZLjiL+sb6Yxfn7yMlMYeakTKYO70FMtB7LpZQ6vdW7XDQagr7fDm3cc/cXf+25t6Siso7F+Xt5Oa+IvWVV9EpKONmySe0Uf/oXUEpFrIf++RlvrN3DxgcvIT7G/rbMV+25R9wua1KHWL77tQGsvPtCnr05m4FpnZi/fAcTH13B3X/byJbiCrtLVEoFqbxCF9n9UoIi2E8nZE8/0F7RUcLFw3pw8bAefHGoqWWzZH0xb63bxzn9unLLpEymjehJrLZslFJA6fEath88xtypve0uxSsRG+6eBvfozK+mj2Tu1KG8tW4fLzuKuOP1DfToEs8N4/txXU5f0jpry0apSJZX2HRJvVDot4OG+ymSEmO57dz+zJqUycrPD/Ni7m7++MHnPLViJ1eM6sXMSZmM7pNsd5lKKRs4nC46xccwMj00RtppuLcgKkqYMrQHU4b2wFlynJdzi3hr3T6WbChmTJ9kZk3O5NIRvYiL0ZaNUpHC4XSR0z8lZEbXhUaVNhqY1olfXDWCvJ9dxINXDqOiqo473yhg8m9X8NgHn3P4aLXdJSql/OxgRTWFpSdCpiUDuufutc4Jscya3J+ZEzP5zxclvJRbxJ8++oI/r9zJZSN7ccukTLL6drW7TKWUHzgKSwGYMEDDPWxFRQkXDOnOBUO6s6v0BC87ingrfx9vF+xndEYSMydlcvmoXiExVEop5Z3cnS6SEmMZ1quL3aV4Tdsy7dC/W0cevHI4jp9dxMNXDed4TT13Ld7I5EdX8Md/7+CQtmyUCgu5ThcTB6QSFSV2l+I1DXcf6BQfw80TM/nwrvN55bYcRmck8+THO5n86Ap+9Np61u0uIxiOBFZKtd3eskqKy6uYNCh0WjKgbRmfEhHOG5zGeYPT2O06wSuO3byZv5dlmw4wIr0LMydmcuXo3iTEastGqVCR62zqt08MoX476J673/RL7cjPrxhG3ryL+NX0EdTUNTL3rU1MenQF85dv50BFy1eTUkoFl1yni26d4hnUvZPdpbSJ7rn7Wcf4GG6c0I8bxvcl1+nixdwi/rzSyTP/V8i04T2ZOSmT4iOV/P7fn7d4bnqllH2MMeQ6XUwamErT2c1Dh4Z7gIgIkwd1Y/Kgbuwtq+SVvN28sWYP72w+gADujnxxeRXzlmwG0IBXymbOkhOUHKsJ+uultkTbMjbok9KBn112Nqt/9nWSE2Np/lVrVV0D85fvsKU2pdR/Oax+eygdvOSme+42SoyLpqKqrsV5+8u1J2+3pRuKW72Uo4oMuU4X6cmJ9E3pYHcpbaZ77jbrnZzY4vSeSQkBrkR5WrqhmHlLNlNcXoXhv+2ypRt8ejlgFcQaGw15hS4mDAi9fjtouNtu7tQhJLYwNDJahKPVLe/VK/+bv3wHVXUNp0zTdllk2X7wGEcq60KyJQMa7rabnpXOb64eSXpyIgKkJycy57z+HDpWzawX1nKipt7uEiNScSttMW2XRY6T49tDNNy97rmLSDSQDxQbY64QkUVANlAHrAG+Z4ypE5ELgLeBXdZTlxhjHvZp1WFmelb6l3q5WX27cvtr65n9Uj4vzBqnBz4F0IdbD7U6r3NCDA2NhugQOgxdnZm8Qhf9u3VstXUa7Nqy534nsM3j8SJgKDASSARme8xbZYwZY9002M/ApSN78YdvjyZvl4sfvLqO2vpGu0uKCMs/O8gPFq2jT9dEEmJP/fGIEjhaXc/1z+a1umevwkN9QyOrC8tC6iyQzXkV7iKSAVwOLHRPM8a8ayw07bln+KfEyPXNrAwemT6Sj3eUcOcbG6hv0ID3p/c2H+D2ResZkZ7EO3eex6NXjzqlXfaHa0bzuxmj2FJcwbTH/8PSDcV6zqAwtWX/UY7V1Idsvx28b8s8DtwDdG4+Q0RigZto2rN3mygiG4H9wN3GmM9aeN4cYA5A375921Z1BLl+fF+q6xp4eNlW5r61iT9cMzqkzkwXKpZt2s+dbxQwpk8yL84aR+eE2BbbZQAT+qdy1+IC/ufNAj7cdohHpo8kqUOsDVUrf3H328N6z11ErgAOG2PWtbLIn4H/GGNWWY/XA/2MMaOBJ4GlLT3JGLPAGJNtjMlOS0tre+UR5NZz+zN36hD+saGY+5du1r1FH3u7oJgfv76Bc/p25aVbc+ic8NVB3Te1A29+byJzpw7h/S0Hmfr4f/h0Z2mAqlWB4HC6OKtHJ9I6x9tdyhnzpi0zGfiGiBQBbwBTRORVABF5EEgD7nIvbIw5aow5bt1/F4gVkW6+LjzS3H7hIG6/cCCvr9nLw8u2asD7yJL1+/jJmwXk9E/hxVvH0Sneuz9mo6OE2y8cxJIfTqJDfDQ3LFzNL5dtpbrZ8EkVemrrG1lbVMakgaEdW6cNd2PMPGNMhjEmE7gWWGGMuVFEZgNTgeuMMSebwSLSU6wR/yKSY63D5ZfqI8zdlwzh1sn9eeHTIn7/bx1v3V5/y9/LT/+2kYkDU3nhlhw6xLX9gO1RGcm8c8d53DyxH899sournvqUrfuP+qFaFSgFe8uprmsM2SGQbu0Z5/4M0ANwiEiBiDxgTZ8BbLF67k8A1xrdzfQJEeF/rzib63L68vTHTp7+eKfdJYWsN9bs4Z6/b+LcQd14buY4EuPOfKhpYlw0D181ghdmjaOsspbpT3/Kgv84aWzUj30ocjhdiDR9txLK2rSrYoxZCay07rf4XGPMU8BT7S1MtUxEeGT6CKqtoyUTYqO57dz+dpcVUhat3s39/9jC+Wel8debzvHZMQQXDunO8v/5GvOWbOLX725nxfbD/OHbY0gP0XHSkSrXWcrw3l1C/ktyPUI1BEVFCfNnjOLSET355bKtvLZ6j90lhYyXHUXc/48tTBnanQU3+y7Y3VI6xvHMjefwuxmj2Lyvacjk2wV6PppQUV3XwIY95SHfbwcN95AVEx3Fn67N4sIhady/dDP/2LDP7pKC3vOf7OKBtz/j4mE9+MuNY4mP8c9RvyLCt7P78N6dX+OsHp25840C7nh9AxWVeq6gYLdu9xFqGxpD7pJ6LdFwD2FxMVH85cZzmDgglZ8u3sh7mw/YXVLQWriqkIeXbWXa8J48fb3/gt1T39QOvDlnAndfchbvbT7AtD/pkMlgl+ssJTpKGNc/xe5S2k3DPcQlxEbz7M3ZZPXtyo/f2MDH2w/bXVLQeeb/nPzqnW1cPrIXT16fRVxM4D72MdFR/GjKYJb8cBKJcTpkMtjlOl2MzkjyekhsMNNwDwMd42N4YdY4hvbswvdeXUeu7h2e9PTHO3n0ve1cObo3f7p2DLHR9nzkWxoyue2ADpkMJsdr6tm0ryIs+u2g4R42uiTE8vKtOfRP7cjsl/PJLyqzuyTb/enDL5i/fAffzErnsW+PJsamYHdrPmTyqqd0yGQwWburjIZGE/Lj29003MNI145xvDI7h55dEpj1wlo27Su3uyRbGGP447938NiHn/OtsRn8/hr7g92Te8jkhUPT+PW727l+oZ5lMhjkOkuJi47inH5d7S7FJ4LnE698onvnBF6dPZ6kDrHc/Pwath+MrD/9jTHMX76DJ1bs5DvZfZg/Y1RQnntdh0wGn1yni7H9ksPm2gka7mGod3Iir82eQEJMNDcuXENhyXG7SwoIYwyPvredP690cl1OX35z9cigPoOmDpkMHuWVtWw9cJSJA8Kj3w4a7mGrb2oHXp09HmMMNyxczd6ySrtL8itjDL96Zxt//U8hN03oxyPTRwR1sHtqacikfikeWHmFZRgDkwaFR78dNNzD2qDunXh19ngqaxu4fmEeByuq7S7JL4wx/OJfW3nuk13cMimTh68aHjLB7tZ8yOT1OmQyoBzOUhJjoxmdkWx3KT6j4R7mzu7VhZdvzeHIiTquX5hHybEau0vyqcZGw/++vYUXc4uYfW5/HrxyGNZJSUOSe8jkTRN0yGQgOQpdZGd2DegxEP4WPluiWjW6TzIvzBrHgfJqbnpuNeWVtXaX5BONjYb7l27h1bw9fO/8Adx/+dkhHexuiXHR/HK6DpkMlJJjNXx+6HjYjG9303CPEOMyU3j25mwKS08w8/k1HKsO7S/tGhsN9y3ZxOtr9nD7hQO5b9rQsAh2T+4hkxcM0SGT/uQobLrcRChfL7UlGu4R5NzB3fjLDWP5bP9Rbn1xLZW19XaXdEYaGg1z39rE4vx9/Piiwdx9yZCwC3a3lI5x/PWmc/jdt3TIpL84nKV0TohheO8udpfiUxruEeais3vwp2uzWLf7CHNeXhdyX9jVNzTy08UF/H39Pn7y9bO46+KzwjbY3USEb4/TIZP+4nC6GN8/JagOdPOF8Noa5ZXLR/Vi/ozRfLKzlNsXrae2vvH0TwoC9Q2N/GTxRpYW7Gfu1CHc+fXBdpcUUDpk0veKy6soclUyMcz67aDhHrG+dU4Gv5o+go+2H+YnbxZQ3xDcAV/X0MidbxTwr437ue/Sodx+4SC7S7JFS0Mmf6VDJs+Ywxme/XbQcI9oN07ox88vP5t3Nh/gnr9vCtrRGLX1jfzotfW8s/kAP7/8bL5//kC7S7Kd55DJhZ/sYvrTOmTyTDicLrp2iGVIj852l+JzXoe7iESLyAYRWWY97i8iq0Vkp4i8KSJx1vR46/FOa36mn2pXPjD7vAHcdfFZLFlfzP++vYVgu5Z5TX0DP1y0nuWfHeLBK4cx+7wBdpcUNE4OmbxlHKXHdchkWxljcDhLmTgwNeQOevNGW/bc7wS2eTz+LfCYMWYQcAS4zZp+G3DEmv6YtZwKYndMGcT3zx/IotV7eOSdbUET8NV1Dfzg1fV8uO0QD181nFmT9ULgLblwaHeW/895J4dM3rBwNft1yORp7XZVsr+iOiz77eBluItIBnA5sNB6LMAU4C1rkZeA6db9q6zHWPMvknAfzhDiRIR7pw3hlkmZLPxkF4998LndJVFd18D3XlnHiu2HeeSbI7h5YqbdJQW11E7xJ4dMbtpXzlQdMnla7vHt4XC91JZ4ey2px4F7AHdjKhUoN8a4B0rvA9Kt++nAXgBjTL2IVFjLn/K1vojMAeYA9O3b9wzLV74iIjxwxTCqaht4YsVOEuNi+MEF9vS2q+sa+O7L+Xyys5Tffmsk3xmnnw9vuIdMThiQyk8WF3DnGwV8tO0wEwak8PTHTvaXV9E7OZG5U4cwPSv99C8Y5nKdLrp3jmdgWke7S/GL04a7iFwBHDbGrBORC3y1YmPMAmABQHZ2dnD0ASJcVJTw66tHUl3fwG/f305ibBS3BLgVUlXbwG0vrcVR6OJ33xrFNdl9Arr+cOAeMvmXlU7++MHn/HPj/pPzisurmLdkM0BEB7y7337uoG5he5yEN22ZycA3RKQIeIOmdsyfgGQRcf9yyADcfwMWA30ArPlJgMuHNSs/io4Sfn/NaC4Z1oOH/rWVxWv3BmzdJ2rqmfXiGvIKXfzx26M12NshJjqKOy4aTLfO8V+aV1XXwPzlO2yoKnh8cfg4pcdrw+aSei05bbgbY+YZYzKMMZnAtcAKY8wNwMfADGuxmcDb1v1/Wo+x5q8wwfINnfJKbHQUT16fxflnpXHvkk0B6d0er6ln1gtrWbOrjMe+M4ZvZmX4fZ2RoLSVs4BG+heu/x3fHp5fpkL7xrnfC9wlIjtp6qk/Z01/Dki1pt8F3Ne+EpUd4mOi+etN5zC+fwp3Ld7I+1sO+m1dx6rrmPn8GtbtOcIT12Vx1ZjIbRf4Wu/kxDZNjxS5zlIyuibSJ6WD3aX4TZvC3Riz0hhzhXW/0BiTY4wZZIy5xhhTY02vth4PsuYX+qNw5X8JsdEsnDmOURlJ3PH6elbuOOzzdVRU1XHTc2vYuLecp67L4opRvX2+jkg2d+oQEptdE1SA2y+M3APBGhsNeYVlYXlUqic9QlV9pU7xMbw4K4ezenTme6+sO/nnrC9UVNZx03Or+Wx/BX++YSyXjuzls9dWTaZnpfObq0eSnpyIAGmd44kS+Gjb4aA5niHQth44SkVVXVj320HDXXkhKTGWV24bT9+UDtz20lrW7znS7tc8cqKW6xfmsf3AMZ658RwuGd7TB5WqlkzPSufT+6aw69HLWXv/13ngyuF8tP0wC1ftsrs0W7h3UMLpYtgt0XBXXknpGMei2ePp3jmemc+vYUtxxRm/VtmJWq5fuJovDh/nrzedw0Vn9/Bhpep0bp7Yj0tH9OS3729n3e72/6IONbnOUgakdaRnUoLdpfiVhrvyWvcuCSz67gS6JMRy03Or+fzQsTa/RunxGq5/No/CkuM8e3M2Fw7t7odK1VcREX47YxS9kxO547X1YXPZRW/UNTSyZldZ2B6V6knDXbVJenIir313PLHRUdywcDW7Sk94/dySYzVctyCPItcJnps5jvPPSvNjpeqrdEmI5anrsyg5XsNPF2+MmP775uIKTtQ2hPUQSDcNd9Vm/VI7smj2eBoaDTc8m8e+I5Wnfc7ho9Vcu8DBviNVPH/LOM4dHP4/XMFuVEYy9192dkT139399gkDUmyuxP803NUZGdyjM6/clsPxmnpuWLiaQ0erW132YEU11y7I40BFNS/OGhcRe02hYuakTKYNj5z+e66zlKE9O5Pa6ctH7oYbDXd1xob3TuKlW3MoPVbDDQtX4zr+5aMh95dX8Z0FDg4fq+HlW3MYHwG9zlDi7r/3Sk7gx69vCOv+e019A/lFR8J+CKSbhrtql6y+XXn+lnHsO1LJjc+tOeWizfuOVPKdBQ7Kjtfy8m05ZGeG/5/CoSgpMZanrx/L4WPV3P238O2/b9hTTk19Y8T85ejtKX+VatX4AaksuCmb2S/lc+VTq6hrMBysqCZKhNhoePN7kxjdJ9nuMtVXcPffH/rXVp77ZFdYXvEq1+kiSiCnf2TsZOieu/KJr52Vxk0T+rKnrIoDFdUYoMEYDNKmETXKPu7++6PvbffJgWrBJs/pYkR6EkmJsXaXEhAa7spn3v/s0Jem1dQ3RvzpZUOFZ//9jtfCq/9eWVvPhr2R028HDXflQ62dRjbSTy8bSsK1/55fdIS6BhMx/XbQcFc+pKeXDQ/u/vuH2w7z3CfhMf7dUegiJkrI7tfV7lICRsNd+UxLp5dNjI1m7tQhNlWkzlS49d9znS7G9EmmY3zkjCHRcFc+0/z0sunJifzm6pERfa3OUOXuv/dMCv3++9HqOjbvKw/787c3Fzm/xlRATM9K1zAPE+7++4xncrn7bxt59ubskLyY9JrCMhoNTIiwcNc9d6VUq0b3SeZnId5/dxS6iIuJYmzfyOm3g4a7Uuo0bpmUydThPUK2/57rdJHdrysJzb4PCnenDXcRSRCRNSKyUUQ+E5FfWNNXiUiBddsvIkut6ReISIXHvAf8vA1KKT8SEX43Y3RI9t/LTtSy7cDRiOu3g3d77jXAFGPMaGAMME1EJhhjzjPGjDHGjAEcwBKP56xyzzPGPOzzqpVSAXXq+PdNITP+fXWhdUk9DfcvM02OWw9jrdvJd1ZEugBTgKX+KFApFRz+238/FDL991yniw5x0YzKSLa7lIDzqucuItEiUgAcBj4wxqz2mD0d+MgYc9Rj2kSrjfOeiAxv5TXniEi+iOSXlJScYflKqUDy7L9vCIH+e66zlJz+KcRGR97Xi15tsTGmwWq/ZAA5IjLCY/Z1wOsej9cD/aw2zpO0skdvjFlgjMk2xmSnpenl1pQKBZ799x8Fef/98NFqnCUnIuJ6qS1p068zY0w58DEwDUBEugE5wDseyxx1t3GMMe8CsdZySqkwkJQYy1Mh0H93WP32SDqfjCdvRsukiUiydT8RuBjYbs2eASwzxlR7LN9TrCMdRCTHWofLx3UrpWw0pk8y8y4N7v577k4XXRJiGNa7i92l2MKbI1R7AS+JSDRNQb3YGLPMmnct8Giz5WcAPxCReqAKuNYE6692pdQZmzU5k7xCF4++t51z+nUlK8gOEsotLGX8gFSio0LvqFpf8Ga0zCZjTJYxZpQxZoTn0EZjzAXGmPebLf+UMWa4MWa0MWaCMSbXH4UrpewlIswP0v773rJK9pZVReT4drfI+wpZKeUzSR2Cs/8e6f120HBXSrWTZ//9+U+L7C4HAIfTRWrHOM7q0cnuUmyj4a6UardZkzO5ZFgPHn1vGwV7y22txRiDw+liwsDUkDyLpa9ouCul2s3df+/RJYHbF62norLOtlp2lZ7g4NHqiO63g4a7UspHTum/v2Xf9VdzndpvBw13pZQPjemTzH2Xns0HW+3rvzsKXfTskkBmagdb1h8sNNyVUj516+RMLrap/97YaMhzupgU4f120HBXSvmYiPD7GaPp3jnw/ffPDx/DdaI2Ik/x25yGu1LK55I6xPL0DYHvv+fujNzztzen4a6U8gs7+u+OQhd9UzqQ0TWy++2g4a6U8qNA9t8bGg15ha6IHwLppuGulPKbQPbfP9tfwbHqem3JWDTclVJ+5dl/n+vH/rvDGt8eqRfnaE7DXSnld+7++7+3HuIFP/Xfc50uBnXvRPcuCX55/VCj4a6UCgh3//03fui/1zU0sraoTPvtHjTclVIB0XT+mVF075zAj17zbf99075yKmsbtCXjQcNdKRUwyR3ieOr6LA5W+Lb/7h7fPkHD/SQNd6VUQGX17cp9lw71af891+liWK8udO0Y55PXCwca7kqpgLvt3P58/Wzf9N+r6xpYt+eIDoFs5rThLiIJIrJGRDaKyGci8gtr+osisktECqzbGGu6iMgTIrJTRDaJyFg/b4NSKsSICL+/xjf99/V7jlBb36hfpjbjzZ57DTDFGDMaGANME5EJ1ry5xpgx1q3AmnYpMNi6zQH+4tuSlVLhwFf9d4fTRXSUkNM/xccVhrbThrtpctx6GGvdvupduAp42XpeHpAsIr3aX6pSKtz4ov+e63QxIj2Jzgmxvi0uxHnVcxeRaBEpAA4DHxhjVluzHrFaL4+JSLw1LR3Y6/H0fda05q85R0TyRSS/pKTkzLdAKRXSPPvvG9vYfz9RU8/GveXakmmBV+FujGkwxowBMoAcERkBzAOGAuOAFODetqzYGLPAGJNtjMlOS0trW9VKqbDh2X+//bX1VFR5339fW1RGfaPRcG9Bm0bLGGPKgY+BacaYA1brpQZ4AcixFisG+ng8LcOappRSLUruEMeTVv/9njb03x1OF7HRQnY/7bc3581omTQRSbbuJwIXA9vdfXRpupbVdGCL9ZR/Ajdbo2YmABXGmAN+qF0pFUbGWv335Z8d4sXcIq+e4yh0kdWnK4lx0f4tLgTFeLFML+AlEYmm6ZfBYmPMMhFZISJpgAAFwPet5d8FLgN2ApXALJ9XrZQKS7ed25+8Qhe/fncbY/t2ZXSf5FaXraisY0txBXdMGRy4AkPIacPdGLMJyGph+pRWljfA7e0vTSkVaZr676O5/IlPuP219bzz4/NISmx5FMzqXS4aDdpvb4UeoaqUCire9t8dhS7iY6IY0zc5sAWGCA13pVTQ8ab/7nC6GJeZQnyM9ttbouGulApKTePfu/Prd788/r30eA3bDx7T88l8BQ13pVRQcvffWxr/nldoXVJPw71VGu5KqaDVWv/d4XTRKT6GUelJNlcYvDTclVJBbWzfrtw7ran//pLVf3c4XeT0TyEmWiOsNfo/o5QKerPPa+q/P7xsK2N/+QGFpSfILypj6QY9+L01Gu5KqaAnIkwZ2h1joOxELQBHq+uZt2SzBnwrNNyVUiHh6Y+dXzrXeFVdA/OX77ClnmCn4a6UCgn7y6vaND3SabgrpUJC7+TENk2PdBruSqmQMHfqEBJjTz0aNTE2mrlTh9hUUXDz5qyQSillu+lZTRd0m798B/vLq+idnMjcqUNOTlen0nBXSoWM6VnpGuZe0raMUkqFIQ13pZQKQxruSikVhjTclVIqDGm4K6VUGJLWLmEV0CJESoDdZ/j0bkCpD8uxk25LcAqXbQmX7QDdFrd+xpi0lmYERbi3h4jkG2Oy7a7DF3RbglO4bEu4bAfotnhD2zJKKRWGNNyVUioMhUO4L7C7AB/SbQlO4bIt4bIdoNtyWiHfc1dKKfVl4bDnrpRSqhkNd6WUCkMa7uqMiEiyiPzQ7jrsJiIXiMgyu+tw83xfgq229hCRH4vINhFZ1Gx6tog8YVdd/iAit4hI7/a+joa7OlPJQMSHexBKpo3vi4hEn34p2/0QuNgYc4N7gojEGGPyjTE/trEuf7gFCP9wF5GOIvKOiGwUkS0i8h0RKRKRbtb8bBFZad1/SESeF5GVIlIoIkHxpotIpohsF5EXReRzEVkkIl8XkU9F5AsRyRGRFBFZKiKbRCRPREZZzw3KbQIeBQaKSIGIrPXcQxSRp0TkFuv+OSLyfyKyTkSWi0gvuwpuTkTmuv8/ReQxEVlh3Z9ivUeXiIhDRNaLyN9EpJM1f5r1fq4HrrZxE1py8n0B5gOdROQtq95FIiIA1s/Qb61tuMbGek9LRJ4BBgDviUiFiLwiIp8Cr4TKXyet5NgD1s/OFhFZIE1mANnAIutn68yvIWiMCeob8C3gWY/HSUAR0M16nA2stO4/BOQC8TQd0usCYoNgGzKBemAkTb9Q1wHPAwJcBSwFngQetJafAhSEwDZtse5fACzzmPcUTXsfsVbtadb07wDP2127R50TgL9Z91cBa6yaHwTuBf4DdLTm3ws8ACQAe4HB1vu32HPb7b618L5UABnW584BnGvNKwLusbveNmxXkfX5f8j6+Uls6bMXrLdWcizF4/ErwJXW/ZVAdnvXGfR77sBm4GJrL+M8Y0zFaZZ/xxhTY4wpBQ4DPfxfold2GWM2G2Magc+Aj0zTO7mZph/Ic2l6gzHGrABSRaSL9dxg3abTGQKMAD6w9iR/TlPQBIt1wDnW/3MNTeGXDZwHVAHDgE+t2mcC/YChNL2XX1jv36t2FN4Ga4wx+6zPXQFNnzW3N22pqP3+aYypsruINmopxy4UkdUispmmHbrhvlxh0F9mzxjzuYiMBS4DfiUiH9G0F+z+xZTQ7Ck1HvcbCJ5t9Kyr0eNxI0011nn53GDaJjfP9wP++54I8JkxZmLgSzo9Y0ydiOyi6a+MXGATcCEwCNgFfGCMuc7zOSIyJsBlttdXfXZOBLgWXwm5ulvJsdtp2kPfKyIP8eUsa5eg33O3vjWuNMa8SlMPcSxNf6KdYy3yLZtK87VVwA3QNMoBKDXGHLWzoNM4BnS27u8GholIvIgkAxdZ03cAaSIyEUBEYkXEp3snPrAKuJumFswq4PvABiAPmCwig+Bkz/QsYDuQKSIDredf9+WXtJXn+6KCRCs5BlBqfZczw2Nxn7yHwbYH2JKRwHwRaaRp7/YHQCLwnIj8kqb+VDh4CHheRDYBlTS1AYKWMcZlfSG8BXiPpt7zFpr2eDdYy9RaXxA9ISJJNH3eHqepLRUsVgH3Aw5jzAkRqQZWGWNKrC+FXxeReGvZn1t7YHOAd0Sk0np+0IRps/elCjhkd00KaDnHptP0M3MQWOux7IvAMyJSBUw80xaUnn5AKaXCUNC3ZZRSSrWdhrtSSoUhDXellApDGu5KKRWGNNyVUioMabgrpVQY0nBXSqkw9P8BDnEbtXWRlAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "# evaluate model and get scores\n",
    "n_input = 14\n",
    "score, scores = evaluate_model(model, train, test, n_input)\n",
    "# summarize scores\n",
    "summarize_scores('cnn', score, scores)\n",
    "# plot scores\n",
    "days = ['sun', 'mon', 'tue', 'wed', 'thr', 'fri', 'sat'] \n",
    "plt.plot(days, scores, marker='o', label='cnn') \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 451,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting torchviz\n",
      "  Downloading torchviz-0.0.2.tar.gz (4.9 kB)\n",
      "Requirement already satisfied: torch in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from torchviz) (2.0.1)\n",
      "Collecting graphviz\n",
      "  Downloading graphviz-0.20.1-py3-none-any.whl (47 kB)\n",
      "\u001b[K     |████████████████████████████████| 47 kB 2.5 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: jinja2 in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from torch->torchviz) (2.11.3)\n",
      "Requirement already satisfied: filelock in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from torch->torchviz) (3.6.0)\n",
      "Requirement already satisfied: networkx in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from torch->torchviz) (2.7.1)\n",
      "Requirement already satisfied: typing-extensions in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from torch->torchviz) (4.1.1)\n",
      "Requirement already satisfied: sympy in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from torch->torchviz) (1.10.1)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from jinja2->torch->torchviz) (2.0.1)\n",
      "Requirement already satisfied: mpmath>=0.19 in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (from sympy->torch->torchviz) (1.2.1)\n",
      "Building wheels for collected packages: torchviz\n",
      "  Building wheel for torchviz (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for torchviz: filename=torchviz-0.0.2-py3-none-any.whl size=4150 sha256=1c05e15ebc5054ff55f8177578c9d73a43f218c32b89700c7d04893a440da63d\n",
      "  Stored in directory: /Users/ajitkumarsingh/Library/Caches/pip/wheels/29/65/6e/db2515eb1dc760fecd36b40d54df65c1e18534013f1c037e2e\n",
      "Successfully built torchviz\n",
      "Installing collected packages: graphviz, torchviz\n",
      "Successfully installed graphviz-0.20.1 torchviz-0.0.2\n",
      "Requirement already satisfied: graphviz in /Users/ajitkumarsingh/opt/anaconda3/lib/python3.9/site-packages (0.20.1)\n"
     ]
    }
   ],
   "source": [
    "!pip install torchviz\n",
    "!pip install graphviz\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 453,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'model_visualization.png'"
      ]
     },
     "execution_count": 453,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "from torchviz import make_dot\n",
    "\n",
    "# Assuming 'model' is your PyTorch model and 'train_x_tensor' is your input tensor\n",
    "# Make a forward pass to get the output\n",
    "output = model(train_x_tensor)\n",
    "\n",
    "# Create a dot graph of the model\n",
    "# You need to provide a sample input tensor (for the shape)\n",
    "dot = make_dot(output, params=dict(list(model.named_parameters()) + [('train_x_tensor', train_x_tensor)]))\n",
    "\n",
    "# Save the dot graph to a file (or display it if you are in a Jupyter notebook)\n",
    "dot.render('model_visualization', format='png')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 458,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7fb74d49f1c0>"
      ]
     },
     "execution_count": 458,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAACaCAYAAABFT5jfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAABXtklEQVR4nO29eZAk2V3n+X1+RXjcV95XHV19lFql7lZTLWmGgUUnLLsMNoAhGEOmHVuNiRkzGGwOjWG2M7sYjHYxsRgY1iAZEkisgdQ6aGlawLQatdTqq/ru6rqrqyrvzMi4L7/97R8e76VHRkSekUd1+ccsqzI9/Pj5i/d+/vz3fu/7CKUUAQEBAQF3BsJhGxAQEBAQcHAETj8gICDgDiJw+gEBAQF3EIHTDwgICLiDCJx+QEBAwB1E4PQDAgIC7iAO3OkTQj5GCLlCCLlOCPnMQV8/ICAg4E6GHGSePiFEBHAVwIcBLAB4CcDHKaUXD8yIgICAgDuYg+7pnwVwnVJ6g1JqAvgbAD93wDYEBAQE3LEctNOfADDv+3uhvS0gICAg4ACQDtuAjRBCPgXgUwCgRiLvnZw5tqPjRUIQVcMwLQu6ZW/7OFkUoIZCO7rWZti2jZZp7fi4WDgEQggamo7tBt4IgJgaBiFkx9frBaV0R9dnhGUJiiyjqelwdhA2jIQUSKK4w6v1p9HS4O7wGEUUEQ4p0HQDlrv9o9k9D4qdlh0ASAJBJByGYZowbGfbx7F7HhSaYcBydlbyAgFiqgrLsqFZ228vkiAgEh5ce92V7QBiERWWbUPbQVtn39d+8sorrxQopUM9r7+vV+5mEcCU7+/J9jYOpfTzAD4PAHeffhf9949+GVq9Dtd1IYgi4uk0QAFD0yDJEpr1OpRQCFqziVRuCCNhET/z0GlcuDmHl1frqBYLIITAti3kxiZgaC04jgNZVqBrLYBSxFJpHFMF/OR77hvYjd5aXMbTswVUigUQQYBlGhienIZjWTB1HYqqorKWRzgSQatRhxqNIZ3J4MMnRhBSZPz3C7eQXysAoLBME6nsEARJgkAIHNtGs16D1HY2uVQC/8u7T0IdUEVqNJve9QsFUOpdP5nNAZRCUhS4joNmrQZBEOA4NkCB3PgE3pOU8a4TM/jvr1zAfLkOyzBg2zbCkQjUaAyiJEFvNWGZJkRRhG1ZyAwN4ycm05gcHR6I7ZZl49uvX8ZyqQrHsmDbFiLxBMJqBEQQYOo6LNOAY9sd9eKkCnzgXffgH1+/iOsVDVqjDtd1IIgS0kMjcBwberPpPWEphdW2/aFMGKdPzAzEdkop/u7VC5irtNbraSiEaDwBQRCgt1oAAQytBUmWYWgaMsOjGA8RfPjB03j58nW8WWyhXikDlMKlLnKj4zANHZZhQJRk2JYJx3GQzGRxd0zE+07fPTDbf3j+Mq5VdTSrVbjUBSEEqdwwqOtCbzUhKyE0qhUo4TC0ZgPJbA5DYRk/+9B9uD6/hOeXK6gUvPZimyZy4xMwdc0rB0WB1mgAhCAaT2BSFfGhB+4bWEfnmfOXcbnUQqtea/saAfF0BgQEeqsJSVbQrFehhDzbU7khZEIS/teH7sONhWU8t1xFeS3P2/rQ+CSvL7KioFmrgQgC1GgMUxEJH37w9MBs7wUhZLbfZwft9F8CcIoQchyes/9lAL+y2QHJTA6xRBJuu+LYpoV4OgNJliHJCmQlBMs0kchkISshwKjzYyOxOEKqCv9gtWWKEAQRaiyOSDwBSilkRQGc5sBvNhyJYFidAqUUhqZBbzahxmJQwiqo62BofBKtRh3Z0TGE1Shgr/cWBEHA8MQkXJcCoBAEEbrWgmkYCEciyMUmAFCv4ljGwG0ngoCh8Sm4rgvbMmEaOgRRQiQah661kBsbR7PuOf5ILAFRFAHfu0FmZBSO7QCUQhAFmIYBQ9OghMOIpzMAAOpSiMLgKz4BkBsdh+M4cGwLRttxxONx760olYLWqINSikgsAUmRAWj8+GQ2h1gyCUq98nUcB7ZmIqSGEY7G2ra7/KE7aNLDw7BtG67jwGi14Ng2Iuk0iCBAVkIwwips20IqN+zVeXO9zseSKajRaLvOE4BSr46HwojEYgAhoG67zkMfuO2JdAaRWLzdXlswdR3xVBqiJHltVlFgGTri6WkooTCgr9sejsQwPLneXgkhXgeBEIQjUajROCilXrnbg2+viUwW0URig69Zt10OhWAZOhLpGcih0AbbIxienOqynbouQmoE4UgMAIUoSYA1eNt3woE6fUqpTQj5twD+AYAI4IuU0gubHuTYEAkBIQTUsuBYFoxGDQBgmAZEUQJ1bC884NhwnXZIh7pwLdtzRu0nqq61YBuG1xDCYUiK4jUC2wKl238t3i6OafJr6I4NxzSg112o0Ri0ZsN76DgWZCkKQl1Ypg4Cz2k5pglRUiAI3hbXdWBpLdiWBUsgCClKe0/AtcwBW07gmgaE9vVt14FrWXCoASukwNJaEAlAXBeiKEIAhWMZAPWcoGtbIJILSWB3A1i6BtvQAdeBqka8rQLxtiE2SNPhWhaI7F2fgoLaFmzHgdmSYRs6CCgI9X7E9v5U9F7tqesA8OpcZ73R4doCVDUCQRAAQYBjGkB0cCEGAHBtGxAdSITAIV5ZOrYFoynDcRxQ2wIhBMR1IQmCV3ddr+5S6t2r33at2YRjGJ7/V8OQJBkQiFdn5J0GwTaHui6o65Wd0LbNNk3oDa+Xqxs6ZEWB6diQpXZ75R0dCtcyPafYtt3QNTiGDkpduKGQ95AgBHBsUGf7odtt2059vsa2YJsWjIbn2HVDhyTLoI4DidlurdvumN6bFLPdNHQ4hg6XunBMBaGwCoAAjgPXGbyv2QkHmrK5Ux588EH6xD/8j67tpmmiXq8jm812faZIErLpFFqajmq9wXwOZ3l5GWOjY13bI+EQUonEwGw3DAPFSq3rOtyGsbGexw1n0iCEIF8qY+N3Uy6XoaoqwhvCOAIhGM5mPGc0ABzHwVqpDLdH3VhbW0M6nYYkbegvUCCViEENh1EsV2DanY1S13VomoZ0Ot11zlwqCUUZTGyZUoq1Ygl2j7h8qVRCNBpFqMfYTUxVkYjHUK7WoBmdb06u42KtsIaRkZENF/PuOaKqA7O9VKnC6BHb1jQNhmEilUp2fRZWFKSTCTSaLdRbrQ3nBFZWete3WERFIjaYBy6lFNV6HS29+63TMi1Ua1Xkcrmuz2RRRC6Thm4YKFfrvdtrD9vVkIJUIjGwEEm1VkdT737zsW0HpVIJw8Pd4XFJFDG0he2jo6NdNrLva5/DO69QSh/u9dmRG8j1I4oixnsUtmmaKIoCxnp8xohGVEQjnY2RUgpLa2FsOLevBQ4AoVAI4yO97bN1DWNDm9swNtSjgRAgFotBHZCT6YcoihjtcX0AII6NbDa7qZPOZbode6PRgGFEej6oBwkhBMO53teQCRCNRhGJRPoen04msNF627ZBXLtnXRwkhBBk06men2mahkajgaGh/jbEY1HEY9GObZRSOIZ2ILanEgmkevSbTNOEIpJN26saDneNSfH2ukVbGQTJRBzJRLxru+M4ELb47jezfXx4aN9t3ym3pQyDaZqYm5vr6glvheu6mJubg3OIr1eUUiwsLEDv0avY6rjFxUVomrb1zvsEpRT5fB61Wm3Hx66tre3quEFBKUWhUEC1Wt3xsY1GAysrKzuub4NE13UsLCzs2AbDMDA/P3+otu/WBsuydtXOB4lt25idnYW7g4wu4Gj4mn4ceafvuq43IOf7abVaeP7557u2O47DK0iv42zbxjPPPAPbtrs+2+mXuh0opT1tfPHFF9FsNnvaQNsDb70+e/XVV1Eul/set9+2u66LCxcuIJ/P9y37fvd87do13gj6fWf7bfvly5extLS0Y9tLpRJef/31nnVqP2zvdZ1arYaXXnqprw39jmu1WnjhhRcOrNx72dBoNPDiiy9uWu69jjMMA88999yB2A709hm6ruPZZ5/t+RnzGUfB1+yEIx3Tf/jhh+kXv/hFNJveaDchhFcSwzC6Ytuu62J8fBzHjx/HpUuXUKlUus6paVpXeIRSilwuh7vvHkz6GqNQKODSpUtd8W9N0xAOr+fVs/sSBAFnz55FPp/HtWvX2hkx6+i6jlAo1HWcJEl4+OGHB/oa2Wg08Nprr3Hb2bUMw4Asy95AXdtm1mjPnj0LTdPwxhtvdB1nttM0RVHsOI5SirNnz3aPEeyBVquFV155Zds2OI6Ds2fPQhRFvPjii13lyxpxKBTiGT2A1ws8c+YMksnuOPtuoZTitddeg9EeV9iqzjuOg+PHj2N8fByvvPIKrPZ4QK/j/LY7joO77roLo6OjA7X99ddf52+x27F9ZmYGk5OTePXVV2Ga5raPO3HiBMbHxwdmOwBcvnwZpVLJGyhv2wD09xnZbBb33HMPLly4wN9it3Pc8PAw7rrrroHavpHbNqYPeI68Xq+jXq9DFEW0Wi1EIhEQQlCpVKBpGiKRCAzDwHA71Q0A7+WUy2VIksSPEwQB1WoVuq4jHA7Dtm1kMhl+3CCxbRuUUszNzSEUCsEwDIiiiFAohHK5DNu2IQgCRFGEJEkYGRnhTsZ1XSwuLvLjJEmCoigolUqwbS8rSRAEyLLcPcA4AFzXy7Oem5uDoijcJlbW7O2CfTY2NsZt3+w4x/FedxVFgeu6fQe0B2m767qwbRvRaJTb7rouQqEQbNvG+Pg4dzKEECwtLfFBcV3XEYvFYFmWF6e1LKiqCk3TMDU1xe9n0PaXy2VomsbrfLSdhlmpVKDrOiKRCDRNw9jYGK+7rutibW2N16tWq4VYLMZ7xoZhIBKJoNVq7avtpVIJhmF02OC6bpftExMT3AbXdVEsFmGaZtdx5XKZ265pGsbHx/etvTabTVSr1Q6fQQjhPkNVVZimiaGhIW674zio1+uo1Wo9j2M+ih23H7bvhCPv9MPhMJLJJBLtzBp/b4UNbEmSxHturFcQDocRi8UQjXoDW5ZlwTAMxDZkK7Dj9mNwNBQKQVEUzMzMdNluWRZM0+T2AZ4jJIRAUZSO49iDr1ePkhDSMxtlr0iSBEmSuA1+2AM1Fovx+/E/hPzHUUpRq9WQ6JNpIUnSwAe62EO0l+2u66LRaHg5++3rCoIAQRBACIEsy5iYWFcGqdfrUFW1602EfZfyPuTqh8NhZLNZ3mP015tarYaRkRH+luL//sPhMO84AN4bD+tk9LJ9P+pNOBxGLpfra/vw8DB/09po+9DQUEcvmRDS1ctnx23cPghUVUUikUA8Hu+yvdlsIpfLQZblnr7G76NM04RlWR1tm51vv3zNTjjy4Z2XXnqp7+emaWJpaQkTExMdjc//igV4T/CFhQWMj49vmnUyaOezWdmygdlkMskr2V72AwZr/1b1YmVlBYqiIJPJbLrf6uoqZFnecr+DtD2fz0MQhJ4phH6q1SqazSbGxsY2te8gbW82myiVSpiYmNg0RbfZbKJYLGJycnLT/Q7Sdk3TkM/nMTU1talN293vINurZVlYXFzc0tf0228j+53Rs1l458gP5LL4Wq+fUCiEiYkJLC4udsQy/cc5jtPh8Dc730HaLggCJiYmUK1WUa/Xe+7PHH4qleI904Oyf6trjY6OwrIslEqlnvsDnQ7/KNk+PDzs5fOvrfW1vVKpoNVqYXx8nL8FHAXbo9EostksFhcXeU90oy3swTA1NQVRFI+M7aqqYmRkBPPz8zwEt9EWXdextrZ24LZvZb+iKB2+xm8D+505/MnJyQP3NTvhyDv9rVAUBZOTk1hcXISu6x2vlaZpdjn8owRz/LVaDdVqlY/qU0ph2zbv4fvDKEcFQghGRkZg2zaKxWJHfNhxHKysrHQ4/KMEIQRDQ0MQBIFnIjH88fSteviHASEEkYg332F+fp6PNQDrYcBSqbRlD/8wIITwENT8/DxM0+xor61WC6urq5icnOxKYjgK+B3/Rl+j6zp3+PsR8hskRz6m34uNr2GSJGFychKf//zn8cEPfhD33HMPGo0GPve5z+G3fuu3eEhn43G1egPLpTJ6Tpvtg0CAExNjcFwXt5ZWQLd5rCIKODbR7UQIIRgfH8ff/u3fIhaL4cMf/jBs28bv//7v49d+7dc64ot+NF3H3OrajmwnoJgZHYEsS7ixsARnm5E9kQDHJ8Z6NsTh4WE8++yzuHXrFn71V38VlFI8+uij+Imf+Ancf//9PW23bBs3F5e3XXYeFBO5LGLRCG4uLMHcpiIiAcWxsZGeYb1sNouLFy/isccew6//+q+DEIJvfetbSCQS+NCHPtTTdtd1cXNxGba7k7AoxVAygUwqiYWVVTSNbSoyUorJ4SyiPSaTRSIRqKqKz372s/iP//E/QlEUPPfcc5idncXHP/5x/qa4kdnFZeg7UOIEgGQkjNGhHPLFEsqN1tYHeMZjOJVEOtk9WyscDiOTyeCzn/0sfvM3fxPxeBwXL17E008/jX/9r/81H6/YyOJqHg19Z5IjEUXG1NgIKtUaVitVbLe9pKIqRodyXXbIsozx8XH84R/+If7lv/yXGB8fx8rKCr70pS/ht37rt/j4oh9CCPLFEkr1nWnuMJ/R0nQs5Atc4mEroqHNHzq3pdOfn5/HyspKx6sSpRSnTp3C1atX0Wg0UK/X8eCDD+LKlSsd+wDeQ+KBBx7AwloRL681YWiap+xnmYin0jB0DYIgQpJlaI06RFn2lDFDIURkCWPZDEzLwotLFWiGAUEQYZsGYqk0DK3FNUL0VhOiJEEQRMQFF9NjIyiXy7hx4wYPGTC7crkc8vk8Xn75ZZimiVOnTmFxcRErKysdtgPAgw8+iGKlhnMrNWiaZ6tlGkikM3DamQGCKEJrNCBKEizLS1WMqBEko3UkY1G8tFhEw7BBBALbshCJxQECiKLk5UhrLYiSDAJAlQSMD2Xh2DYuXbrUZTsbwH3ppZdAKcXQ0BCazSZefvnlDufjui7OnDkD3TTx0nIVTV2HIAiwLQvRRBK2ZUKWFVB4SpKECHAdBxQUsXgCIinjhBrGa4trKBntzAnbhhJWIQgCJFn2tImYJhN1EVIUpOMxJGPA66+/3mW74zjI5XI4d+4cRFGE4zhQVZXbzvZzXRf33HMPwmEVLy8WUTO8V3zbsqBGowAIJFmGY9uwLBO0LZTn2DbiqTTu1Q1kUkmcX8hjWfcymlzHgShJCKmeU7ctE47tQBAEuK6LsKpClkQcC4fx2muvdQwssmyj06dP49y5cwiHw8jn85iYmMArr7zStd/U1BSGh4fx2uIaioaXzePYNmQlBFGWIEmyJxBGXTiWzdUi46k0JqQaRodyuLKUx9sNC7ZlgboUFBTReMLLP29r6LDrhdUI3mVaSCcTePPNN2GaZlc7PHPmDF5++WUkEgksLy/jnnvuwWuvvda1XzabxfHjx3FhMY9FzYXj2Dw8FFLVdv33sn5MQ4coSjANHZF4AlliY2psBLP5Al4veoKFhBDYto14MgXbMuG6lNdDQgiUcBjT1TpGh3K4evUqarVaR2eNUop3v/vdOH/+PJaXl1EsFvHAAw/g/PnzXbZHo1GcPn0a15fyuNqwYRoGAK8+ReMJuI4Dx7EhyQr0ZhOiLMHUdYQjUcQFF1Ojw8iXK3hhpQpD17mviaczMHStnUAhej5LFCHJCnLC5tlBt6XT1zQNrVYL1WqVO35RFHkaHnv9UhQF8/PzPDvDNE3EYjGk02nvSyEEkqJAb7Xg2BZs24ah65BkBZZptlP7KFzDgG1ZCEeiEHxZHJIsg5gmHMcG2g4KhMA0DMjtfHrXcUCIAKHdS9Z1HYZhoFwu8woiSRJc10U8Hsfy8jJ0XYckSVhaWuL357bFzYaGhrwwEAEkWQE0DY7jTQDR2w8c2/LuxaUuXMuEqeuIJpIQJCY+RzyHblPeWNk5bUuDKEmew3UdEBAIotdzYBlH5XKZO3sWQkilUlhdXeVpqWwGK/ucpUaydDVRkkAEYX0STFt613FdOJbVTgl1YGiaZ7so8U4aEUSIktfA/ZN7DF0HEQTPdrruVAEv5GRZFsrlsied23b+giAglUqhUCjw3HgWrmLptK1WC9PT0+2ccRWCJEG0KSzLhOs40DUN4UgUruu2643n1E1dgxqNe989t12AJIvQWk1Q14VA0VYytQBQEIHAbTtkoS0WyEKV5XIZlmXxNy5JkhCJRFCr1VAoFGBZFqrVKorFIs9gajQamJmZ4bnzgiBClAUYmqc86uo6wmLECylaFigoF/dTwioEUQQh3lsVIQSSEoLRDm0IgveQtEwDoIAgiaAuhWPZEGIiSPumWX1nKajMdkVRYBgGn6Huui6q1WqH7ZOTk+t5/4IASZFgNgyvM0Ap5FDIs8O24RAC16UwjSZEUeppu65pQPvh6VIK23ZAqQtBUNr11EJIiIAI676mVquh0WjwuuzvzTPbw+Ew9zWiKELTNAwNDa2/HQttX6O1PL9i27BM0+somA6oa8ClLmxN88pWFNtii57tsqzA0HXP17Q7BexHCYkAPMl1SVa2DOvdlk5fkiSIoohkMtkz5soagyRJHZOA2MMBYAO+FLISQm7Mc0bltVW06jVkRkZhWyb0VhOZkVHUyyUkMlnEEknYjXUpASIIyI6s55nblgWt0IRlGJAUBZnh9YkvtF7ithFC+touyzLC4XDHBChmO+sBEkK8RiaKyI2Ow6UU5fwKWvUalFAYoiShVioiMzIGvdWAEgojmcnC0jtfzTND6xr2ruuilF+FobUQTSSRGV7P/bcbnnQBS7VL+mSH/b1PlmqqKEqX7ax3JQgC4LogAkF2eBQUQGUtj1ajBlkJQZYV1EpFpIeG4Tg2RFFCOjcE2+4MiSQz6/o6FEB5bRVao45wNIasr9wtzXulZvYkEgn+MOJlCa9nJklSx+Q3tl88Hodt2+tpmxRI5TwtlmqpiGatClGSEEumUCnkkUhnIbbjuulhlkK5LrsRS6YQS6b435ViAa16jddFdn3bMrmDAoB4PM4fWP43P0opRFFEKpXqsJ0QwvP0me0UFIlUBkgB9WoFzVoVlm4gGk+gUliDGosjnkyhXFhDengYoiACvgVC1GgMkeh62nO9UobeakEQBGTT4/w63vjUenJFJBLpKFu/7YIg8HRk//fCxrK446RANJ5ANJ5Aq1FHvVqB1mwiMzyKWrkEWVGQzg2huLqM9NCwJx/dLPM6EgqrUCPraZTNRh2mroFSF7FEEvFUmu8Ly2vnoihCUZSe7dVfP1gHiH0v8Xict2F20nAkCjUShaFrqJaKnqz6yBi0RgO2ayEzMoJSfhXJVAbhSAS0XmYXgijLyI2uT0bTNQ22VYNjWVAj0Y72Stv33I8jn7L58ssvd23fSnZA13VUq9VNJy0JgoC1Ygk380Wsx/ko/7dSLvdQhCQQ4OI9dx2H7Tg4f2MOvEvQplqtIhaPQRQE+OOHIRE4c+ok77X3wzRNFIvFTSctCYKAWqOBy3NLvuu39dNBUe6wnax/Rl2869gUQiEFb1x9G86GcfyW1oJABITDoQ7bBbh4z6kTkNtvJP1gE8qmpqb67iMIAgzDwJtvz4L2sL1arSKeSEAgBH7bKXVxanzECxdcuwFjw4CEYRqwLRvRaKTDdlAX9x+fQkRVN7WdTaKbnp7uO3grtN9M3rh2Azb11xlmew3xeKzd0EnHZ5PZFMaHh3D55ixqeucDzLYtaJrWHr/xhxFc3D0xinQyseXU/fn5+U3TONn289duQLPZudbLsNFoIBQKQ5alDbYDuZiKE1MTuLW4hHxto4qn1ztPpVIdtgMUU7k0xtpx8c3a69zcHKampvqWO+s0XL01h4rGYvrr5yuVy8j0qu8AYiEJp08cw8paAXOFzvE713U2zH9Z/2woHsGJqYktfU2xWEQoFOqa/+O3XRAEzC4uY6Xa6LK92Wp5nQ0ulc4+I1BE4D2nTqJar+PKwgpIh6+hKJXKyGT89+0RD8l4113H+6Zs8i/kKP68973vpbvBdV1669Yt6rrugR+/sLBALcva1XX3em3G7Ozsro4vlUq0Wq3u+rqDsH1xcZGaprnj42q1Gi0Wi7u+Lru2YRh7On43tmuaRldXV3d9XUopXV1dpa1Wa9fHr6ysUE3Tdnycbdt0fn5+19eldG9tZi91rtVq7bnc93qO3Za767qbtnMAL9M+fvVo5XQNEEII1/LYDbs5llLK48J7gbZTNvdy7E6n2FPqSQzsRQyKEAJJknatBErpus7NTtmr7YCXEVOtVjft2fWDtuPuu7GBHbeb6zJUVT0U21ld263ttB2G2m27abV7yru5bqPR2HN6ZSgUQqPR2HW5s1nTO8Wf7rpT3rFO/6233sLTTz+9q2MNw8Bf/uVf7tjxU0rx5S9/GY1GY+udN+GHP/wh3nzzzV0f/9hjjyGfz+/4uKeffhoXLmy+kNlmUEpx6dIlPPXUU7s+x5NPPolr167t+Ljnn38er7766q6vC3gTsr7yla/s+vhvfvObPNtqJ7z22mt47rnndn1dwAvPfOlLX9r18d/5zncwPz+/4+Nu3bqFJ554YtfXBYAXX3wRm82834ynnnoKly9f3tWxf/u3f4vFxcWtd9wE13XxF3/xF7tq8+zYVmu7abDrXLlyZdft7LaM6W9E1/WO1EzA6wE0Gg0MD3cuuE0pxYkTJ3j++8rKSk8HOT8/3zM2nUwmMTMzg8XFRRSLxZ7HTU5OdsUn0+l0z/OVSiXMz8937F+pVPhg70buu+8+3jtpNBp4++23u661vLzMNYn8SJKEe++9F4Zh4OrVq13HlUolhEKhLs0QQRBw3333dfVIms0mrl+/3nEeTdNQr9d7lvupU6f4AiamafZsrIVCAdFotEufhBCC++67D7Ztd33XgFdmoih2SVX0s91xHFy6dKmjd+u6LhYWFjA9Pd1l+9jYGL+narWKW7dubbvcQ6EQ7r77bn7PG4+r1Wp8gLzXPW88n+M4uHz5csfbHG2PSfTSGxoZGeHjW7VaDTdv3uyyYXV1Fel0ums+g6IouOeee+C6bld5AV4HqVqt9vy+vRTXTo2c5eVlPhPaf/+u67bHBdZh9VUQhL7tLZ/PIx6Pd9WXTCaDyclJAOh77PLyMtfT8dOvrVJKcevWLT6DnrGwsICxse55LKqqcjXNGzducLVgP2wcaeN1JiYmkMvloGlaz7babDZhGEaXvAmr75Ik3b4qm9vBMAw0Gg3k83lEIhEeZlEUBXNzc1wMjOmotFot7hyq1SrK5TKazSYkSYJhGFwobWlpyVv1R1FAKeXnnpmZQbVaRaVSQaPR6DpueXm56zhBEHpWJDanoFAocCVKwMsaqNVqfADOsizkcjlYlsUrqaZpXGOFqUlS6ilfrq6udqhJOo7DU1UNw+BT9WVZ7thP0zSu2c9UKNPpNE8Z9cNWc2Ll7rouLMuCJEm83Akh0DQNIyMjXGER8Jx+o9FAqVTiTs22bYTDYZTLZRSLRf63aZpcCZV916VSqeueWcqi3/ZUKgXHcbpsd12XpzrKsgxRFNFsNhEOhzE7O8szxBqNBkZHR7lYGOA1OH/5MWVRSZJ4uVNK+ffEnBm753K5zIW7/LbW63X+t2VZSKfTHZk3vWxnwniNRqOn7SMjI3wWLOB1hur1OiqVCmRZ5oqtoihibW2t415Yx4PZWavVUC6X+fdqmibC4TAcx+HyBOz7SqfT/HM/rL1Vq1WoqtoRnmFpkSzLi9U7QRBQqVRQKpX4/qy9EUJQLpexsrKCUCgE13URi3mD6czp92urgiBgbW1t220VALeDyaPX63WeJs78jGEYiEajXICNHedX1t3oZwzD4Aqe6XQa9XoduVxu07ZqWRYWFha66vtWYbp3hNNnuhdMzhQATxv0qxGyhu3PcPDnwLPKzt5+aDuNj6WR+QuTEMKdCRuhZw5ZEAR+/Y3H9bKdOTcvH1fmKXh84KV9bqY86D/WNM2O9E5/GiVzhoD3EGGx8l7HsbgsKxvmJEVR5LpGm5U7O445UGa73xFsZrvruny+Avs+mA2yLHO9EzbfQpZlfg6mc7/RdkmSvKUO+2SFMJlk9n2Hw2EIgsCdB5PS3XgOdl1/SjAA3qD9kwbZvW28ZxZHZ86G1U+/7eyeN6sz7AHD6o/fduaQN9rOHsz+eRb9bGfHs+OY42HrKjDb2bVZG9rMdn+99NdPJsesKAqfKMfOwY7ztzeWRsu+M//aDhsxDIOXvb+N76StAp2TEVm5s/tgZRIKhfi8If9xzG52P2wuCLOf1QF/2RFCoOs6/87YPv3a6nbGw94R4R1WCbeCaWUzWdR4PM4133vRS8VTkiTuhPza/QsLCxgeHu4rm8qO24jjOFuOHbBBJ7amALOdPe17wUIVmUyGp5OxCu4fcKbUW0aQUtr1is7wN+iN19iq3FkvhRBPq8evI9RryUjXdbG0tMR7aRttANYnzJTLZSSTyY789O3azspgs/rPdGwURcHo6ChUVeWNtdd31i+8w8od8EIZxWKRv7lttl7vftje7ztbWVlBNpvtqqN+2xuNBorFInRdRzab7ZumyPAvFMSwbbtvnd2oeeS/f397Y8Jmo6OjfSWWmfOs1+toNps8fLaVzUD/tso6Mb38BaW0Q72VOWd2Hv9xuq5jZWUFk5OTPQehmUNnoVK2GEsmk9ly0JeVmSAIfcM77winvxPY/bIwQavV4pNH2IQKYL1nYVkWlpaWMDo62rMBsgrIXqP7OZ9B284qQz/bgXUHGovF+k4uWV5eRigU2ldhNPYmUa1W0Wg0oCgK0ul0X4fWL1+eUsqdfTweRzqd7uiZ7pftuq6jVCrxEBlzHBuvu7S0hKGhoS6HwbI0SqWSN4kpm+157/thu2EYfIYxU2vtZfvy8nLPBe8ppdA0jaupZjIZ/gDZD3tZ6GliYqLLwbH7WV5e7hJRZG210WhwJ6+qKu8c9brn/bCfLQTTS6yPUsrDsRMTE+sT5tpvxbquo16v8xn5bD0Q9nazE8gm0sp3nNPfCHsdZLFOy7KgKAri8TgPF7GYpSiKXGqXUoqVlRXous7fBPa7Um1me6PR4K+vzHb2+reysgLTNPmDiVLKY6uZTKbvAif7ZbNhGKhUKjz2mUwmO0INzzzzDGZmZjAzM8PDILquH6iz72W34zg8NhyNRpFKpfi0e0opvvvd7+Ls2bN8MRD/Skz+ntph1ZNyudxlOyvzv//7v8dDDz3EF2GxbRutVguVSoV3Cg6ijrOHzOrqKmKxGO81u66L5eVl2LbNHwjsYVqv13ndZ47Sr7F0kLCOSalUQiqV4m+hlmVxvbDx8XHuVxqNBhqNBtd8isfj/O1lL/Zv5vTfsSmb24W9hsXjcYyNjWF6ehqxWAyFQgGXL1/Gn/7pn+L5559HIpHAo48+igsXLqBer2NpaQm///u/z5czZOEX1gM/iIep3/bR0VFMT08jnU6jXC7j6tWreOyxx/DNb34T0WgU3/ve9/DEE0/wZd3+6I/+CKurqxAEocP23eYc78TmUCiEZDKJdDoNy7Jw6dIl/PZv/zZmZ2fRaDSgaRqWl5dRr9dx7do1fOYzn8Ha2hoymQwURUGz2ewbIthPuymlCIVCyGazsG0bP/jBD/C7v/u7KJVKfICWOaEnnngCf/zHf4x6vY50Og1VVQcyh2MQtjuOg2eeeQa/8zu/09P2733ve/iDP/gDVCoVpFIpRKPRfStvv+Nm9Y+NL332s59FoVBAvV7H+fPn8eijjyKRSOBHP/oRvvCFL+Dy5csoFAqIxWKYnp7G2NgYEonEgT5YWe/db78gCCiVSvjDP/xDvl7G97//fXz9619HLBbDX/3VX+E73/kOrl+/jkajgVwuh+npaR4e3u8OzR3f0+/F+fPnUa16ejNsweN4PM57TLIs87U8/b0lNjAjSRIeeeSRQ+lpLC0t8TRKTdOg6zrv1bFBTzYByq8NwnqyAPDII4/sy3J0DEopXnjhBT54yF6LY7EYfxVnr7wsY4dlQrBsk0wmg9OnT++bjb24ceMGz+tmA4mVSgXpdLpDa0gURVQqFSiKwrOa2EDe+9///oEuAr9dbt26hbm5uY5B0H62szVio9Fox6SxD3zgAwO3nVKK559/nrcdtg1Ax4Czruv8TYOpXrLQRzabxX333TdQu7aLZVl47rnnOvwA+65ZBiHgxfFZwkG5XEY0GuVZVjMzM11pm3tls57+OyJ7Z9CwLAu/EiZTg2QzTk3T5NkWbODFtm1ks9me+fUHbf/a2hpPd2OqmLIso1wu89l8bBCbOQJBEDA6Orr1BQaAIHiLX7PFpFmPqdFo8IwFlt0CeA9fWZah63rXIO9Bs7a2xhu2JEk8ZZbVC5aF1Ww2eR0KhUIYGho6VLvZ4uMAtrSdLWQOePnm/jV7Bw170LCUSpY1B6yHA/11gdXlSqWyqc7TfsLKgmWalctlHotn98QeUKZp8g4Xy+hxXReapvEQ5kESOP0esEHNkZGRjoEi9ppMKUWpVEI2m+34HFhfHeiwiMViyGQyHQOzftsB7wFWKpUwNDTUsZ29Vh5ETzSXy/G3p152ttrKjf7BcX/j2GrN3f0gmUxC07S+ZVutVrniYq96wfK4D4NkMomhoSH+nfeynY3t9LKdjV3sB9lsFuFwmD/0/YkHGx3iQdQFdm729mtZFpcVt23bky33peEmk0lkMpmudtaLXp8fdCcxCO/sAjaIm0wmN025O6qwRt5oNPjA9FGDDXz1mt18VOmXvXM70G9W6X6wH9kqO7k2+5+FOZlDZ2mVLMzJHnSyLEOWZSiKwifD+fPojyJBeGfAEOKtsbqwsMAXcL6dYL0TURSxuLjYsfj3UYE1LE3TbosHK8sy2qvo22HAHPB+X4PNOm42m3BdF5FIBMlkks8P2Wv983dgezl01kNnPXK/Q1dVlQ8C+ydQvRMJnP4uEUURFy9exOXLl/Gxj33ssM3ZMYQQnm/++c9/Hh//+Me7tE8OE0II6vU6vv3tb+PTn/70bdEAv/GNb+AXfuEXeurfHGVc18VXvvIV/Lt/9+8GVgcopahUKjxMx2Ywx+PxHXUyekUiXNft6dBZ1g9z6GySFdO7Z4PCjNuhTu0HgdPfAXNzcyiX11elyWazWF1dxRtvvNG1r6IouPfee49kxdI0jYuWUUqRSqXwwgsv9Fy4hVKKkydPdgmZ7Qc3b97k2VKA17gnJiZ6Ko6Gw2Hcfffdh1K+juPg4sWLXb36+++/H4VCgQ+AAl75jY6OHtgA+Vb4v3s/Z8+e7RKR6/fdLy0tdYmmbcR1XTzxxBMYHR3Fr/7qr/JxrtnZWdy8eXNbtjqOgzfeeAPvf//7eSYRG19gDp1lSLFMGODOdebbJXD6O6BarSKfz6PVanWILd26datDbCkWi3WILR01mIjT8vIyIpEIEokEWq0W5ufnuZ4QE6ibmJjwreq0v1SrVb7ObigU6lm+uq4jmUweiD39cF0XjUYDq6urXGuICZ4tLCzwrC42ZlKv14+M09/43bPsGEVRMDs7u63vnklJ1Go1qKraoUvPevC2beOhhx7iyp3MEbPvmKUwsu+YZcGwNsQmKZ09exaTk5MdulqBU98bgdPfIbIs89muzLGziuof8NnpIiYHCZsh6M+Q8IuvsdhnKpWCZVkHOtDLhLP85cucCOvd+QWzDgvLsnj2EQAkEgmudgh4NjOFz6M0UM6++3g8zlUwN67HzPR2+n33LHzCZr4qitIheMYmgjHRtI2whwD7jjcK37F2JEkSTp8+HTj5ARNk7+wAlq7Vi0KhANd1eUocU+I7ihWW9e764TgO8vk8n3fAJnftN/3ErFzXxdzcXIfw3WGWLxu03dh2+gmusbeBw8YvccAe+lulCzIFSj/+tMXV1VVIktR3DgIhpKOnv/E7dhwHS0tLPfPtj3IbOuoE2jsHAJtpx1QCj1LvbjewlDamqphOpxGPxw+tAZqmicXFRUxPTx8JB9qLo5iyyWaRM82gUCjUFXLZzTlN0+QibX7l1J1SKBQQCoUONVz3TmTftHcIIbcIIecJIa8TQl5ub8sQQp4khFxr/59ubyeEkD8ihFwnhLxJCHloL9c+ahBCuGO8desWSqXSoYcg9gKbpDU8PIzJyUkYhoHZ2VkulHbQ9ybLMkZHR3HhwoWOgdKjAqUUL7/88pGwzS/2tri4iMXFRciyjKmpKYyMjOxJ4dM0TayurmJlZQUTExN7cvi1Wg3nzp3rWqktYH8ZREz/f6KUFnx/fwbAU5TSzxJCPtP++z8B+GkAp9o/jwB4tP3/bQ1TI/RTKBTw5S9/Gb/xG7+xaa9UUZRDnb27HVg8PRQKcRmH//bf/hv+xb/4F7j33nu3PF5V1V31fJksw8YMmRs3buCtt97Cz/7sz25q816c0WY2sQUtelGv17m0RC/YxKPdXnu7YngvvfQSzp07h1/8xV/kEtbRaHTL8mDSF5vxd3/3dyiVSviVX/kVaJrGt7MMGj+u63KZ417Mzs5yQbWNtomi2DF4GzA49hTeIYTcAvCw3+kTQq4A+ElK6TIhZAzA05TSewghf9b+/a837tfv/LdDeGd5eZmvYcl0eIB1sSjHcfiSeqzBs89isRgeeOCBQ7R+e7z11luoVCo8B5rp+DOZZpZ54V9SkeVlT09P7ypv3XEcPPfcc3xgEVifws5WsmLXZBkjrutyLaGzZ8/2XdBmt7DevK7rHZkkfnkLv4wBG9hkA6L3338/stnsrq5tmiaef/55AOAD2RuvzcqDOWM2OEoI2ZZY2ptvvsmFBjeKn7EBWlZ3mQ2WZUEURZw8eRITExMd52s2m2Dtt5fN7FxM6I3dB9Peef/733/bh0kPi/2ckUsB/A9CCAXwZ5TSzwMY8TnyFQAj7d8nAMz7jl1ob+vr9G8HmFNi64sahtEhxOZXs2RLApqmyTW1bxdc18Xa2hqfBMPCP2zaul9Migmj7VUMi6lVtlotGIbBl1OUZbljaT02MMhS/vqtADYISFuMr1Kp8F4xE6+zLItnrjCbmCLr5OTknkNikiShVCrBMAyuOskcJhMA9K/Zy9IhdyKWZts2SqUSvxeW2WXbds97MwxjU4VISZJQKBRgWda2bAa8N6LD0Fa6U9ir0/+nlNJFQsgwgCcJIZf9H1JKafuBsG0IIZ8C8CkAA5cb3Q/i8Tiy2SxyuRyAdRGrjbDp5/4l6Q5bjXO75HI5iKLIMzQ23mOxWOQSvRvZ7T0y551Op3tek63AtXGlJwBcKXI/GB4eRjQaxfj4eNdnfsG1Xuwldi2KInK5HHeGrDzYYuubOUn/2rubkcvl+NiJn0KhgHQ63fcclNKeA7GKoiCbzXbZDIArT/ZbvnAvA80BmzOw7B1CyH8F0ADwv+MOCu9sF5YmydajzWaz+7bs3EHDFo7xKzjuN41GA4Zh7Dpcsh8cdPaObduYn5/H5OTkvlyTttVkXdflK1gNimKxyOURAgbPvmTvEEKihJA4+x3ARwC8BeDbAD7R3u0TAB5v//5tAL/WzuJ5H4DqZg7/nQaLgU9MTGB0dBS1Wg1zc3OoVqv8Nfp2zfaJxWLQNO1As3rC4TAfZzgKsIf6QQiusbDhwsICxsfH983hMwXMQTt8pvLKJrIFHCx7GSUZAfAjQsgbAM4BeIJS+vcAPgvgw4SQawA+1P4bAL4L4AaA6wC+AODX93Dt2xYWehgZGcHk5CRs28bs7CyeffZZ1Gq1I+PEdgILM/zZn/3ZgV3TcRx88YtfPLQlCHvxrW99C6urq/t+HcMw8LnPfQ7JZLJneGsQXLp0CV/96ld7LvC9V2zbxp//+Z/3negYsL8Ek7MOiEajgatXr/aM91JK8eSTT2J4eHjH2TyUUoyNje2btotlWbhw4cKW+7mui1u3buHEiRPbPncikejav1Qq4datW9vK2lhcXNz2RDjXdXH8+HE+RrBdZmdnO0T2NmN1dRXZbHZbi9BQSnH69GmEQiEsLS1hdXV128610WhgYWEB99xzz6bHuK6Le++9F6qq4urVqx0pllsxNzeHSCTCx6r6wbKSms0mrl27tq3vglKKpaWlrmyffvuOj49jZGRky30D1tnP7J2AbaLrOprNJtbW1hAOh2FZFhRFgSiK0DQNJ06cQDwex9LSEgRBgGVZfNk6NgOSZU74Bb3GxsbQaDT2zW7HcdBoNLC2toZQKMSXrWNZSEyQi+mjLy4uchkKls0UDoc77G61Wn2dSavVQqPR4Mv4sZRXdpzjOIjH43x93cXFRSiKwrNLbNvmx20UPmu1Wjt2+rVajS8ezhwsk+htNpsQBAGqqvLsk6WlJf79+lNL/WmJhmFgeHgYlmUhFAqh0WigUqmgVqtxp8muweaARCIRfg3LspDJZLC0tMRTHFldYdfQdR2jo6MwDAOqqqJWq/FlHpmOEbOr2WxCVVX+N7tP13WxuLjIv3OWqcbmbsiyjGQyCdd1uZAbm2G7sX5bloVEIsG/N9d1sbKysu36HTj9wRE4/QOCraHLMjgUReE55myREOYsWa43+5vlQ5umyXPPBUFALBbjjWi/IITAcRwubRuNRrljCIVCXbM7md1M85xNsGFOiRDC1R37lZPrunyQm6WCEkJ42W0sI+ZgWH43k95lToqJi+2mnNh34BcJY/fPMlaYQBx74LD79eeg+8XMZFnuEDNj98HKkuXbs0lm7BoAeNrkVteIxWJd9xwOh2GaZpeeDRPeY3Mr/Lb4c/KZXYAnjMdSdpn97PsGuus3ewiya2+s3yzd1v+9HUT9vhMJwjsHhOu6fWdybgfDMFCtVvkKR+FwGNFoFKqq8h7VfsB6YTvFcRwUCgXouo5EIoFkMtnVeNlCF352Wk6UUhSLRViWhZGRkU0dBJsvsRPYvITt0E9wrR/sAcFWedoMSikWFhYwMjKyozg+29e/rutGXNflufQ71Y1iDx/WMdloM3uLIcRbbW43g7d+rfyA7RGEd44AgiDsKVshFAohkUjw3pw/XMQ+j8ViCIfDA136kPX6dsPU1BQopajVasjn8wiFQshkMpsqJ+6mnJhmfT6fx8TExEDXWGU93e3A5g3sNJtmqzx6lu2STCZ3LS/Ryyb2QF9dXUU6neYLo+8GVk9Y+KhcLqPVaiEWi+HYsWP7uu5twM4InP5tBnstjkQiPEzC0gUbjQbPq/a/CRzW+rfM1lQqhWQyCU3TkM/nQSlFJpMZmLYKC7UoioL5+XmMjY3tSVRsNzA5iv04r23bqFQqmJmZGdg9sTckTdMwMTHRsdj3bs/XarVQKpUAAOl0+kDnbQRsn8Dp3+Ywx6qqKtdaYT24RqOBcrkM13X5m4Bt24jH4wf+usweVKqq8p7g2toaXzBjrw6CzYOYmprC22+/jbm5OXzoQx86UKfzjW98Az//8z8/0JnkjuPgT/7kT/CJT3xiIPdCKcW1a9dQr9dx/PhxTE5O7vq8LAtHkiS+gP3o6CgkSfIeVLX6ts4jEIJ4LAZBCB4QB0Hg9N9hsIdAOBzmgmiANyZw7do1fPWrX8Xdd9+NRx7ZnsCpJEk4fvz4wAbT2CBvuVwGpRTnzp3Diy++iE9+8pObhkVYamovWYd6vc6zhgAvpfGVV17B8PBwXxVTSimOHTu2Y5VTx3Hw9ttv9xyIPnXqFCqVSs/UyEwm03ehkUKhgEKh0NP5slnHKysryOfzfDshBCdOnOgKPbVaLczNzfV15K7r4jvf+Q7uv/9+xGIxntEzNTXFB8qZM98qK6xer+MLX/gCfvEXfxGTk5MdiwjNreTx3HwRjmNDZDZSClGS4dg2QADHdiBKIiTHwj9/7+mBC+QF9CZw+u9wWONnaZMf/ehH0Wq1sLCwwDNE2HqlLOWPDS6ypfSOHTs2UJs0TUOhUECj0UA2m8VHP/pRrKyscPE2f8qlIAgwTRPpdBqqqvZ0+mzN1nq9zsW7fuzHfgxLS0s9UyZd10UsFtv0odAPy7KwtrbG0zX9ypGUUqytrXH1SLaMZjab5Q6xF8ViEYVCAc1mk2fEiKLIBcpGRkawsLDQIdoXCoUwNTXV5fTr9TpKpRJqtVqHuisTUJNlGe95z3sgSRIWFxf5ClrpdLpDG2h1dRWNRgO6rndkE+m63pGC+0u/9EtwHAdra2uQJAnHjh3zygOASwSsrqwgFFZh2xYisTiUsIry2ipCqgqt0UAym0MmHsPRTSd55xE4/TsIlmHBnKM/vbLVanFFSJaGt19jASzNkoWgmINkjpPlngPrkrxbzbxlcwY0TYMgCNxRsfQ/dv/sWqZp7ureWGoiW0fBv76s4zg8TZEpSAJezvxWK0NZlsXLAvAGd23b5umorEzYA6zfd8NSLw3DQKvV4oPEzDZWJky1FEDPsmV1hT08WF1xHAfNZpPXFQC8vDfao8ZiuOvdD6CcX0WtXILeaiKVGwZ1XTTrVUTiCWSGRwFt/+aZBHQTpGzeQTB9nK1gvVnLshCLxTA+Ps57xIN4CLiui3q93jdX33VdLCwsIJfL8bxvwFOp7BUCsiwLzWaz7/WazSYqlUqHnDXLgd/p2AbTpOk1aLu6usoVSTeiqmrfrCTDMHqGhEqlEh+r2Qibf7Dx+7Btu2dYptFowHGcvqqnsViMPwQo9dbS7ZWqWywWEY1Ge74hiaLIs4sWV/J4ZXYZgijycA4ooOk6IqoKCgpQQBBFiK6NDz5wOtDiGSCbpWwGTj+gJ6xe6LqOWq0GXdchSRLi8Tii0ei+ZwQ5joPl5WWoqopMJrPnzJJms4liscgzVfaDQapssrz88fHxPQ+6sxj90NDQnrV6FhcXMTIysmUZ9vIrbIZvv3UWgkyfwRHk6QfsGNYA/VlBtm3zQVPA63nH4/FN8+53iyiKmJiYQLFYxNLS0p4Wm2ezeSVJwvz8PCYmJvbF5kHDQkWDgM2G3QsshLWdh1C/0BP7CTg8AqcfsC3YzMtMJoN0Os3HAdhqWuFwGPF4nIciBpV/n81meUYKkxHebSw+FAphcnKS91bD4fBA8941TeNaQXuFDRQPAn9cfi/4l2Lcy/H9FhoKOBgCUYuAnlBKsZxfw42Fpa6fm4vLmF1exVq1DoMCo6OjSCQSqNfrmJubw9LSEi5fvszTMvcCy++fmJjA97//fTz99NO7Pid7cE1NTeH73/8+fvCDHwxUyvrxxx/vSKvcCz/4wQ9w8eLFgZzrW9/6FpaX9750RblcxmOPPbbrMqtWq/jqV796W8qHv5MIYvoBPaGU4rsvvYm8hXYqpwRKXcihEExdhyBKACgkQcBH7hpDKh7D+fPneWbOCy+8AEEQ8IEPfGBHvbozZ86g0Wjg2rVrXWGEcrmMubk5nDlzZstzhsNh3HfffZifn8fa2lrX/pVKBfPz87j//vv7nou1jeHhYUxOTuLixYt9B8IppXxZwV7xbpaZdNdddyEWi+HNN9/c1P5SqYRIJLLpPIMzZ85A07S+kt2MrSSfFUXBu971LiwtLWFlZaVnebDQTrlc7pt6ylaEm5mZwZUrV7hCKIOts7xRMdN1XUxPT+/r2sZ3GkFMP2BXEFGE2dRhGhoEQYTrOFBjceitJiRZhus4SLXXP7VtG4ZhYHFxEZIkYXp6Gq7r4ubNm1wxkWWp+BeOZ+mOADAyMgLHcaDrOjRNw/LyMhclY9LJ2WyWywdomoZwOMxj3yy3PZVKIZVKAQCXpqhWq3wflpUyPDyMfD4PwzC4BLRt21wIrtFoYGpqimcGtVotrK6udqQqWpbFZSAcx+G570wmmClHNhoNTE9PQ9d1qKoKTdP4bFYAHemlTCGzXC5zm/1lRSnFyMgIL/NWq9WzrEKhEFfWvHHjBtfGYaqgpmkikUjwsmo2m1zDiKWiWpbF1VXZzNurV692lLskSWg2mxgbG+MPKaYLxZQ+2YxdSikWFxeh6zoikQh0Xcfw8HDXAyJg/wicfsCm5Ma8BcAt00RxeRHVYgGyoiA1Og45FILVavK8ftu2MTQ0xCfyMKlelukjSVKHPDJzYI1GgzsG/2f+np9hGNypsXP5pY2ZCN1GxVFRFLnQm1/y2R/jZqmCzMEx3R4mI+3P82ea8CxnnU2WYk6UHcvOzY6PRqP8gcKO9ZeVruswTZMvrC5JEk9X9ZcVmz/hlxzWNA2KovCU1u2UFTsXmwfBbGXrODD1Vqalz+5HlmUkEomOcwHoyuhiUspMftuvs8TSZZl8iL+MA/afILwT0BNKKd64+jbqpt3+u/0Pf/MnIARwbRvjcRWKLPHGHIlEdtSIWW+VhU7YJCJZlrnT2I58tOM4fJF2/+IriqJAVdUtB1ht20axWISu60gmk0gkErynzRaF2UxUbTvSyswG1hs3DAOlUgm2bfNrbmYfe1tik8vYQ0BVVe54++G6LhzH4RPjTNPkksvsYSQIAhfr205527bNJ4OZpsknkLGB8+3qPLEHZ8BgCMI7ATuGEIIH7rlrW/uyjgObYbu2tsYXQmFyz5ul6imKwnui7FysR85CL2xykX8hEkVREAqF+GIisizza/mF51qtFqrVKtesZ73icDjMHTQhhD8c2ASs1dVVKIqCdDrNFw3ZDCYet5WGEJs3UK1WoSgKRkdHeW+afW5ZFjRN428AbDYxc6TsTYU5cpYZwxyxfzvr1bOePSurSCTS8cbDrs2ONU2Tfw/+c/pnSzObUqlUx4M5yM45ugROP2DP+MMkLJTil3suFAoAwHujmz0E2DYW9/fP0tz4QDBNk08eY06JOTb2UFAUBYlEgi+TyB4EmqbxWcdMoE5VVf4gSCQSSCQSME0TpVKJx7/ZKlM7dWqsPMrlMkzTRDKZ5OsNGIaBtbU1NBoNaJrGH07sHpjsseM4aLVafNlItsDLxtAP0xtisXv/Cl3sQeCXofDnz/uPYQ/XcDjMvw9/bzxw7LcngdMPGDjMgWyUe9Z1veshEIvFei652O+8QO8HArDeU/WHMOr1OizL6nhLEEURsiwjGo3y3qlt2x0PELaYi6qqyOVyEAQBtVoN8/PzUBQFmUyGz25lPfTV1VUkEgm+khSzqVaroVqt8lATANy6dQutVosPuMZiMcTjcSSTSW4T6/GzUAxbwYv1rv1vFn4n36vMAgIYgdMP2HfYQ2Djwi+apqFer/PVv/L5PFRVxb333rut826UgvD3Vln4guF3whsfCix84Q9bsMHScrnMnS1z2hcvXsTx48exvLyMarWKj3zkIyCE4Nlnn0Uul4OqqiiVSvja176GRqOBn/qpn0I0GuWzWePxOI4fP84XIwe8gc9//Md/xOnTp2FaFt66MQslFOYZOLKiQBQEgBAAvmUJXRf3zUzsWC004M4lcPoBBw5zztFolGfIUErx5ptvYmFhgadXAr01XNj2sbExHD9+fNvXZP9v9lAA1gco2YOBZaJYloVisYjf+73fwyc/+Umk02nMzs7i+eefhyRJOH36NObn57G4uIiVlRXUajW4rovjx49DURS88cYbEAQB1Wq1wy42hnD+/Hm8973vRTKdhlYysVSpQAEBDAPU1RFNJNGolSHJCixDhyjLiMXimGi2EPPJIgcEbEbg9AMOHfYQmJychCRJWF5ehiiKaLVa/KHgum5HbvfY2BjXiB+U5AODDVBu7D2zh9Njjz2G2dlZVCoVjI6OYmVlhUsgs3EFURRx4sQJTE1NdeTsz83NgVLata9lWfjxH/9xqKoKh7ZlI8IqAArTMNrX9jKHTF2DqetIRqJB+CZgxwROP+DIEA6H+Xq6W0EIQaPRwNzcHO+5s9RONvjJ9hsU7OGUyWRQKBRgmmbHg4ENkPplqFnWDXvDOHbsGJrNZtcC6v59HcsGAZDKDaFRLcMyDZi6Acs0EFYjqBTWIEoiZCXEHwQBAdslcPoBR4Z77rlnV8f5s3larRYqlUrHoiMsA4U9EPzZLLvl1KlTPe1YWFjou0bu+973PhSLRRiGgbGxsb7XlyQL78lGQIkLhBMASaBSqSASiXgDwSMxgKKtUU+Risd2fR8Bdx6B0w84MuzWCfuzeXrl+1uWBcMwUKvVeCYPm/XK0kxDodCOcsx77cMygxzH6crpd10XKysrUBSlYzGXXiiKgtMnj/G/KaWYmyOYmpoKJjAF7JnA6Qe8I9mY3hkOh3nYiMXm2duBpmkdbwcsns+kCLb7MGAaPPV6vWNegG3bWFpaQiaT4fIDO8E0zdtC/z/g9iBw+gF3HCw2z8I9sZgXHmFvB2wmcLPZRKlU4tpCbMYuWzC+19yCSCSCp556Ch/84AchiiJf8Hx8fLxDw2a7UEpx7tw5nD59OnD6AQMhcPoBAW1c18Wzb16EToV1jaF2JielLkzTgq7rMJp1nL3nBEKKgh/96Ec4c+YMTp48CcBL97x48SIefvhhLC0t4Wtf+xr+w3/4D3AcB5qmdYSR/LB5C722v/jii7j77rs7lCjZLOLgQRCwUwLBtYCANrZt4/FXL2F2JQ9QQJREuK6LZCaLWrnMs22ymTR+9vQxqOEQHn/8cUiShFQqxd8gbNuGJElcZZTN+CWE4N577+2pG99qtXDu3Dme6cNgk8kkSeLCaEwD55FHHgkmZQX0JBBcCwjYJoQQjM+cAAWFY9nQmg3orRbUaAzxVBqCKIAY+vq+4+NYXV3FrVu3uHwDGxNgM3kVRYGmaTh27FhflU42lrC4uAhZlvkANFuHgAmnKYoCy7IwMTERrEAVsCsCpx8Q4MO1TEAKe9mQjgW4NgilgGsDrgOAwjHXnX4qleJrBPBzuC7q9XrXfANKade+DFmWkUwmO47pSNPswVaqnwEBvQhqTUBAG1EU8ZP3Hofd7qED3gIlumEg3V5dCgAISSOihnm4ZiOO42B5eRmTk5PbvraiKHj3u98NAFx6uVKpYGJiIojbBwyUwOkHBLQhhCCXSXdsazab0DQNuVxuR+fZS+jFNE0UCgVMT08HDj9g4Gw504MQ8kVCSJ4Q8pZvW4YQ8iQh5Fr7/3R7OyGE/BEh5Doh5E1CyEO+Yz7R3v8aIeQT+3M7AQGHj38xlJ3iOA6WlpYwMTERTMQK2Be2U6v+AsDHNmz7DICnKKWnADzV/hsAfhrAqfbPpwA8CngPCQD/BcAjAM4C+C/sQREQcJR5++238dprr+3IgRuGge9+97t8MZTtUq1W8cwzz2B0dJTH61n2DpN/7vcTDOoGbJctwzuU0h8SQo5t2PxzAH6y/ftfAngawH9qb/8y9WrgC4SQFCFkrL3vk5TSEgAQQp6E9yD5673fQkDA3qGUolgsdmXXsKUKmea/n2w2CwAoFosd213XRTgcRqlU6grPpFIpSJKEUqnUda1XX30V+Xwep0+fRr1e52mgf/fym7ClUHuNYsIM5vI7ruPgAycnMD48tLdCCLgj2G1Mf4RSutz+fQXASPv3CQDzvv0W2tv6bQ8IOBJYloWLFy/ytEu2/iybics+k2WZL4by4IMPglKKS5cu8Zx8lqc/PT2NCxcu8BWwmCz03XffjXQ6jQsXLkDTNJ7Hz1bQmp6exquvvopIJIJTp04hk81CFxQs54ugrotkNoe1pQWEIhFojQbCagTDE5OwbGfrmwwIwAAGcimllBAysHdLQsin4IWG+qoVBgTsB4QQ1Go1CILAlTtVVYVleStVua4LWZZhWRaGhoY6jqvX61AUhS8ozpZMZPn3tm1zuQcA/Brl9qSvrfadOH4SjWoFlFIcu/c0iivLsBULqdxQzxm+AQH92K3TXyWEjFFKl9vhm3x7+yKAKd9+k+1ti1gPB7HtT/c6MaX08wA+D3gzcndpX0DAjpAkCTMzM5iamuq7z9raGmzbRi6XgyzLUFUVgNc52dhBaTaboJR2OG9KKZLJJGRZxszMDBzH4evq+lfxYvumUikQEMDU4YJAtL3FVKxaBRFZhJpMQLAMuE0HwkiwclbA9tit0/82gE8A+Gz7/8d92/8tIeRv4A3aVtsPhn8A8Hu+wduPAPjPuzc7IGCwCIKw5ZvlzMwMdF1HoVDgIRlZlnHs2LGufW3bxsrKSs88e7bU49LSEk6fPo14PN43NZNSiv/5ve+C63r9n3K5BEVWEPU9TAiAUKj3BK6AgI1s6fQJIX8Nr5eeI4QswMvC+SyArxFC/hWAWQC/1N79uwB+BsB1AC0AnwQASmmJEPI7AF5q7/d/sUHdgIDbBUIIVFXF5OQkDMNAPu+94OZyOYRCoQ7HzXT1Ny7nyCZeFQoFjI+PbymZTAiB2tbXoZSiWHCQHc0EIZ2AXbOd7J2P9/nogz32pQD+TZ/zfBHAF3dkXUDAEYQpXE5MTMCyLBQKBTiOg1wux5UvCSGIRCJ8nV8GWzlrenp6x3n4TOI5yN8P2AvBjNyAgE0wDBOGZfb9PKKqGBsbg23bfN1c27YxNTUFXdfx2GOP4dOf/jTy+TwuX76Md7/73VuunNWPp556CuFweNNxh4CArQicfkDAJrx27QZuttx2jjx4GichBNRx8OPHhjE1NsrDNrZt48tf/jLOnDmDBx54ANlsFm+//TZ+9KMfQVEUTExMoFQqQRRFzMzMdPXaKaVYWlrqqa1vWRZyuRyuX7/esX14eBiJRGJfyyHgnUPg9AMCNoESAZplolosIBJPwNCaABFg6hpGJ6fRHl+FpmlYXl5Gs9nEP/kn/wQAcO3aNWQyGdy8eROTk5MghODmzZsAvBW2Jicne4ZqlpaW0Gw2eTjHsizIsoxIJIJKpYJ8Po9QKMRTSiVJCpx+wLYJnH5AwBYkszkkszmYho5GhUCSFSSOn4RtGh37sQXY2QCuIAhoNpuQJInn4QuCAEVRNpVocF0XpmnCNE0IggDTNPlcAUIIPz+bDxAQsBMCpx8QsAmqJECqeYlmguuAuhZCECDVSxAcG4rkpU7GYjE88MADXRo4bK3dTCbTsZ0tttKL06dPwzQ7xxFWVlYwNNR7IlY8Ht/1/QXceQROPyBgE95z90m8u62RQynF4uJih04+C8+Iotjl2AHwHjrT6dkKQkiXE3ccB81mE0NDQ4HUcsCeCZx+QMAmEEJ479p1XQiCsKMceSarsDFff7tQSlGtVrn4WkDAXgkSfgMCtkm9Xse3v/3tvuvc9sK2bTz++OPQdX1X1zQMA1//+teDBdADBkbQ0w8I2AClFJVKBYbROVBrmiZyuRxWV1e7Ztkmk0moqopisdgxSEspRTabRaFQ6Bh0Zbo8/lCO67pYW1vrGBdoNpv8vGxxlmw223fd3ICArQicfkBAD65fvw5N03hevuM4EAQBIyMjuHLlCk+jpJRCFEVMTEzg+PHjuHz5MnRd52EgwzAwPj6OK1eucOllXdcRjUaRSqVw//3382uaponLly93SC67roupqSlcuXKFH0cIwejo6CGWTsDtTOD0AwJ6wJxzq9WCKIrQdR2hUIh/bpomQqEQDMPoGKRlx7GHhK7rEEWRSykzTR6m0Nnruo7jcKlm13W55LLjOB2SDgEBuyFw+gEBPTh27BhGRkb6fq5pGtbW1pBOpxGLxZDJZEAIwcmTJ3nGDqNcLiMajXaFZPyhHZbDn0gkIIoiTp8+3TelM5VK7f7GAu54AqcfELABQghyudym+1BKcfLkSZTLZTQaDViWBUopRkZGurJsFEVBMpnsORjLVtViyyfeddddPIQTELAfBE4/IGAXsFTObDaLTCaDWq2Gubk5RKNRZDKZDnmFarWKUCiEcDjMB2kppajVavyzoaGhLWWWAwIGQZCyGRCwB5jUcSqVwszMDEKhEObm5vCVr3wFtVoNAPDcc89hcXERlFKUy2U89thjmJ2d5YO0IyMjUBQlcPgBB0LQ0w8IGBD1eh1vvfUWXzP3hz/8IYaHhzEzM4NKpYKXXnoJCwsLyOfzWFlZwdraGt773vcG+vgBB0rg9AMCBoRlWXBdF/Pz84jFYiiXyyiVSohEItyxs3TLQqGAoaGhLq2egID9JnD6AQEDgmXbTE5Odjhz0zS7wjeCIHBd/oCAgyRw+gEBAyKRSOADH/hA1/Z8Po9kMtmR588InH7AQRM4/YCAAdGr586E1oK1bQOOCkEtDAjYZ37wgx/g1q1bh21GQACAoKcfEDBQKKUolJdgOA2+LZ4VYJIyFtautHcCktERxKOpwzEy4I4mcPoBAQPmUvmbEMfe5n/H3w/UANTbf7suRab483hX9CcPw7yAO5zA6QcEDBhRBmQVAAUoBShYXB+gLgUFARGCVM2AwyFw+gEB+wEF1pYaqFdM2JaDZDaM3FgUheUWEunuLJ6AgIMicPoBAQNGtU7CmA8jabtIRbxsHtekoIsCMi4F1oCIOnzIVgbcqZCjPCOQEFIHcOWw7dgBOQCFwzZiBwT27i+BvftLYG9/ZiilQ70+OOo9/SuU0ocP24jtQgh5ObB3/wjs3V8Ce/eXo2JvkKcfEBAQcAcROP2AgICAO4ij7vQ/f9gG7JDA3v0lsHd/CezdX46EvUd6IDcgICAgYLAc9Z5+QEBAQMAAObJOnxDyMULIFULIdULIZw7bHgYh5BYh5Dwh5HVCyMvtbRlCyJOEkGvt/9Pt7YQQ8kfte3iTEPLQAdj3RUJInhDylm/bju0jhHyivf81QsgnDtje/0oIWWyX8euEkJ/xffaf2/ZeIYR81Ld93+sLIWSKEPJ9QshFQsgFQshvtLcfyfLdxN6jWr5hQsg5QsgbbXv/z/b244SQF9vX/iohRGlvD7X/vt7+/NhW93FA9v4FIeSmr3wfaG8/9PYGwBOIOmo/AEQAbwM4AUAB8AaA04dtV9u2WwByG7b9PwA+0/79MwD+7/bvPwPg7wAQAO8D8OIB2PfPADwE4K3d2gcgA+BG+/90+/f0Adr7XwH8+x77nm7XhRCA4+06Ih5UfQEwBuCh9u9xAFfbNh3J8t3E3qNavgRArP27DODFdrl9DcAvt7f/KYBPt3//dQB/2v79lwF8dbP7OEB7/wLAL/TY/9DbG6X0yPb0zwK4Tim9QSk1AfwNgJ87ZJs24+cA/GX7978E8M99279MPV4AkCKEjO2nIZTSHwIo7dG+jwJ4klJaopSWATwJ4GMHaG8/fg7A31BKDUrpTQDX4dWVA6kvlNJlSumr7d/rAC4BmMARLd9N7O3HYZcvpZQyeVK5/UMB/BSAr7e3byxfVu5fB/BBQgjZ5D4Oyt5+HHp7A45ueGcCwLzv7wVsXlkPEgrgfxBCXiGEfKq9bYRSutz+fQXASPv3o3IfO7XvKNj9b9uvwF9k4ZJN7Dpwe9uhhAfh9e6OfPlusBc4ouVLCBEJIa8DyMNzfm8DqFBK7R7X5na1P68CyB6mvZRSVr6/2y7f/5cQwsSWDr18gaPr9I8y/5RS+hCAnwbwbwgh/8z/IfXe145sStRRt6/NowBOAngAwDKAzx2qNRsghMQAfAPAb1JKa/7PjmL59rD3yJYvpdShlD4AYBJe7/zew7VoczbaSwi5H8B/hmf3j8EL2fynw7Owm6Pq9BcBTPn+nmxvO3QopYvt//MAvgWvYq6ysE37/3x796NyHzu171DtppSuthuTC+ALWH81P3R7CSEyPAf6/1FKv9nefGTLt5e9R7l8GZTSCoDvA3g/vDAIk4zxX5vb1f48CaB4yPZ+rB1Wo5RSA8CXcMTK96g6/ZcAnGqP2ivwBmm+fcg2gRASJYTE2e8APgLgLXi2sRH3TwB4vP37twH8WnvU/n0Aqr4wwEGyU/v+AcBHCCHp9qv/R9rbDoQN4x4/D6+Mmb2/3M7aOA7gFIBzOKD60o4X/zmAS5TSP/B9dCTLt5+9R7h8hwghqfbvKoAPwxuH+D6AX2jvtrF8Wbn/AoB/bL9p9buPg7D3sq8DQOCNP/jL9/Db236NEO/1B95I91V4Mb3fPmx72jadgJcV8AaAC8wueHHEpwBcA/A9ABm6Prr/J+17OA/g4QOw8a/hvbJb8GKD/2o39gH43+ANgF0H8MkDtvcrbXvehNdQxnz7/3bb3isAfvog6wuAfwovdPMmgNfbPz9zVMt3E3uPavmeAfBa2663APwfvnZ3rl1WjwEItbeH239fb39+Yqv7OCB7/7Fdvm8B+CusZ/gcenujlAYzcgMCAgLuJI5qeCcgICAgYB8InH5AQEDAHUTg9AMCAgLuIAKnHxAQEHAHETj9gICAgDuIwOkHBAQE3EEETj8gICDgDiJw+gEBAQF3EP8/rFWcHvxWiCcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "im_read = plt.imread('model_visualization.png')\n",
    "plt.imshow(im_read)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}