19_Hyper-Parameters.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial #19\n",
    "# Hyper-Parameter Optimization\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsl1877BS8m3yt8t_wq2IWji)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "There are many parameters you can select when building and training a Neural Network in TensorFlow. These are often called Hyper-Parameters. For example, there is a hyper-parameter for how many layers the network should have, and another hyper-parameter for how many nodes per layer, and another hyper-parameter for the activation function to use, etc. The optimization method also has one or more hyper-parameters you can select, such as the learning-rate.\n",
    "\n",
    "One way of searching for good hyper-parameters is by hand-tuning, where you try one set of parameters and see how they perform, and then you try another set of parameters and see if they improve the performance. You try and build an intuition for what works well and guide your parameter-search accordingly. Not only is this extremely time-consuming for a human researcher, but the optimal parameters are often counter-intuitive to humans so you will not find them!\n",
    "\n",
    "Another way of searching for good hyper-parameters is to divide each parameter's valid range into evenly spaced values, and then simply have the computer try all combinations of parameter-values. This is called Grid Search. Although it is run entirely by the computer, it quickly becomes extremely time-consuming because the number of parameter-combinations increases exponentially as you add more hyper-parameters. This problem is known as the Curse of Dimensionality. For example, if you have just 4 hyper-parameters to tune and each of them is allowed 10 possible values, then there is a total of 10^4 parameter-combinations. If you add just one more hyper-parameter then there are 10^5 parameter-combinations, and so on.\n",
    "\n",
    "Yet another way of searching for good hyper-parameters is by random search. Instead of systematically trying every single parameter-combination as in Grid Search, we now try a number of parameter-combinations completely at random. This is like searching for \"a needle in a haystack\" and as the number of parameters increases, the probability of finding the optimal parameter-combinations by random sampling decreases to zero.\n",
    "\n",
    "This tutorial uses a clever method for finding good hyper-parameters known as Bayesian Optimization. You should be familiar with TensorFlow, Keras and Convolutional Neural Networks, see Tutorials #01, #02 and #03-C."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Flowchart\n",
    "\n",
    "The problem with hyper-parameter optimization is that it is extremely costly to assess the performance of a set of parameters. This is because we first have to build the corresponding neural network, then we have to train it, and finally we have to measure its performance on a test-set. In this tutorial we will use the small MNIST problem so this training can be done very quickly, but on more realistic problems the training may take hours, days or even weeks on a very fast computer. So we need an optimization method that can search for hyper-parameters as efficiently as possible, by only evaluating the actual performance when absolutely necessary.\n",
    "\n",
    "The idea with Bayesian optimization is to construct another model of the search-space for hyper-parameters. One kind of model is known as a Gaussian Process. This gives us an estimate of how the performance varies with changes to the hyper-parameters. Whenever we evaluate the actual performance for a set of hyper-parameters, we know for a fact what the performance is - except perhaps for some noise. We can then ask the Bayesian optimizer to give us a new suggestion for hyper-parameters in a region of the search-space that we haven't explored yet, or hyper-parameters that the Bayesian optimizer thinks will bring us most improvement. We then repeat this process a number of times until the Bayesian optimizer has built a good model of how the performance varies with different hyper-parameters, so we can choose the best parameters.\n",
    "\n",
    "The flowchart of the algorithm is roughly:\n",
    "\n",
    "![Flowchart](images/19_flowchart_bayesian_optimization.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to import several things from Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import backend as K\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import InputLayer, Input\n",
    "from tensorflow.keras.layers import Reshape, MaxPooling2D\n",
    "from tensorflow.keras.layers import Conv2D, Dense, Flatten\n",
    "from tensorflow.keras.callbacks import TensorBoard\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from tensorflow.keras.models import load_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE:** We will save and load models using Keras so you need to have [h5py](http://docs.h5py.org/en/latest/build.html#install) installed. You also need to have [scikit-optimize](https://scikit-optimize.github.io/) installed for doing the hyper-parameter optimization.\n",
    "\n",
    "You should be able to run the following command in a terminal to install them both:\n",
    "\n",
    "`pip install h5py scikit-optimize`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE:** This Notebook requires plotting functions in `scikit-optimize` that have not been merged into the official release at the time of this writing. If this Notebook cannot run with the version of `scikit-optimize` installed by the command above, you may have to install `scikit-optimize` from a development branch by running the following command instead:\n",
    "\n",
    "`pip install git+git://github.com/Hvass-Labs/scikit-optimize.git@dd7433da068b5a2509ef4ea4e5195458393e6555`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf2/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.\n",
      "  warnings.warn(msg, category=FutureWarning)\n",
      "/home/magnus/anaconda3/envs/tf2/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.metrics.scorer module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.metrics. Anything that cannot be imported from sklearn.metrics is now part of the private API.\n",
      "  warnings.warn(message, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "import skopt\n",
    "from skopt import gp_minimize, forest_minimize\n",
    "from skopt.space import Real, Categorical, Integer\n",
    "from skopt.plots import plot_convergence\n",
    "from skopt.plots import plot_objective, plot_evaluations\n",
    "from skopt.plots import plot_histogram, plot_objective_2D\n",
    "from skopt.utils import use_named_args"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This was developed using Python 3.6 (Anaconda) and package versions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.1.0'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.2.4-tf'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "skopt.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyper-Parameters\n",
    "\n",
    "In this tutorial we want to find the hyper-parametes that makes a simple Convolutional Neural Network perform best at classifying the MNIST dataset for hand-written digits.\n",
    "\n",
    "For this demonstration we want to find the following hyper-parameters:\n",
    "\n",
    "* The learning-rate of the optimizer.\n",
    "* The number of fully-connected / dense layers.\n",
    "* The number of nodes for each of the dense layers.\n",
    "* Whether to use 'sigmoid' or 'relu' activation in all the layers.\n",
    "\n",
    "We will use the Python package `scikit-optimize` (or `skopt`) for finding the best choices of these hyper-parameters. Before we begin with the actual search for hyper-parameters, we first need to define the valid search-ranges or search-dimensions for each of these parameters.\n",
    "\n",
    "This is the search-dimension for the learning-rate. It is a real number (floating-point) with a lower bound of `1e-6` and an upper bound of `1e-2`. But instead of searching between these bounds directly, we use a logarithmic transformation, so we will search for the number `k` in `1ek` which is only bounded between -6 and -2. This is better than searching the entire exponential range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_learning_rate = Real(low=1e-6, high=1e-2, prior='log-uniform',\n",
    "                         name='learning_rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the search-dimension for the number of dense layers in the neural network. This is an integer and we want at least 1 dense layer and at most 5 dense layers in the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_num_dense_layers = Integer(low=1, high=5, name='num_dense_layers')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the search-dimension for the number of nodes for each dense layer. This is also an integer and we want at least 5 and at most 512 nodes in each layer of the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_num_dense_nodes = Integer(low=5, high=512, name='num_dense_nodes')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the search-dimension for the activation-function. This is a combinatorial or categorical parameter which can be either 'relu' or 'sigmoid'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_activation = Categorical(categories=['relu', 'sigmoid'],\n",
    "                             name='activation')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then combine all these search-dimensions into a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "dimensions = [dim_learning_rate,\n",
    "              dim_num_dense_layers,\n",
    "              dim_num_dense_nodes,\n",
    "              dim_activation]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is helpful to start the search for hyper-parameters with a decent choice that we have found by hand-tuning. But we will use the following parameters that do not perform so well, so as to better demonstrate the usefulness of hyper-parameter optimization: A learning-rate of 1e-5, a single dense layer with 16 nodes, and relu activation-functions.\n",
    "\n",
    "Note that these hyper-parameters are packed in a single list. This is how `skopt` works internally on hyper-parameters. You therefore need to ensure that the order of the dimensions are consistent with the order given in `dimensions` above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "default_parameters = [1e-5, 1, 16, 'relu']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function for log-dir-name\n",
    "\n",
    "We will log the training-progress for all parameter-combinations so they can be viewed and compared using TensorBoard. This is done by setting a common parent-dir and then have a sub-dir for each parameter-combination with an appropriate name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def log_dir_name(learning_rate, num_dense_layers,\n",
    "                 num_dense_nodes, activation):\n",
    "\n",
    "    # The dir-name for the TensorBoard log-dir.\n",
    "    s = \"./19_logs/lr_{0:.0e}_layers_{1}_nodes_{2}_{3}/\"\n",
    "\n",
    "    # Insert all the hyper-parameters in the dir-name.\n",
    "    log_dir = s.format(learning_rate,\n",
    "                       num_dense_layers,\n",
    "                       num_dense_nodes,\n",
    "                       activation)\n",
    "\n",
    "    return log_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given dir."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mnist import MNIST\n",
    "data = MNIST(data_dir=\"data/MNIST/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set has now been loaded and consists of 70.000 images and class-numbers for the images. The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Validation-set:\t5000\n",
      "- Test-set:\t\t10000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(data.num_train))\n",
    "print(\"- Validation-set:\\t{}\".format(data.num_val))\n",
    "print(\"- Test-set:\\t\\t{}\".format(data.num_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copy some of the data-dimensions for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The number of pixels in each dimension of an image.\n",
    "img_size = data.img_size\n",
    "\n",
    "# The images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = data.img_size_flat\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "img_shape = data.img_shape\n",
    "\n",
    "# Tuple with height, width and depth used to reshape arrays.\n",
    "# This is used for reshaping in Keras.\n",
    "img_shape_full = data.img_shape_full\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = data.num_classes\n",
    "\n",
    "# Number of colour channels for the images: 1 channel for gray-scale.\n",
    "num_channels = data.num_channels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the performance on the validation-set as an indication of which choice of hyper-parameters performs the best on previously unseen data. The Keras API needs the validation-set as a tuple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "validation_data = (data.x_val, data.y_val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function for plotting images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function used to plot 9 images in a 3x3 grid, and writing the true and predicted classes below each image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        # Show the classes as the label on the x-axis.\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot a few images to see if data is correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAAD1CAYAAAAh4CzYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9a3Bc55nf+Tt9v1/QN6AblwZAEKBIUZREWhrasiTOKDOemYzt2Bp7K3Fmc6upnc1mK1PZqkltKpvUftraZLMftqYqFTuVTCo1O+VMKpZrLduyLEsjWzfeRRIkSAANoIFuoO/3e5/9QJ5XgHiRSALdTeL9VaFIoLtPv6ffPv/zvM/7XBRVVZFIJJL9iK7fA5BIJJJ+IQVQIpHsW6QASiSSfYsUQIlEsm+RAiiRSPYthvt5st/vV6PR6B4NZfCIxWKk02ml3+PoJXKOH3/kHH/CfQlgNBrl9OnTuzOqR4Djx4/3ewg9R87x44+c40+QS2CJRLJvkQIokUj2LVIAJRLJvkUKoEQi2bfc1yaIRPIg5HI5FhcXabfbtNttut0uhUKBRqNBMpkkn8+L5/r9fsbHx7FarQwNDWGz2RgbG8NisfTxDCSPK1IAJXtONpvl/fffp1ar0Wg0aLVarK6uUigUOHfuHLFYTDx3bm6OF198kaGhIQ4cOIDf7ycQCEgBlOwJUgAle8bGxgZLS0vEYjHOnj1LvV4XVmAmk6FarVIul3e8plAosLCwgMfjoV6vMzw8zJNPPonZbMZkMqHX6/t0NpLdptPpUKvVKJVKnDlzhmq1yuTkJB6Ph0AggMfj2fMxSAGU7BmXLl3iz//8z1ldXeX06dM0m0208muqqqKqKp1OZ8drNjc3yWQy2Gw2QqEQ0WiU559/HrPZzNDQkBTAx4hms0k6nebGjRv883/+z1lfX+fb3/42hw8f5uTJk1IAJY8mW1tbZLNZlpaWiMfjpNNp6vU6rVYLAEVRMJlMGAwGbDYbJpNJWIaNRoNyuUy9XqdYLJLP59na2sLlcmG32+VS+DGi0+lQKpUolUpUKhWq1SrdbhedTodO15v9WSmAkl2l2+3yzjvv8POf/5z5+XnOnDlDq9US4geg1+vx+/04HA5mZmYIhUIUCgUKhQIbGxtcvXpVWAc6nY6PPvqITCaDx+PB7Xb38ewku0mtVmNtbY319XVarRZ6vR6n04nX68VsNvdkDFIAJbtGu92m1WqRTqdZXV0llUpRqVQAMJlMGI1GPB4PFouF4eFhHA4H0WiUYDBIsVikUCjQbrdZXl4WotlqtahUKpTLZdrtdp/PULKbdDodYfmpqoqiKNhsNlwuFyaTqSdjkAIo2RW63a4Iabl48SIffvgh9XodVVWx2+34/X6i0Sjf+c53CAaDQghdLhdWq5Vms0mz2eTNN98klUqRz+dJJpN0u12q1SqVSuU2f6Hk0aZerxOPx0kkEgA4HA4OHTrEiRMnsFqtPRnDngtgu91GVVUR/6UoCorySWEGnU6HXq/f8fderf8lu4eqqtTrdcrlMsVikVwuB9xc7losFgKBACMjIxw5coRwOIzT6cRoNApfYLPZpNFoiJAXo9G443uy/f+SRxttA6zZbJLP5ykWixgMBoxGI263uyebHxp7KoCNRoOFhQXy+TyXL19ma2sLm82GxWIRwuf3+zl48CAWiwW73S4+hF6ZwJLdQVVVKpUK+XyeRqMB3Lyju1wuDh8+zLe//W1GRkaYnJzE4XBgMBh2OLsXFha4evUqH3zwAWtra9RqNdrtNmazmdnZWWZmZnA6nf08RckuUa1WyWazXLt2jbfffptiscjU1BQ+n6+n4gd7LIDtdpvNzU2SySRnzpxhZWUFp9OJw+FAr9djNBoZHR0VO3ytVguz2YzNZsNg2J2hfdrilOwdrVaLZrNJt9sVlp/X62ViYoJf+7VfY2hoCI/Hc8dQlmw2y9WrV1lbW6NUKomQGYPBgN/vJxgM9swxLtlbms0mhUKBdDrN0tISrVaLubk5RkdHe77Lv6cCWKlUePfdd1lYWODatWtkMhnMZrNY3uh0OpxOJxcvXsRsNmO32zGbzYTDYWw220O9t9lsxmq1EgwGmZubw2az4fP5ZBzZHqHT6RgbG8Pr9fL7v//7HDlyBLvdjsvlYnR0FL/fj9Vqvat7I5lMcvHiRVZXV2m1WtLf9xhTr9dJp9PkcjkajQYmk4mZmRmmp6d7vsu/pwJYr9c5d+4cFy5cIJ1Oix3BO6HT6bDZbBiNRsbGxh5aAF0uF263m9nZWVwu1z2tD8nDo9PpCIVChEIhhoaGeP755zGZTCKDw26339USV1WVdDrNtWvXyOVywm8seTxpNpvkcjmKxSLNZhOLxcLExAQHDhzAbrf3dCx7KoBWq5UTJ07g8/lIJpOUy2VsNhtms5lGo0G1WqXRaFAsFmm32xSLRVRVFR+O5gzXHOR3QlviamlSzWaTVqtFPp/HYrGg1+tZWFggHA4zNjYmfYs9wGw243K50Ov1GAwGscn1abrdLmtra6TTaZaXl8nn81QqFVRVxWw243a7hajKJfDjQzqdFjngFosFt9tNIBAgEAj0fI73VAAdDgevvPIKuVyO1dVVisUioVAIr9dLLpdjc3OTQqHAysoKhUKB+fl5SqUSqVSKZrMpqoFoMWJ3PIFbznRtV1GLKteo1+sEg0HK5TInTpzo+R1mP2Kz2T6XBd/tdrly5QqXL18Wm2TdbhcAi8VCJBJhYmKCiYkJRkdHpQA+JiQSCbH5YbFY8Pl8RCIRwuHw4+UD1Ov1DA0NYTab0ev1VKtVPB4PTqcTp9OJy+WiUqng9/splUr4/X6q1Sqbm5s0m02CwSBOp5NMJkMmk7nt+IqiYDQa0ev1eDweTCYTFy5cYGlpaUfYjcViwWw2y82QAUErh1WpVFhcXOTKlSukUikxX5r1Nzs7SzQaxW63YzAY5Pw94miB8uVymWw2S71eF3sBBoOhL3O8pwJoMpmYmppCVVW63a6I9tbpdOJ37bFWq0WpVKJer7OyskK1WmV8fByfz8fGxgaJROI2v5BOp8NoNGIwGAiFQpjNZv71v/7XJJNJGo0GtVoNs9ksKkvI+MLBoNVqcePGDZLJJG+88QbvvPOOyBW2WCw4HA4mJyf56le/SjgcJhAISNfFY0ClUqFYLJJIJFheXkan04m51fzFvWbPA6E/76ZDt9sVAbH1el0Exbrd7h15pNtRFEX4mLZHjmvhExaLBafTKTdABoxut0ulUqFQKFAqlUQSPCCqvgwNDREIBPB6vbsWEiXpL41GQ1j+2i6/ZsD0yzgZmG+WTqfDarVisViw2Wx0u12MRiM6nY5IJMLw8PAdX6coCq1WiytXroiYw0qlgsfjIRwOMzc3x6/92q/1NMFacm86nY6oFVgoFHbc4ILBIM8//zyHDh3i8OHDwrUhefTZ2tpifn6ejY0Nut2uCH1zOBxSAOGTFLhPW2r3ugC0ZXShUBA7zVoGwdDQEF6vF7fb3dcPWXKTTqdDuVwmn8+TTqfJZDJid99oNIpiCeFwmGAwKOJCJY8HzWaTYrFIvV4Hbl7n2rXZr9XZQAng/aItpfL5PD/60Y/48MMPWV5eBmBmZobf+Z3f4eDBg3g8HsxmsxTAPrO5uclf/MVfsLa2xocffsjm5iapVAqAiYkJpqenOXHiBF//+tdFsQTJ40OhUGBtbY1sNouqqgQCAf7aX/trjI+P43K5+jKmR14AtQT81dVVrl+/Lkqsu91uJicnGR4elqXUB4Rqtcq1a9dYWlpicXGRfD5Pu91GURTcbjejo6OMj48zOTkpYjgljz7aKq1Wq1EoFKjVaiiKgtVqZXR0lEgk0jc3xyMtgLVajdOnTxOPx1lfXxfiZ7Va8Xq9hEIh3G63DJ/oM+12WyTAx2IxVlZWqFQqwlVhMBg4fPgwv/3bv834+Dg2m+2uwdOSRwtVVUmlUhQKBS5evMivfvUrarUa4XCYyclJDh06JCI4+sEjLYCNRoPl5WWWl5fJ5XLU63WsVqvIA3a5XNhsNnkh9ZlOp0O1WhVB7qlUikajQafTEUUTwuEwR48exeVy3VYKS/LooqoqxWKRVCpFPB7n+vXrIjLD7/czMjKC3+/HaDT2ZXyPpAA2Gg3S6TTr6+t88MEHLC0tkclk0Ol0nDhxQjRVCYVCWK1WeTH1mfX1dX7yk5+wvLws+oNoFWPGx8dFmSwtaF7O1+ODZgEuLS2xtbVFrVbD7Xbj9/vxeDwikaFfc/5ICmC9XhdhFB9++KFoum0wGHj22Wf52te+xujoKKFQqN9DlXBTAH/wgx+QSCRIpVJiF1ArfDE3N8fExARer7fPI5XsNt1ul1QqRSwWEzc/RVFE7b9+++cfSQFsNBpsbGyIjA8Ar9eLw+FgeHiYYDCIw+Ho8ygl5XKZXC7HxsaG8ANplp/H48FutzM3N8czzzzDyMhIv4cr2SO01LdarQaA0+lkYmKCUCjU942uR1IAK5UK165dIxaLUa1WARgbGyMYDHLgwAGmpqb6/sFKblb9uHLlCvPz86yuropCp2azWVjoX/7yl3nppZdkyMtjiqqqZLNZ4vG4KGji8/k4duwYkUikb74/jUdOALXQF62gomZRaH4kLeVN+pH6R7fbpdvtit7AiURiR5FTRVFEsVStRYJMd3v80Po8l0olca3a7XY8Ho+oG9nv2NxH6lvX6XRot9vk83muXLnC+vo6zWYTm83GCy+8wBe+8AVmZmak9ddnGo0G9XqdK1eu8Nprr7G5uUmtVhMCqBWvGBsbE0Hq8ob1eNHtdkWDrNXVVa5evYrBYGBkZITp6WmOHz+O1WrtuwX4SKVGNBoNURqrUChQrVYxmUw7ttV71U5PcncqlYqYp3Q6LQrd6vV6YfkFg0FGRkZkmNJjSrfbFZafVvBCURQcDgd2u10UO+43j5QFuLa2xjvvvMP8/DzXr1+n0WgwNTVFKBRibm6OgwcPysT5PqOqKpcvX+bs2bO89957LC4u0mw2abfbOJ1ODh8+zMjICL/3e7/H7OwsgUCg30OW7AHNZpPz58+zsrJCLBajVCoxOjrKzMwMoVBoYG56j4QAdrtdOp0OhUKB1dVVksmkuKN4PB6CwaDwJ0n6i9bSYHV1lVQqRbVaFUtfk8lEMBgkEokwMjIi0hQljx9a0VutF1Cr1UKn0+FwOAaqwMUjIYDJZJL19XXeffddXnvtNYrFInBzN+k3fuM3mJ6eljF/A4Kqqly9epUf//jH5HK5Hd3dhoaG+PrXv87U1BRjY2OyQMVjTKfTIZ/P76j4o5W/MplM0gK8H0qlEolEgtXVVRYWFlBVVVQOjkajTE9Py14fA0Qmk2F5efm21pZWq5WZmRkOHDiA0+nc9c0qLen+Xh3ldDrdwFx8jzOqqtJoNHYUP9UquA+C709joAWw0WjQbrc5f/48r7/+Ojdu3KDT6WC325mZmSEajTI1NSUS6CWDTbfbpVarUalUMBqNd630DTcvFs1S0KxErY3Cp/sGt9tt0Wz78uXLIuB2O1r3wMnJSY4ePSo61kn2hna7TSwWY35+nnw+D0AoFOLIkSNEIpGBuQkN9Deg1WpRr9dZWlrinXfe2VHsdHR0lLGxMUZGRqQj/RGh2+3SaDREmMy9mp9rrQ50Op0QwO39Y7a3SW21WlQqFRKJBO+///5dOwhqbVPn5uYwmUxSAPeQdrvNxsYGKysrlEol4a+PRqP4fD4pgPdCVVXa7TYffvghCwsLnD17lkKhgKqquN1uIpEIzz33HOPj43Lp+wiRyWT4y7/8S3w+32cGPzscDsbHx0VlH51OJzZUtKKaGlq3sUKhwNWrV0Wu8XY0ETUajUQiEQKBALOzs7t/kvucVqtFNptlY2NDhL/c60bXbwZWAJvNJu+88w6vv/466+vrZLNZXC4XPp+PaDTKyy+/TCgUwul09nu4ks/J5uYmf/Znf/a5nhsIBHj++edxOBwiu6dQKFCv1zl37hw3bty44+u05kp3o91u4/V6OXjwINPT0/d9DpJ702w2icfjrK2tkclkdvgAB5GBE0CtcqyWSK9to8PNXcSnnnqK2dlZUfxA7iIOHi6Xi+HhYYrFovD/aHzei6FWqxGPx7FYLNjtdhRFoVarCUtve1qdhrZc1uv1eL1eTCYTNpttR9jF1NQUgUAAl8s1MMuwx4lms0kikSCRSNBsNkX6m9FoFJ3+nE7nwHz2AyeA3W6XfD5PLpcjHo8Ti8XErt7k5CSvvvoqkUiE0dFRLBbLwHyQkpsoisLIyAhHjx5leXlZuC7ul2KxyIULF8T8Koqyo4+09jct73v798BqtTI3N4ff72dsbGxHma1wOMzMzAxDQ0PSB7gHVKtVrl69ytLSkghX09qcTk9Pc/DgwYHK1R+Yb4Dm96vVaiwvL5NIJEQCtcfjET0jhoeHxZd3UD5EyU6Gh4d54oknMBgMlMvlu4amaDu62rxrlX22P2YwGHA6nSJ8YrvFb7VasdvtGAyGHVaexWJhbm4Oj8fDyMjIDjeJ3+8Xqwf5/dldtA2qbDZLLpej2WwCN8tfaZbfoFX7HhgBbLfbZLNZtra2+Pf//t9z6dIl0eHt6NGjfPGLX+Spp57i+PHjcgdvgNHpdLz00kt84Qtf4Ny5c7z99tuiiMWnUVWVra0tisUiCwsLLCws3PYcm83GiRMnhBWxXegmJyeZnZ3F6XQSDAaFOGohNFqoy3bR1Ov1Qkhl0Yzdo9vt7ghFWlpaolKpoNfrmZ6e5ujRo4yOjg6cy2pgVKTT6VAsFslmsySTSeLxuPD9OZ1OwuEwfr8fm80mxW/AcTgcOBwOwuEw0WiUTqdzR99ft9vFZrNRLBZFeMyncblcRKNRIYDb6wZGo1Gi0ehtAijpH5oQNptNDAYDdrsdn89HKBQayCLFA6MkpVKJ9957j9XVVWKxGNlsllarhaIoBAIBDh06RDgcll/yR4ipqSl8Pp9Y/t5pGdxsNkXTpO1LYA2tebbRaLzNmrNaraKDnPxe9BfNH2s0GrFarXg8HgKBAGazmZdffpmTJ08yNDTU72HexkAIoFbkNJFIiPaWmjWgKApmsxm32y0bHD1i2O12Gae5j9BE0Ol04na7cTqdYiUQDocHKgVOo+8CWC6XSSQSLC4u8vbbbxOPx28LnZBIJIONJn6RSIQ//MM/pFarCat9amoKk8k0kFZ63wVQa3GZSCRYWloiHo/fMZdTIpEMNoqi4HK5OHnyZL+H8rnpuwDWajXW1tZYX1+nWq3SarWEr8jhcGCxWHA6nXLnVyKR7Dp9V5Rarcb6+rroG6HFDm1vnKPVEJMCKJFIdpOBUxSdTofT6cRisXDy5EkOHDjAsWPH8Hg8sn+ERCLZVQZOAPV6PT6fD7/fz9/4G3+DV155BYfDIcVPIpHsOn0XQLvdTjQaxWw2c+rUKarVKoFAALfbzdjYmEikluInkUh2m74LYCgU4rd+67fodrt861vfEu0TFUXBarUOVP8AiUTyeNF3AdTr9aKXrwyalUgkvUS5n1JFiqKkgJW9G87AMaGq6r6qty/n+PFHzvEn3JcASiQSyePE4OWmSCQSSY+QAiiRSPYtUgAlEsm+Zc93gRVF8QFv3vp1GOgAqVu/f0FV1eYuv9+/AV6+9asNCKqq6tnN95DspA9z/MfA3wfat97n76qqup+c+j2nD3P8ZeD/Bo4C31ZV9b/s5vHF+/RyE0RRlH8BlFVV/Vfb/mZQVfX2eum7837/E/C0qqp/dy+OL7mdXsyxoigvAx+oqlpVFOV/AF5SVfVbu3V8yb3p0RxHARfwT4DX9koA+xIHqCjKfwDqwNPALxVFKbLtA1UU5RLwu6qqxhRF+VvAPwJMwAfAH6mq+nkbjf53wP+22+OXfDZ7Oceqqr617df3gb+1N2chuRd7PMexW8e4d6Pnh6SfPsBR4KSqqn98tycoinII+BbwRVVVj3HT7P6btx77rqIox+/x2glgEvj5ro5acj/s6Rzf4u8Br+/SeCX3Ty/meM/oZybI9z+HJffrwLPAR7fS4azAFoCqqn//M177beC/3Ie1KNl99nSOb1kVx4EXH36okgdkr6/jPaWfAljZ9v82O61RrfWXAvxHVVX/6QMc/9vA//iAY5PsDns2x4qi/AbwvwIvqqp6ezs5Sa/Y6+t4TxmUMJgY8AyAoijPcHPpCjd3nb6pKErw1mNDt5a290RRlDnAC7y3J6OVPAgxdmmOFUV5Gvi3wO+pqrq1ZyOW3C8xdvE67gWDIoB/CQwpinIZ+IfAAoCqqleAfwb8VFGUi8AbwAh8pu/g28D/q8o8v0FiN+f4/wQcwPcVRTmvKMprvTgByWeya3OsKMoJRVHiwKvAv711zF1H5gJLJJJ9y6BYgBKJRNJzpABKJJJ9ixRAiUSyb5ECKJFI9i33FQfo9/vVaDS6R0MZPGKxGOl0el81JJFz/Pgj5/gT7ksAo9Eop0+f3p1RPQIcP963DJ2+Ief48UfO8SfIJbBEItm3SAGUSCT7FimAEolk3yIFUCKR7FukAEokkn1LP8th3ZVut0un02F1dZVsNksmkyGbzWI2m7Hb7Xg8Hubm5jCbzZjNZnQ6qeMSieT+GTgBVFWVVqtFvV7nl7/8JWfPnuXcuXOcO3cOn8/H6OgoR44c4Y/+6I/w+/34/X4pgBKJ5IEYOAEEhABubW2xsrLC1tYW5XIZvV6P2Wxma2uLTCaDwWDA4/FgNBr7PWTJPeh0OtTrdWq1Guvr6wBMTU3hdDof6Hi1Wo1Go4HBYMBoNKLX6zEYBvKrLLkH7XabcrlMrVYjHo/TarUIhULYbDZcLhd2u33PxzBw35put0upVCKXy3H+/HneeustGo0GiqJQqVSIxWKYzWbOnz/P2NgYoVAIq9Xa72FL7kG1WiUej7O4uMh3v/tdut0u//Jf/kuefvrp+z6Wqqqsr6+zsbGB1+vF5/NhtVrxeDzcKrcueUQolUp8/PHHrKys8Kd/+qdks1m+8Y1vcOjQIY4fP87s7CyKouzpvA6UAHa7XVqtFul0mq2tLbLZLJXKzYrbOp2OTqdDu92m0WgIK0DWMxx8Wq0WxWKRfD5PIpEAoNF48Cr2lUqFTCZDu92m0+ng8Xhwu91SAB8hut0u1WqV9fV11tbWiMfjpNNpYrEYVquVgwcPimt7Xwhgp9OhUqmQzWb5r//1vzI/P8/169f7PSzJLlAsFrl06RKxWIxKpYLRaHzgG5eqqqyurnLmzBlxQzx69Ci///u/j9ls3uWRS/aCVqtFrVZjcXGR//yf/zPr6+tkMhkajQY/+9nP+OCDDxgdHeXYsWPo9fo99fH3XQC73S7tdptWqyWWvrFYjMXFRYrF4h1fo6oqnU6HVqtFs9mk2Wyi0+lQFEX8KxkcGo0G2WyWQqFAu91+KJ+tqqrU63VKpRKlUolyuczIyAidjmz+96ig+YSLxSKrq6skk0larRaqqpLJZCiXy1QqlZ6s7vougKlUiosXL5LNZrl69SqZTIYzZ86QSCTE8vfTaHePYrGI2WzG4/EQCASw2WwMDw/j9Xp7fBaSO6GqKqqqUigUuHz5MpubmzQajYfy2SqKgs/nIxqNsrCwwMLCAul0mnq9jtFolBtijwCFQoHr16+zvLxMOp2mWCzSbrcxGAzMzs4SDAYJh8MYjcY9j/DouwBWKhUWFxdJJpOcPn2aTCZDPB6nVCrd9TXNZpNMJkO322VpaQmXy0Wz2cTtduNyuaQADgiqqgpfz+bmJul0mm63+9DHtdlseL1eFEWhWCxSqVSEP9BgMMgVwIBTr9dJp9NkMhnhy4ebfv5AIMDY2Bgulwu9Xr/nY+m7ACaTSV5//XUymQybm5vU63Xq9fo9X1OpVJifn8disRCLxbBYLAwPD+NyuXjllVdQVRWn04nb7e7RWUjuRKFQYGtri6WlJWKxGN1ul7m5OcLh8AOHwMDNG2C1WqVYLLK1tUUul6PZbNJqtTCZTFIABxTthri2tsZbb71FLBbbsRmm1+uZmpriySefJBAI9GRMfRfAdDrNu+++Sy6XEz68z6JSqbC0tCSeq9Pp8Pl8OJ1OwuEwkUgERVGkAPaZcrnM2toa6+vrJBIJbDYbU1NTTExMPFSMlxYnWiqVyGQyFItFms2m9AMOOJq/P5VK8dFHH5HJZGi1WuJxg8HAxMQEhw4d6tkqrm8CuLy8zOXLl/nwww+FAxS45xJJr9eLH4PBgKqq1Go1sczqdru899575PN5XnzxRcLhsLQG+kC1WqVWq3HlyhXefPNNEdoQCoU4evQo0Wj0gS1AzVG+vLxMNpvd5ZFL9pJqtUqpVCKVSpFKpSgUCjtuWnq9nmAwyPj4+EOtEO6HvgngpUuX+N73vsf6+rpY8n7Wro9er8fhcKDX67FYLCImsNlsUiwWKRaLvP766/z4xz9Gp9Pxm7/5mz3xI0h2UiwWSafTfPDBB/yn//SfUBQFh8PB2NgYL7zwAtFo9IE3QlRVZWNjg0uXLpFMJnfFpyjpDaVSiY2NDRH7t335q9PpMJlMjI+PMzc31zPDpecCqFkHWljE9u3uu520x+PB4/Hgcrnw+XyYTCYsFgutVot4PE61WiWZTFIul8UdJZfLsbKygtPpxO/3S0uwR6iqytbWFjdu3CCZTNJoNAgEAhw5coSZmRmcTidGo/GB5kMLfdIEtlwu78EZSPaKUqlEIpEgn8/vuHHpdDqGhobw+XyYzeaeXqs9F8DNzU0SiQTLy8skEgkR63e3lBedTseBAwd46qmnCIfDzM7OYjKZsNlsYpmVTqd5/fXXWVxcFK9bXV3l5z//OdPT03zpS1+S4RE9QlVVLl++zE9+8hMWFhao1+tEIhH+zt/5O4yMjBAIBB4oYFmL/9PS6q5evUq325WZQI8QGxsbnD59mlgstmPezGYzBw4ceOjNsQehZwKohSlsbm5y/fp1kskktVqNVqt1291AC2Ww2+0YjUZGR0eZmpoiFAoRDocxGAxYrVZqtRqFQgGj0Sieq10U+XyepaUljEYjm5ub2O32nm2t70dUVaVardJoNEQqY7VaFSXM/H4/Xq/3gYsWdLtdstksuVyOcrlMu9JY5wsAACAASURBVN0WKwGr1YrBYJBVgQacWq0m5m/7NW82mxkZGWFsbKznef09EUBVVSkWi5TLZX784x/zgx/8gFwux9bW1m13cZ1Oh9vtxmq1cvjwYSKRCC+88AJf/vKXMZvNOBwOYS22Wi2CwSBbW1v86le/Yn19XVyE8/PzbGxs8MQTTwgr5Etf+hJOp1OK4B7Q6XRYWFggkUhw+vRpzp49i9lsJhgMEolEmJqaeqjKPc1mk1/96ldcu3aNpaUlAHw+HxMTE0SjURwOB1arVYrggKKqKqlUivn5eRKJhLjmtYpOX/nKV5idnSUcDvd0XD0TQC3dLZfLsbGxQb1ep91uizuBoiiYTCbMZjOBQACn08nY2BhjY2NEIhECgYAof6RhNBrxeDy0222REN9qtXYUS/B6vaysrKDT6UQWgkyX2320m1wqlSKfz1OpVESWjtPpxGq1YjKZHujYWpGMVCrF+vo6lUpFbKwEAgE8Ho+0AAcYLWe7Wq1SKBSo1WpCAHU6nbjmg8Fgz/O5e7YE1mKAarUaxWLxtt07q9XK5OQkgUCAb33rW0xPTxMIBHC5XLjd7js6R/V6vbAqXnjhBfx+P7/4xS/4+OOP6XQ6dDodlpeX+f73v8/hw4d5+umnUVWVoaGhB74YJXem3W6zsLDAuXPnxA5fOBzm5Zdf5oknnnjgpW+n0yGfz5PNZrlw4QLvv/8+m5ub6PV6Dh48yNe+9jUmJyexWCzSsh9AVFUlmUySzWZF6uL2zA+z2YzNZiMSiTA2NobFYunp+HomgFpeaLvdpt1ui7/rdDq63S5GoxGfz8fw8DCHDx/miSeewG633/OOoFmNqqoyMjJCvV7n/PnzKIoiCiYUi0VKpRIul4tKpUKz2ZSO8z2g2+2Sz+dJJpNUKhU6nY4IfQkGgw8sTlqMZ6lUIp1Oi2whnU6Hx+NhcnKSUCgkxW9AUVWVSqVCPp8nn89TKBTEY3q9HqPRKPzEvSiA+mn6nglitVrx+XyMjY3xzW9+k7GxMSYnJ8WmxudBS6FxOp1cunSJtbU18vk8mUxmj0cvUVVVpKZtbGwQi8XElzwSifDiiy+KJeqDUK1WOXv2rMgoKZfLOBwOPB4P4+PjzM7OYrPZ5PJ3QFFVVdT5y+fzOx7zeDw899xzTE1N9UX8YAAE0Gg0MjQ0RCQS4cSJE4yNjYl4oM+LTqfD7/djsVgIhUL4fD5RMEFDs/o0S1SyO2g9XBqNBvl8XlRmAfB6vRw8ePChytW3Wi1WVlZYXl4mn8/TaDQYGhrC7Xbj9/sZHh6W4jfAqKoqUhZrtdqOx+x2O9PT00xOTvatlmNfBHD7zu/ExAR/8Ad/wOjoKKOjozidzge+YDRx2/6jUSqVeO+990gkEpw6darnvobHGc2/W6lUKJVKeL1eotEoIyMjD73ZpLkx8vk8rVYLnU6H0+kkEAjgcDh26Qwke4UWkra+vi5ifnU6nTB8nn76aUZHR7HZbH0ZX18EcLswTUxM8J3vfGfXkp/vJH5aTboPPviAZDLJs88+SzAY3JX32+9ovlZtl69cLnPgwAFmZ2f3RAABHA6HaJ4jGWw0AUwkEiJzx2AwYLFY8Pv9PPXUUwwPD/etr09PBLDb7YpGNrlcbk+Ov7W1JZzk6XSaarUqHteCqg8dOiT8i5LdQVEUDAYDJpMJl8vF0NAQnU6HbDZLqVSiXq9jMpnu26rXQl/K5TLxeJyVlRVRINfpdBIKhXC5XDKcaUDpdrsUCgXK5TIrKyssLi6Ka1/L+9V2gPvpw+2JALbbbebn5/n444/Z2NjY8aXdjS9wp9MhFouxvLzM4uIi8Xhc5ARrQdNer5df//VfZ3JykqGhoYd+T8lNtJ14q9UqKvlqBQsymYyo0mO32+9rrjudDuVymUwmw+XLl7ly5YooeBoIBJienpbzOMC0223W19fZ2tri4sWLnD59WlyTWjETu90ucvz7Rc8CoXO5HJubm1Sr1YdWe23Z1el0aDQaVCoV4vE4S0tL5HK5HcUVtNJZJpMJk8n0wIn4kjuj1XDc/qXWenXE43EuXLiAx+MhHA7fM1RF6+/S6XREn5d8Pk8sFqNare4om7S9/4tkcNE2x1qt1o7QN5PJxNDQ0ECkpvbMAlxaWuLs2bNsbW099PE6nY5YXiUSCTKZDG+88Qbnz5+/rUacwWDA6XSKsBq9Xi8FcJfRYrmGhoYIBoNkMhnW19f52c9+xurqKpOTk5w6deqeO33pdFpUeMlms1SrVVHsNJ1Oi+dpcyezeQabbrdLrVajXC7vKHoKN6MDjhw5wtTUVN8b2vfs3ZvNprgb7MaxtO5RsViMVColmqtox9eEzul0EolECIVCWCwW2TNij9ACk0OhEBsbG6JeYzKZxGQysby8fM/sm1wuJxLlc7kc9XqdTCZDpVKh1WrtmDOr1YrL5ZI7+QOMFgCtVevW0PzxWgpjv634vscBPgipVIof/ehHbGxs8Itf/IJMJkM6nRbVoeFm4xyHw8HRo0f5xje+QTgcFknz/Ta7H0eMRiMnT57kwIED+Hw+3G43m5ubrK6ukkgkuHjx4me+Xq/XC/eGVuS23W7fdgEdPHiQU6dOicIYksGj2Wxy5coVUa4OEBsf0WiUU6dOMTw83Pdezo+UAGq9IHK5nKgqe7fS6Hq9HqvVisfjYWpqikAgIMomSXYfrail0WgkEomwsbFBq9Vic3NT5PPeKwDd6XQKQTMYDKJ8Wrvd3vE6zYIYGhqSczmgaCmv2WyWVColAuO1UncOh4NgMIjX65UW4P1w+fJlfvjDH7K1tcXHH38sKkvcCa1BUjQa5eDBg7hcLlkUdQ/RBNDlcvFbv/VbPP/88yQSCVZWVkQduHs1LZqamiIajQqLbmVlhddee43NzU2uXLmyo/qzXq+X3d8GFC1uc2triwsXLnD69GlhoGhpr8PDwyIcrd+rsb5lgmzn04HL2oXyaYshkUjw0UcfiaT7OxVU1dBKMXk8HoaGhvoWaLmf0HxyWoaGVsqsUqmI2o9344knnmBubk78Pj8/z4ULF1AUhYWFhR3PlTvAg0u326Ver1Mul9nc3GR9fV08poVLORwOUeGp3/Q9E6RQKHDt2jXRwrJarfLLX/6Szc1NIXAaa2trfPzxx6LenxYKc6el1dGjR/nbf/tvE4lEpOXXJxwOB5FIhHa7zcjIyD2XwJ/OBDIajXi9Xkql0o7507J6NjY2cDgcfY0hk9xOo9EgHo+ztrZ2W3/vUCjE4cOHGR0dHZgbWE8F8E6FCLQeD1qliHw+zw9+8AOuXr1KsVjckdHx6dfeawk0NTXFb//2b/fdxN7PWCyWB96p1ev1O/yC26lUKmQyGXQ6nRTAAaPVaomWCNu7vgHCHz9ITcp6IoB6vZ7JyUmOHTsmfHcayWSSH/7wh8IcrtfrxONxarXajuBJQNT50/7/aRRFYXR0FJ/PJ3sCPyZsr+KjoVWClimNg4fWvKperwuXh9FoFE3Pn332WSYmJvaXBajX65mdnUVRFNLpNNevXxePxeNx/vzP/xz4RNQ+vet3J+70d4PBwNTUFIcPH2ZiYkIK4CPO3ZbMLpeL4eHhHo9G8nnQAqC3h6SZzWasVivT09N88YtfHKjeLT0RQG2HcHR0FI/Hg9VqFWEO20tj3cm5vf137QPVKslq3eFMJhPRaFREmE9OTjIyMtKLU5NIJNvQlsCpVErEb2ptbC0WC2azeaDCl3pmAR44cIBgMMh7772Hw+GgXq+L6h4PcjyHw4HFYiEcDjM0NMR3vvMdDh8+LJrwWCwWaQE+4tzNzSHndXCp1Wpcv36dxcVFcX07HA68Xi8ulwur1TpQfvmeCKCiKFgsFrrdLuFwmIMHD1IsFkW1kM8Kkv00ZrNZNFGemZnB5/MRiUTw+XzY7faB+5AlD8b2TTNtdSDFb3DRsngqlYrw4WvzpmX6DNoc9kwAXS4Xdrud3/zN32R6epqVlRWuXr3KysoKf/VXf3XbjtG98Pv9fOUrXyESiXDq1Cl8Ph9OpxOTySQ+4EH6kCX3j1Zqf/tGmBYALW9ug4cmfo1GQ9Tl1ELYDAaD+Bm0YiQ9W4zrdDrhCxwfH0dVVRHLFwqFREPzbrdLs9kUneK0XsDbE+lDoRAjIyOEw2FGRkZ2rZq0ZHBot9uUSiUqlQqqqoqqPg6HYyACaCV3RyuPpgnd9hvXIIkf9CEQOhQK4fF4iEajnDhxgkQiwTPPPEMqleLMmTNkMhmuXbtGpVIhGo0SCoWYm5tjdnYWuHmn8Xq9HD58WDTcljx+5PN53n//fRKJBJ1OB4/Hw4svvkg0GuXAgQP9Hp7kDmiurvHxcdrttqj/6Xa7GR4eHsgeLj0XQLPZjNlsFgGsdrudZrNJMplka2sLk8nE5uYmiqIQDAaJRCIcOHCAJ598UviDbDYbgUBANsN+jGk2m2SzWQqFggijGB0dZXp6WgY/DyCa28lgMIj0U4vFItLfbDbbQGZk9X0/2uVyMTc3RzQaZXJyUlR7aTabImTG5/PtWObq9XpsNpuo9ix5/LBaraKp+rFjx/D7/Tz33HOEw2HZ0GpA0el0eL1eXnnlFTY3N4Gbcb5zc3OMjIzg8Xj6PMLb6bt6mM1mQqEQcLNDnEQCn/SLVhSF48ePEw6HOXToED6fr99Dk9wDm83G7OwswWCQq1evYrFYGBkZwefzDWQXv74LoERyJ0ZGRnj11VcBOHDgAA6HYyAvIMlOtBhdg8HAqVOnKBQKOJ1ObDYb4XC438O7DSmAkoEkEAjwu7/7u/0ehuQ+0el0otXlyZMn+z2cz2QwEvIkEomkD0gBlEgk+xYpgBKJZN8iBVAikexbpABKJJJ9i3I/VVgURUkBK3s3nIFjQlXVQL8H0UvkHD/+yDn+hPsSQIlEInmckEtgiUSyb5ECKJFI9i1SACUSyb5lzwVQURSfoijnb/0kFUVZ3/a76bOPcN/vZ1YU5S8URbmhKMoHiqJEd/s9JDvp9Rxve99vKIqiKopyfK/eQ3KTPlzHX1YU5ayiKG1FUb6528fX2PNcYFVVM8AxAEVR/gVQVlX1X2mPK4piUFW1fZeXPwh/D8ipqnpAUZRvA/8H8K1dPL7kU/RhjlEUxQn8z8AHu3lcyZ3pwxyvAv898E928Zi30ZdiCIqi/AegDjwN/FJRlCLbPlBFUS4Bv6uqakxRlL8F/CPAxM0v+x+pqtq5x+G/CvyLW///L8D/oyiKosrt7p6yx3MM8L9z8+b2v+zRKUg+g72cY1VVY7eO0d3Lc+inD3AUOKmq6h/f7QmKohzipvX2RVVVjwEd4G/eeuy7d1n6RIA1gFt3pAIgi8j1hz2ZY0VRngHGVFX9//Zm2JL7YK+u457Qz3JY3/8cd/lfB54FPrrVTMUKbAGoqvr393Z4kl1g1+dYURQd8H9xc3kk6T+P9HXcTwHc3hW9zU5r1HLrXwX4j6qq/tP7OO46MAbEFUUxAG4g8zADlTwwezHHTuAI8ItbF9Mw8JqiKL+nqurphxyv5P7Zq+u4JwxKGEwMeAbE8mby1t/fBL6pKErw1mNDiqJ8Vt3814A/uPX/bwI/l/6/gSDGLsyxqqoFVVX9qqpGVVWNAu8DUvwGgxi7dx33hEERwL8EhhRFuQz8Q2ABQFXVK8A/A36qKMpF4A1gBO7pO/ge4FMU5Qbwx8Cf9GD8ks9mN+dYMpjs2hwrinJCUZQ48Crwb28dc9eRucASiWTfMigWoEQikfQcKYASiWTfIgVQIpHsW6QASiSSfct9xQH6/X41Go3u0VAGj1gsRjqdVvo9jl4i5/jxR87xJ9yXAEajUU6f3j/hVseP778IDDnHjz9yjj9BLoElEsm+RQqgRCLZt0gBlEgk+xYpgBKJZN8iBVAikexbpABKJJJ9Sz/rAUokEgkA7XabbrdLo9Gg07m9vqper8dsNlOv19nY2KDdbmO32zEYDFgsFoxGI1arFbPZfF/vKwVQIpH0lW63S61Wo9Vqkc1mqdVqtz3HZrMxNDREKpXijTfeoFarEQ6HcTqdBINBnE4noVCIQCBwX+/dNwHsdrt0Oh3K5TLr6+vAzZM0Go14PB5MJhNGoxGd7uFW6aqqiveqVqsA4s5xq6KwRCLpIc1mk0qlQqvVolwu02w2SafTNBoN0un0HQXQbrfj9/vJZrPcuHGDer1OsVjEYrEwNDSE3W7nmWeeeXQEUPsQLly4wJ/92Z8BMDU1hdfr5eTJkwwPD+P1erHZbA/1Pp1Oh3q9TqlU4saNGyiKwqFDh3C73ej1eimCEkmPyWQyXLt2jXQ6zccff0w+n+fatWsUi0U2NzeFobIdzcJrNpusr6/TarXQapnqdDoMBgN/8id/wpNPPnlfY+m5AKqqiqqqFAoF1tfXWVlZYXV1FUVRcDgcqKpKu93eFWHqdrtUKhU2NjYoFousrKyg1+sZHx/HbrejKAp6vX4XzkryedEs8nK5TKvVEr4fp9OJ0+ns9/Aku0in09mxvG00GjQaDRKJBMvLy6TTaWKxGIVCgXg8TqlUIpPJ3FEAte9Lp9Mhn8/Tbrd3iKBer6dSqdz2us+i5wLYbDZpNpu8+eabfO973yObzRKPx3E4HMIqs9ls+P1+DIYHH16r1aLZbHL69Gn+3b/7d+TzeVKpFD6fD5/Ph06nY2ho6KEtTMnnR3Nyl0olfvrTnxKPx0mlUlQqFX7nd36Hr3zlK+h0uod2e0j6T7fbJZfLUalUOHv2LLFYjPn5ea5du0atVqNUKtFsNqlWq7TbbWq1Gp1Oh3b7zr3V6/U6yWQS+GTDZDeq2fdUAFVVpVarUalUiMfjXLp0iWazSavVwmKxoNfr0ev1mEwmTCbTQ71Xq9WiUqmwtbXF5cuXKRaLNJtN8WFvv3tI9hbNqm+32xSLRXK5HLFYjKWlJZLJJMVikePHj9PtdqVL4jFBVVUajQbVapX19XWuX7/Oxx9/zIULF+h2u5/7+lMUBZ1Oh6IoO6y9T6/cHnQ11zMBrNVqNJtN3nrrLT766CPOnTtHsVjE7/fzxBNPMD4+zje+8Q2Gh4cJh8MP/X6Li4vifZLJJA6Hg6985SuMjY0xMzNDMBi87y1zyYORzWZZWFhgc3OTt99+m1Qqxfz8PLlcjnq9TqfTIZlMUqvVMJlMWK3Wfg9Z8pB0u12xpD179izvvvsu+XyeVqt1X9abw+HA5XJht9vxer13vUHq9XqGh4fve5w9EUBVVWm1WtTrdRYXF3n//feJx+M0m01MJhPRaJSZmRmeffbZ+97FuRuZTIb5+XlWVlYol8t4PB7m5uaYmJjA7/djt9t35X0kn021WmV1dZVYLMYvfvELUqkUmUyGRqMB3Lx7a7uBcvn7aPFpIdMEarsFmEgkuHHjxh1fv13Qts+99ner1YrH48HtdhMOh+/6/dDpdLhcrvsef88EMJvNks1mWVlZYXFxkVKphKqqBINBXnrpJSKRyK7444rFItVqlStXrvDOO+/QarU4ePAgU1NTHD16lEgkIsWvRzSbTWq1GktLS/zoRz8imUySTCapVCo7gl1VVeXq1av8t//235ienubEiROYTCZpoQ8wmutqfX2dpaUlrFYrgUAAm81GJBIBboqS5tKyWCwYDAb0er1YAtvtdkZGRrBYLHg8HsxmMz6fD5vNhtvtxm63Y7fbcTgcWK1W3G73PV0khw4duu/z6IkAdrtd8vk8m5ubrK+vs7a2Jh7zer2cOHECv9+PxWK5x1E+H5VKhUwmw/LyMqdPn2ZkZIRnnnmGmZkZZmdnCQaDconVI5rNJuVymbW1Nf7qr/6KXC5HsVik2+0CO+/+y8vLvPXWW9RqNQ4fPgyAyWSSPsEBpdVqUavVWF1d5Ve/+hVer5eZmRmGhoYIBoMizlav12MwGDCbzcK332q1hMU2PT2Ny+VibGwMh8PB9PQ0Xq+X0dFR/H4/RqMRg8EgjrHb7KkAqqpKs9mkXq8Tj8fFljfA+Pg4MzMzPPPMMzidTsxm8wMvf1RVpVwu02g0+OCDD7h06RKXLl1CVVUcDgeTk5OMjo5itVoxGo3youoRqVSKy5cvc+PGDarVKo1G466+Hy02rNPp0Gq18Pv9zM3N4XQ6mZiYwGq1yuXxANDpdOh0Oly4cIH5+XmuX7/OxYsXiUajTExMiJubXq/H7/djNpt5+eWXCYVCWK1W7Ha7iARwuVxMTU1htVrxer2YzWYCgYBY9mpzrtfr92zu91wAq9Uq5XKZxcVFrly5QjqdBmBmZoZXX32VqakpYf4+KNqWez6f5yc/+Qk//OEPKZfLqKqK2+3m8OHD4g7zsLvLks9PPB7n7bff5vr16xQKhXsK4MbGBolEgvn5eX7+858TjUb56le/yujoKF6vV1iD8ubVX9rtNs1mk3fffZfvf//7pFIpNjY2ePbZZ/nSl74kNji0TYlgMMirr74q/PBut5tKpUIul8NkMuHz+XYInDa/vZrnPRXAZrPJ0tISqVSKGzduEIvFaDabuN1uAoEAY2Nj+P3+hw5GVlWVYrFIJpOhUChQqVSwWCy43W5GR0cZGxsjFAo9VFyh5POjBThnMhlisRibm5viotDu6polXiqVxGaIFi5Tr9fJ5/MsLi6KcCa32y2sAUnv0eZmcXGRVCrF8vIymUwGo9HI5OQk0WiUYDCIx+PZMUc6nQ6r1YqiKMIPaLFYcDgcYnmrhbn0gz1VhFKpxE9+8hOuX7/Ou+++y9raGsFgkImJCQ4dOsSJEyewWq0P/aXudDpsbGywvLxMMpmkUCgwNzfHsWPH+MIXvsDJkydF/q9k76lUKpRKJRYWFnj33XepVqu0Wi3MZjPDw8NYrVZcLhd6vZ5r166xubkpXqvFC66vr/Pmm28yNTXFK6+8gtvtFk50Se/Rculff/11Tp8+zYULF4jH4zz11FO88MILHDp0iCeffPKOqyyn04nD4RAip/kDoXeW3t3Y8yVwvV4X/p9GoyHuCBaLBbPZ/MA+Oe2OVCgUKJVKrK2tsba2JnaXh4aGmJmZIRKJiDuPpDdoed7VapVarUa328Vut+N2u5mdncXpdIpUxGazidlsplKpUC6XxQ5hp9OhVCqRz+dZXV3FZDIxPT2Nx+Ppq8Ww3+h2u7TbbbLZLPl8nvX1dTY2NkS6mtPpZHx8nFAoJMpSfXpu7uS6GJT523NV0E5e+9EcnppF9iDOTS2uMJfL8dOf/lRYC7f6fwJw7Ngx/vAP/xC73S79fj1EC3mKxWJsbW1Rq9Ww2+1MTEwwOzvLP/7H/5iRkRGR9nT69GlisRgfffQRZ8+epVQqkcvlRNZOLBbju9/9LpFIhH/wD/4BTz31FFarVc5pD9BSFwuFAm+88QZra2u8++67XL16Fb1ej9Pp5IknnuCv//W/Lqy8R+3m1FOzSEtnabfbVKtVstms8ANoBQ/v9eFpdyMtnS2dTgvLL5lMkk6n6XQ6Yok1PDwsdw77gObH06L+TSYTfr+fYDDI2NgY4XCYVqtFq9UilUoBsLKygtVqFf5A+OQCTKfT6PV66vX6ruWASj4brZBBqVRifX2d1dVVCoUC9Xodv98vQl6GhoZEKuujRs9zgdfX10XFh5WVFYxGIzabDY/Hw/Hjx3E4HHd9vVY4oVAosLy8TD6f5/z58xQKBWE1TE5OMjIywsTERA/PTLIdzbrTgp1DoRCvvPIKExMTIghdu+k98cQTTE5Okk6nuXbtGgBbW1tC5BRFETFgWjyYvKn1hlKpxMWLF1lbW+P1119nZWWFZrOJ3+/n1Vdf5dSpU0xOTuJwOB5J8YM9FsDtgZDal7ZSqVCpVIjFYiiKgtFoxG63EwgEGB4exu123/V4m5ub3Lhxg2w2y9WrVykWiywtLQl/hMFgwOVyEQ6Hcblcj5Qp/jihxfJpMWEWi4Xh4WECgYC4UDSXiBYaoYVCaUVwtddqO8aaYD5qS6xHmVarRTqdZnNzU6y0AoEATqeTyclJYbAYjcZ+D/WB2VMBtFqtnDhxgpGREeLxOPl8nnq9TqPRIJ/PiwKl2tb40tLSPX07tVqNYrEo/BJaaS1FUXC5XFitVp5//nleeOEFZmdn9/LUJHdBVVVWV1f56KOPWF1dBW6mJ165coV6vc6xY8fu+Lp8Ps/Kygr5fF6IH4DRaCQSiTA+Po7b7cZsNj+y1sajRr1eF/U6m80mer2eQCBAIBAgGAzi9XofafGDPRZALUbI4XDg8/mw2+10Oh2RJK1Zbtpy5/r16595d9++NNLQ4svcbjdTU1M888wzeDweaSn0iWw2K+LEut2uaGRjt9vvWO9NC5jPZDLU6/UdPj6dTofH48Hn82G1WuVufg9ptVpkMhnS6TStVksULfZ6vTidzseiluaefpsMBgN+vx+r1crXv/51nn32WdLpNPl8nnw+TzqdJp1Oc+PGDTqdDmazGbPZTDQavaMvcGhoiEgkQiKR4Gc/+5moAKvT6QgGg+LH6/XuSl6x5P5RFIWRkRGefPJJ6vW6sPwSiQQOh4OtrS0AkRmSSCTI5/N8/PHH1Go1VFXFYrGIDZBGo8H8/Dz5fJ7nn38en8+Hw+GQ89sDqtUq165dIx6Piw2oRCJBtVrlzJkzOJ1OvF4vwWBQFDR41IyOPRVALR/Q5/Pxta99jWazycbGBqlUilgsxrVr11hYWGB5eRlA7N4ePXr0jrW9pqenee655zhz5gy//OUvdwigz+djdHRUCKCkf4yMjHDkyBHW1tZQFEVU83U4HCJMaW1tjUKhwPnz51ldXeXy5cvU63XhDul0OqKazNWrV0kmk6ytrTE6Ooper5cC2AOq1SqLi4usrq6Kis2JRIJ0Os2ZM2dQFIWpqSkOHz6M1+sVwe2PEj1ZT2ibHYqi4PV6MRgMGI1GnE4no6OjBAIBut2uCJA+ePDgHTdDtJxQ7UPW6/U4HA6cTidHjhxhdnaWUCjUi1OS3AVFUUTtEH+Y1AAADspJREFUNm2+tGpAKysr/PjHP8bhcIj2hysrK+RyOVqtFiMjIwwPDzM7O0smk+HMmTOiT2y9XicWi4nNEnmT23s090Wz2RRuCS2MLR6PYzAYRAZWMBjkyJEjwojR6XQ7LHqj0YjFYsFisQgrsdVqUa1WRWUYo9G44/ruBT1zqGjLW5vNhqqqzMzMiAZJWriE9sHcbaevUqmQzWbFB2QwGAiFQgSDQU6dOsUXvvCFe+4iS3qD1qc1EolgtVqFBZhMJrly5cqO8ubahsfExAQzMzM899xzfOtb32J+fp5MJsPW1pZIbzx37hyFQoFAIMDU1FQ/T3Ff0Ol0RNQG3Lw+NR/upUuXuHLlirhWo9EoL730Ej6fj+npaQwGA9lsVlT2cTqdYgPl/2/vXGLbutI7/ju85JX4EimKpPjUw5ZkNzGVxB3EcBQkjqUsDCiTV4Ms2gIpUKBAEAyQQbsYoIsCRRctii6CLgdNs+piUqBJO4s0mdSOE6d1Dbt2RplkLMeyRIoiadEURYl6UacL655IjhM/RhQp6/wAw6RJ33suD/m/5/F9/89iYWGBTCaDw+Ggr68Pj8dDIBB4MAXQ4nZpMXe7k2TFDuZyOWq1GqZp0tXVRTweVwWO9CJ547Hi9jo7Ozl06JBa511bW9sS6GzNCDweD/39/Tz88MMMDAzQ3t6ugmzX1tbUInypVCKfzzM3N0e1WlXhMZr60NLSQiwWo1arUSwWWVlZUa9Ztlhwsx/L5TKTk5Mq0sNutzM3N8fa2hq5XE7F+m4eoCwtLXH9+nXsdjulUgm3200kEsHj8RCJRAgEAnW/xl317bl27RrvvfceExMTLC8v097ezokTJ+jv76evr0/H/jUJVrL7kSNHcLlcnDt3jrfffpv5+XlqtZoa/dntdlKpFP39/Rw/fpxjx47hdDrxeDxUq1UOHz5MMBhkamqKxcVFtbM8NDREPp+nra1NT4XrSCAQYHh4mMnJST788EOVtXMrUkry+TwnT57c4thjzfCsUeKtvn7WdNput9Pe3o7T6WRgYIBgMMirr77K008/Xfdr3BUCuLKywvLyMsVikenpabWQbrfb8Xq9+Hw+7R7cZAgh8Hq9xGIxkskkiUSCUqlEsVikVqshhFAjjO7ubnXHtwLnXS4X0WiU1dVVfD4fy8vLW1Ioc7kcNptNC2AdaWlpIZlMYrPZ6Onpwel0qhEefFvj2cr8+b6SlnfCMAzlFuRyuVhaWiKXyzE7O4vT6axruM2uEMCpqSm++eYbPv30U06dOsXy8rLyF/N4PHg8Hj0VakKskBVrmlsoFDh37hzValVteI2OjjI4OEggENhyEwuFQoyOjpJOp8lkMly9epXLly+Ty+U4c+YMlUqFp556ikQioW98dSIcDvPiiy9SqVQ4cuQIhUKBd999l7GxMZWTvdnF535ztK21xsXFRcbGxmhpaSEYDDI3N8fg4CCPP/543fp4V6jG4uIihUKBYrHIjRs3kFLidru3WGrp/NDmw+Fw4HA4CAaDqvbD9evXWVhYwOVy0draSjweJxqNfmcEb5omoVCI1dVVwuEw5XKZ8fFxVlZWVE64VV9kt4Ve7BZM0yQcDuPz+ajVairUzDIdWV9fVwYm1nR3fX2dlZUVZVxyt1jvtWqN5PN5MpkMPT09dbq6m+wKAZyZmeHSpUtMTk6yvr6uwl56e3vZv3+/qhmhaU68Xi8HDx6kt7eX/v5+arWaWg+yfORuvYFZOcBWXKjP51PmqTMzM6yurvLoo49SqVRoaWnRcYF1wmaz0dLSQjweJxQK8frrrzM3N6cEb2ZmhmvXrqmpcLFY5LPPPmN2dpZcLqeyve4FKaVyE7+fSm/3wq4QwEqlQi6XY35+HkDtMEYiEZWWo2leTNNUO3rRaPSu/5/NZsM0TTo7O1UZRZvNpuq9WHnh1o9UT4Xrg81mU5lZt+7MptNp5e9oObNfvnyZWq3G7OzslvcKIdSGyK2jxs3TZ8uGy7LeklLu3SmwlJJCoaDiwux2O6FQiKGhIbq6un7QPkuz+2ltbeXgwYMEg0Eee+wx4KYrUKlU4vLly5w+fZpkMsnhw4f1OnAD8Pv9W2J6/X4/qVQKv99PNpulXC4r4bPq84RCIfr6+pibm2NsbIxyucyVK1fua7T4u7IrvjFzc3PKi8wwDPx+P4cOHSIWi+mpzwOOw+EgmUzi9/vp6+ujXC5TLpdJp9NMT0/zxRdfIKXkkUce0QLYAKxNSAvTNOnt7VXu73BzBGktdzz00EMMDAzw5JNPMj09jZSSbDarQp12mqb9xkgpmZqaolgsbvlwLAPVUChER0fHrrfj0dwZK5XywIEDGIZBNpvl6tWrTE9P8/nnnyOl5IknnlC1RvSGWOMwTVMFT1sCaGX7ZLNZFTS9vr6u6kZvDq2xsCI86r200bQCWKvVuHLlijJLKJfLtLa20tbWhtvtJhqNEg6HG91MzQ5hmqYa9V+4cIGzZ8+SyWSYmJjAMAxeeOEF9aPTAtg4LDcnu92uBNBKd52amiKdTpNOp5mZmWF+fp6xsbHv1Iu22WyqrG29Z3hN+02RUioHaGsx1eVyEYlEVDFl+NZ+fWlpiWq1et/BmJrmxgqsDgQC9PT0kEqlCIfDSCkplUqMjY0xPj6uqtBpGoNVLMnv9xONRkkkEqoMAqBiBwuFArOzs1sygwzDoKOjg1gsxsDAAIODg/e0aXY/NPUI8KuvvuLkyZNq6BwIBEilUirZWkqp3EIWFxdZW1vD7/frtaAHEJvNRigUor29naGhIVwuF6dPn1aB0u+//z4HDhygv78f0zR1ZlCDcDgchMNhDMNgcHAQwzC4cOGCMlSAm2v61k6+lU9sGAYOh4P9+/cTiUR49tlnGR4e3uIeUw+aWinW1tZYWVlRH5LNZsNut7O+vs6NGzdYXFwkl8tteY9hGFvuOJoHByuf1DLG7ejoUN6BuVyOQCDAwsICXq9XF1FvEFYftba2kkgkqFarZLNZZYywOUjaSoe02+34fD7cbjcDAwMkEgk6Ozt3xNykqQXwVqzt9HK5zNmzZ6lWq3z88ceUSiU6OzvxeDy89NJLtzVT1TwY2Gw2+vr6CIfDZLNZzpw5Q7Va5fz58ywvL5NOpzEMA9M0dXB8AxBCYJomPp+PEydOcPToUUzTxOVykc/nyefzLC0tUalUME2Tjo4O2traOHLkCJFIhOeee05ZY+1EbOeuEsDV1VUWFha4ceMGk5OTLC0tUSgUqFQq+P1+HRO4R3A6nQghCIVCJJNJ8vk8169fp1KpUCwWaWtr08a4DcYKVzNNk3g8Tnd3N4ZhqOUqm82G0+kkHo/T3t6uDDFisdiObm7uKgHMZrOcOnVKuYU4nU727dtHJBJhZGSE3t5eent7G91MTZ1pbW3FNE2OHTtGPB7nk08+4a233qJYLPLRRx/R09OjTFk1jcHKHrHqAQ0PD/Pll1+qhIZ0Ok0sFmNkZERtmDidTjo6Ona0nU0tgJaNtmV8WavVKJVKKpbI5/ORSqUIBoMkEgm6urr0l34PYAXWhsNhWlpamJycxOFwUKvVyGQyGIahdoN1SEzjMAwDwzCIx+PE43FVL9rv92Oz2eju7iaVSuHz+RpWT6RpBdBut3P06FF8Ph/nz5/n/PnztLW1qQLbqVQKr9er0uG6urpUXQHN3sDpdGIYBrFYjIMHD1KpVBgfH1fuxJZhql4LbA4SiQRer5elpSUWFhZwu92EQqGGujk1rQBuNmGcn58nk8nQ2dlJf38/yWSSkZERPB4Pbrcbu92O3W7XYQ97DKvffT4f0WiUbDbL119/zerqKsVikXK5rKZhmsbj9/vx+/2NbsYWmlYAhRCqpvAzzzxDX18fbrdbld+zKo5Zdw8tfnuXrq4uXnnlFcbHxymVShiGoTZEdnpNSbO7aGoBDAQCBAIBkslko5ujaWLi8Tijo6NcvHiRDz74QFUyK5VKrK6uNrp5miamaQVQo7lbLN/AaDTK888/z/LyskqVq2c9Cc3uRwugZtdjGaLu27ePN998EyklhmHctgSrRrMZLYCaBwYhhM4D19wTOkhKo9HsWbQAajSaPYu4l1qeQogCcK1+zWk6uqWUoUY3YifRffzgo/v4W+5JADUajeZBQk+BNRrNnkULoEaj2bNoAdRoNHuWugugEKJDCPF/G39mhBCZTc/NOp73ZSGEFEL8qF7n0Nxkp/tYCNEthPiVEOKSEOKkECKx3efQbKUBffyaEKKw6Rx/ut3ngB3eBBFC/BVQkVL+/aZ/s0spt7WUmxDCC/wSMIE3pJTntvP4mu9nJ/pYCPEL4D+klO8IIY4DfyKl/OPtOr7mh9mhPn4N+JGU8o3tOubtaEjYvBDin4El4DHgMyFEmU0fqBDi18ColHJCCPFHwE+4KWb/A7wupazd4RR/Dfwt8Bd1ugTNHahzHz8E/HTj8X8B/1afq9D8EDvwO647jVwDTABPSCl/+n1vEEL8HvAqMCSlfBSoAX+48drPbze9FUIcBpJSyl/Wp9mae6AufQxcBF7aePwi4BVCaN+rxlCvPgZ4eWOZ410hRF0soRqZOPmLu7gDDAO/D/zvRlK7E8gDSCm/syYghLAB/wC8tq0t1dwv297HG/w58I8b06RPgAw3f1SanadeffzvwL9IKZeFEH8GvAMc354mf0sjBXBh0+M1to5GWzf+FsA7Usqf3eUxvcAh4OTGBx0B3hdC/FivAzaEevQxUsppNkaAQggP8LKUsvQ7tlVzf9Srj2c3Pf058Hf33cIfoFnCYCaAw6CmsFZpt18BfyCECG+8FhBCdH/fQaSUc1LKoJSyR0rZA/w3oMWvOZhgG/p44z3BjdE+wM+Af6pLizX3ygTb18fRTU9/DPxm21tL8wjgvwIBIcQY8AbwWwAp5ZfAXwL/KYS4BHwIROGOawea5mM7+/gY8LUQ4rdAJ/A39W++5i7Yzj7+iRBiTAhxkZubJ6/Vo8E6F1ij0exZmmUEqNFoNDuOFkCNRrNn0QKo0Wj2LFoANRrNnkULoEaj2bNoAdRoNHsWLYAajWbP8v+wh+/4o3CBiwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.x_test[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.y_test_cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function to plot example errors\n",
    "\n",
    "Function for plotting examples of images from the test-set that have been mis-classified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_example_errors(cls_pred):\n",
    "    # cls_pred is an array of the predicted class-number for\n",
    "    # all images in the test-set.\n",
    "\n",
    "    # Boolean array whether the predicted class is incorrect.\n",
    "    incorrect = (cls_pred != data.y_test_cls)\n",
    "\n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    images = data.x_test[incorrect]\n",
    "    \n",
    "    # Get the predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "\n",
    "    # Get the true classes for those images.\n",
    "    cls_true = data.y_test_cls[incorrect]\n",
    "    \n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyper-Parameter Optimization\n",
    "\n",
    "There are several steps required to do hyper-parameter optimization.\n",
    "\n",
    "### Create the Model\n",
    "\n",
    "We first need a function that takes a set of hyper-parameters and creates the Convolutional Neural Network corresponding to those parameters. We use Keras to build the neural network in TensorFlow, see Tutorial #03-C for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def create_model(learning_rate, num_dense_layers,\n",
    "                 num_dense_nodes, activation):\n",
    "    \"\"\"\n",
    "    Hyper-parameters:\n",
    "    learning_rate:     Learning-rate for the optimizer.\n",
    "    num_dense_layers:  Number of dense layers.\n",
    "    num_dense_nodes:   Number of nodes in each dense layer.\n",
    "    activation:        Activation function for all layers.\n",
    "    \"\"\"\n",
    "    \n",
    "    # Start construction of a Keras Sequential model.\n",
    "    model = Sequential()\n",
    "\n",
    "    # Add an input layer which is similar to a feed_dict in TensorFlow.\n",
    "    # Note that the input-shape must be a tuple containing the image-size.\n",
    "    model.add(InputLayer(input_shape=(img_size_flat,)))\n",
    "\n",
    "    # The input from MNIST is a flattened array with 784 elements,\n",
    "    # but the convolutional layers expect images with shape (28, 28, 1)\n",
    "    model.add(Reshape(img_shape_full))\n",
    "\n",
    "    # First convolutional layer.\n",
    "    # There are many hyper-parameters in this layer, but we only\n",
    "    # want to optimize the activation-function in this example.\n",
    "    model.add(Conv2D(kernel_size=5, strides=1, filters=16, padding='same',\n",
    "                     activation=activation, name='layer_conv1'))\n",
    "    model.add(MaxPooling2D(pool_size=2, strides=2))\n",
    "\n",
    "    # Second convolutional layer.\n",
    "    # Again, we only want to optimize the activation-function here.\n",
    "    model.add(Conv2D(kernel_size=5, strides=1, filters=36, padding='same',\n",
    "                     activation=activation, name='layer_conv2'))\n",
    "    model.add(MaxPooling2D(pool_size=2, strides=2))\n",
    "\n",
    "    # Flatten the 4-rank output of the convolutional layers\n",
    "    # to 2-rank that can be input to a fully-connected / dense layer.\n",
    "    model.add(Flatten())\n",
    "\n",
    "    # Add fully-connected / dense layers.\n",
    "    # The number of layers is a hyper-parameter we want to optimize.\n",
    "    for i in range(num_dense_layers):\n",
    "        # Name of the layer. This is not really necessary\n",
    "        # because Keras should give them unique names.\n",
    "        name = 'layer_dense_{0}'.format(i+1)\n",
    "\n",
    "        # Add the dense / fully-connected layer to the model.\n",
    "        # This has two hyper-parameters we want to optimize:\n",
    "        # The number of nodes and the activation function.\n",
    "        model.add(Dense(num_dense_nodes,\n",
    "                        activation=activation,\n",
    "                        name=name))\n",
    "\n",
    "    # Last fully-connected / dense layer with softmax-activation\n",
    "    # for use in classification.\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "    \n",
    "    # Use the Adam method for training the network.\n",
    "    # We want to find the best learning-rate for the Adam method.\n",
    "    optimizer = Adam(lr=learning_rate)\n",
    "    \n",
    "    # In Keras we need to compile the model so it can be trained.\n",
    "    model.compile(optimizer=optimizer,\n",
    "                  loss='categorical_crossentropy',\n",
    "                  metrics=['accuracy'])\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train and Evaluate the Model\n",
    "\n",
    "The neural network with the best hyper-parameters is saved to disk so it can be reloaded later. This is the filename for the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "path_best_model = '19_best_model.h5'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the classification accuracy for the model saved to disk. It is a global variable which will be updated during optimization of the hyper-parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "best_accuracy = 0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the function that creates and trains a neural network with the given hyper-parameters, and then evaluates its performance on the validation-set. The function then returns the so-called fitness value (aka. objective value), which is the negative classification accuracy on the validation-set. It is negative because `skopt` performs minimization instead of maximization.\n",
    "\n",
    "Note the function decorator `@use_named_args` which wraps the fitness function so that it can be called with all the parameters as a single list, for example: `fitness(x=[1e-4, 3, 256, 'relu'])`. This is the calling-style `skopt` uses internally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "@use_named_args(dimensions=dimensions)\n",
    "def fitness(learning_rate, num_dense_layers,\n",
    "            num_dense_nodes, activation):\n",
    "    \"\"\"\n",
    "    Hyper-parameters:\n",
    "    learning_rate:     Learning-rate for the optimizer.\n",
    "    num_dense_layers:  Number of dense layers.\n",
    "    num_dense_nodes:   Number of nodes in each dense layer.\n",
    "    activation:        Activation function for all layers.\n",
    "    \"\"\"\n",
    "\n",
    "    # Print the hyper-parameters.\n",
    "    print('learning rate: {0:.1e}'.format(learning_rate))\n",
    "    print('num_dense_layers:', num_dense_layers)\n",
    "    print('num_dense_nodes:', num_dense_nodes)\n",
    "    print('activation:', activation)\n",
    "    print()\n",
    "    \n",
    "    # Create the neural network with these hyper-parameters.\n",
    "    model = create_model(learning_rate=learning_rate,\n",
    "                         num_dense_layers=num_dense_layers,\n",
    "                         num_dense_nodes=num_dense_nodes,\n",
    "                         activation=activation)\n",
    "\n",
    "    # Dir-name for the TensorBoard log-files.\n",
    "    log_dir = log_dir_name(learning_rate, num_dense_layers,\n",
    "                           num_dense_nodes, activation)\n",
    "    \n",
    "    # Create a callback-function for Keras which will be\n",
    "    # run after each epoch has ended during training.\n",
    "    # This saves the log-files for TensorBoard.\n",
    "    # Note that there are complications when histogram_freq=1.\n",
    "    # It might give strange errors and it also does not properly\n",
    "    # support Keras data-generators for the validation-set.\n",
    "    callback_log = TensorBoard(\n",
    "        log_dir=log_dir,\n",
    "        histogram_freq=0,\n",
    "        write_graph=True,\n",
    "        write_grads=False,\n",
    "        write_images=False)\n",
    "   \n",
    "    # Use Keras to train the model.\n",
    "    history = model.fit(x=data.x_train,\n",
    "                        y=data.y_train,\n",
    "                        epochs=3,\n",
    "                        batch_size=128,\n",
    "                        validation_data=validation_data,\n",
    "                        callbacks=[callback_log])\n",
    "\n",
    "    # Get the classification accuracy on the validation-set\n",
    "    # after the last training-epoch.\n",
    "    accuracy = history.history['val_accuracy'][-1]\n",
    "\n",
    "    # Print the classification accuracy.\n",
    "    print()\n",
    "    print(\"Accuracy: {0:.2%}\".format(accuracy))\n",
    "    print()\n",
    "\n",
    "    # Save the model if it improves on the best-found performance.\n",
    "    # We use the global keyword so we update the variable outside\n",
    "    # of this function.\n",
    "    global best_accuracy\n",
    "\n",
    "    # If the classification accuracy of the saved model is improved ...\n",
    "    if accuracy > best_accuracy:\n",
    "        # Save the new model to harddisk.\n",
    "        model.save(path_best_model)\n",
    "        \n",
    "        # Update the classification accuracy.\n",
    "        best_accuracy = accuracy\n",
    "\n",
    "    # Delete the Keras model with these hyper-parameters from memory.\n",
    "    del model\n",
    "    \n",
    "    # Clear the Keras session, otherwise it will keep adding new\n",
    "    # models to the same TensorFlow graph each time we create\n",
    "    # a model with a different set of hyper-parameters.\n",
    "    K.clear_session()\n",
    "    \n",
    "    # NOTE: Scikit-optimize does minimization so it tries to\n",
    "    # find a set of hyper-parameters with the LOWEST fitness-value.\n",
    "    # Because we are interested in the HIGHEST classification\n",
    "    # accuracy, we need to negate this number so it can be minimized.\n",
    "    return -accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test Run\n",
    "\n",
    "Before we run the hyper-parameter optimization, let us first check that the various functions above actually work, when we pass the default hyper-parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "learning rate: 1.0e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 16\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 64us/sample - loss: 2.2207 - accuracy: 0.2039 - val_loss: 2.0769 - val_accuracy: 0.3326\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 39us/sample - loss: 1.8787 - accuracy: 0.4489 - val_loss: 1.5766 - val_accuracy: 0.6934\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 2s 39us/sample - loss: 1.3661 - accuracy: 0.7220 - val_loss: 1.0646 - val_accuracy: 0.8080\n",
      "\n",
      "Accuracy: 80.80%\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "-0.808"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitness(x=default_parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the Hyper-Parameter Optimization\n",
    "\n",
    "Now we are ready to run the actual hyper-parameter optimization using Bayesian optimization from the scikit-optimize package. Note that it first calls `fitness()` with `default_parameters` as the starting point we have found by hand-tuning, which should help the optimizer locate better hyper-parameters faster.\n",
    "\n",
    "There are many more parameters you can experiment with here, including the number of calls to the `fitness()` function which we have set to 40. But `fitness()` is very expensive to evaluate so it should not be run too many times, especially for larger neural networks and datasets.\n",
    "\n",
    "You can also experiment with the so-called acquisition function which determines how to find a new set of hyper-parameters from the internal model of the Bayesian optimizer. You can also try using another Bayesian optimizer such as Random Forests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "learning rate: 1.0e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 16\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 47us/sample - loss: 2.2126 - accuracy: 0.3096 - val_loss: 2.0888 - val_accuracy: 0.4494\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 38us/sample - loss: 1.9393 - accuracy: 0.4852 - val_loss: 1.7300 - val_accuracy: 0.5342\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 2s 38us/sample - loss: 1.5558 - accuracy: 0.5634 - val_loss: 1.2548 - val_accuracy: 0.7250\n",
      "\n",
      "Accuracy: 72.50%\n",
      "\n",
      "learning rate: 7.0e-04\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 365\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 46us/sample - loss: 0.2217 - accuracy: 0.9350 - val_loss: 0.0633 - val_accuracy: 0.9828\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 39us/sample - loss: 0.0576 - accuracy: 0.9822 - val_loss: 0.0507 - val_accuracy: 0.9870\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 2s 39us/sample - loss: 0.0397 - accuracy: 0.9878 - val_loss: 0.0396 - val_accuracy: 0.9892\n",
      "\n",
      "Accuracy: 98.92%\n",
      "\n",
      "learning rate: 6.8e-03\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 466\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 53us/sample - loss: 2.3098 - accuracy: 0.1117 - val_loss: 2.3022 - val_accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 44us/sample - loss: 2.3015 - accuracy: 0.1123 - val_loss: 2.3016 - val_accuracy: 0.1060\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 47us/sample - loss: 2.3017 - accuracy: 0.1123 - val_loss: 2.3019 - val_accuracy: 0.1060\n",
      "\n",
      "Accuracy: 10.60%\n",
      "\n",
      "learning rate: 9.8e-04\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 122\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 54us/sample - loss: 2.3093 - accuracy: 0.1038 - val_loss: 2.3061 - val_accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 42us/sample - loss: 2.3057 - accuracy: 0.1078 - val_loss: 2.3039 - val_accuracy: 0.1060\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 2s 41us/sample - loss: 2.3053 - accuracy: 0.1073 - val_loss: 2.3054 - val_accuracy: 0.0978\n",
      "\n",
      "Accuracy: 9.78%\n",
      "\n",
      "learning rate: 3.7e-04\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 72\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 48us/sample - loss: 2.3049 - accuracy: 0.1066 - val_loss: 2.3042 - val_accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 40us/sample - loss: 2.3008 - accuracy: 0.1220 - val_loss: 2.2349 - val_accuracy: 0.2808\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 55us/sample - loss: 1.1898 - accuracy: 0.7546 - val_loss: 0.5112 - val_accuracy: 0.9176\n",
      "\n",
      "Accuracy: 91.76%\n",
      "\n",
      "learning rate: 6.7e-04\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 230\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 57us/sample - loss: 1.9215 - accuracy: 0.2895 - val_loss: 0.5522 - val_accuracy: 0.8572\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 5s 90us/sample - loss: 0.3142 - accuracy: 0.9098 - val_loss: 0.1402 - val_accuracy: 0.9612\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 54us/sample - loss: 0.1451 - accuracy: 0.9567 - val_loss: 0.0997 - val_accuracy: 0.9708\n",
      "\n",
      "Accuracy: 97.08%\n",
      "\n",
      "learning rate: 9.7e-03\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 132\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 53us/sample - loss: 0.2108 - accuracy: 0.9304 - val_loss: 0.0595 - val_accuracy: 0.9840\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 2s 45us/sample - loss: 0.0719 - accuracy: 0.9801 - val_loss: 0.0487 - val_accuracy: 0.9874\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 50us/sample - loss: 0.0673 - accuracy: 0.9824 - val_loss: 0.0474 - val_accuracy: 0.9870\n",
      "\n",
      "Accuracy: 98.70%\n",
      "\n",
      "learning rate: 1.6e-04\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 112\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 8s 149us/sample - loss: 2.3076 - accuracy: 0.1080 - val_loss: 2.3030 - val_accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 6s 118us/sample - loss: 2.2995 - accuracy: 0.1135 - val_loss: 2.2653 - val_accuracy: 0.1128\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 115us/sample - loss: 1.7192 - accuracy: 0.4761 - val_loss: 1.1804 - val_accuracy: 0.6904\n",
      "\n",
      "Accuracy: 69.04%\n",
      "\n",
      "learning rate: 7.7e-05\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 154\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 56us/sample - loss: 0.8023 - accuracy: 0.7727 - val_loss: 0.2123 - val_accuracy: 0.9404\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 49us/sample - loss: 0.2085 - accuracy: 0.9384 - val_loss: 0.1236 - val_accuracy: 0.9644\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 125us/sample - loss: 0.1413 - accuracy: 0.9564 - val_loss: 0.0950 - val_accuracy: 0.9746\n",
      "\n",
      "Accuracy: 97.46%\n",
      "\n",
      "learning rate: 2.9e-03\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 156\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 75us/sample - loss: 2.3168 - accuracy: 0.1028 - val_loss: 2.3084 - val_accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 51us/sample - loss: 2.3055 - accuracy: 0.1059 - val_loss: 2.3023 - val_accuracy: 0.1060\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 101us/sample - loss: 2.3016 - accuracy: 0.1119 - val_loss: 2.3027 - val_accuracy: 0.1060\n",
      "\n",
      "Accuracy: 10.60%\n",
      "\n",
      "learning rate: 1.1e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 496\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 10s 185us/sample - loss: 1.8064 - accuracy: 0.6548 - val_loss: 1.0878 - val_accuracy: 0.8440\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 7s 124us/sample - loss: 0.7844 - accuracy: 0.8393 - val_loss: 0.4786 - val_accuracy: 0.9046\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 128us/sample - loss: 0.4776 - accuracy: 0.8791 - val_loss: 0.3288 - val_accuracy: 0.9226\n",
      "\n",
      "Accuracy: 92.26%\n",
      "\n",
      "learning rate: 3.4e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 451\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 62us/sample - loss: 0.1423 - accuracy: 0.9556 - val_loss: 0.0378 - val_accuracy: 0.9904\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 52us/sample - loss: 0.0438 - accuracy: 0.9869 - val_loss: 0.0379 - val_accuracy: 0.9896\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 106us/sample - loss: 0.0342 - accuracy: 0.9892 - val_loss: 0.0370 - val_accuracy: 0.9890\n",
      "\n",
      "Accuracy: 98.90%\n",
      "\n",
      "learning rate: 1.2e-05\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 182\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 59us/sample - loss: 1.9464 - accuracy: 0.4581 - val_loss: 1.0907 - val_accuracy: 0.8004\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 48us/sample - loss: 0.6953 - accuracy: 0.8265 - val_loss: 0.3989 - val_accuracy: 0.8992\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s 78us/sample - loss: 0.4034 - accuracy: 0.8862 - val_loss: 0.2750 - val_accuracy: 0.9256\n",
      "\n",
      "Accuracy: 92.56%\n",
      "\n",
      "learning rate: 2.8e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 56us/sample - loss: 1.8869 - accuracy: 0.3200 - val_loss: 0.5330 - val_accuracy: 0.8646\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 52us/sample - loss: 0.3947 - accuracy: 0.8847 - val_loss: 0.2235 - val_accuracy: 0.9346\n",
      "Epoch 3/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "55000/55000 [==============================] - 4s 78us/sample - loss: 0.2316 - accuracy: 0.9292 - val_loss: 0.1444 - val_accuracy: 0.9592\n",
      "\n",
      "Accuracy: 95.92%\n",
      "\n",
      "learning rate: 3.4e-03\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 5\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 64us/sample - loss: 2.3031 - accuracy: 0.1100 - val_loss: 2.3020 - val_accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 46us/sample - loss: 2.3020 - accuracy: 0.1118 - val_loss: 2.3021 - val_accuracy: 0.1060\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s 70us/sample - loss: 2.3019 - accuracy: 0.1128 - val_loss: 2.3018 - val_accuracy: 0.1060\n",
      "\n",
      "Accuracy: 10.60%\n",
      "\n",
      "learning rate: 1.0e-02\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 104\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 52us/sample - loss: 0.1708 - accuracy: 0.9451 - val_loss: 0.0552 - val_accuracy: 0.9836\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 46us/sample - loss: 0.0630 - accuracy: 0.9813 - val_loss: 0.0551 - val_accuracy: 0.9836\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 46us/sample - loss: 0.0514 - accuracy: 0.9851 - val_loss: 0.0415 - val_accuracy: 0.9904\n",
      "\n",
      "Accuracy: 99.04%\n",
      "\n",
      "learning rate: 5.5e-04\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 277\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 78us/sample - loss: 2.0127 - accuracy: 0.2534 - val_loss: 0.6090 - val_accuracy: 0.8268\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 50us/sample - loss: 0.3406 - accuracy: 0.9004 - val_loss: 0.1451 - val_accuracy: 0.9582\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s 80us/sample - loss: 0.1563 - accuracy: 0.9533 - val_loss: 0.0969 - val_accuracy: 0.9700\n",
      "\n",
      "Accuracy: 97.00%\n",
      "\n",
      "learning rate: 2.8e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 441\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 58us/sample - loss: 0.1397 - accuracy: 0.9561 - val_loss: 0.0428 - val_accuracy: 0.9864\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 49us/sample - loss: 0.0471 - accuracy: 0.9853 - val_loss: 0.0401 - val_accuracy: 0.9892\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 120us/sample - loss: 0.0316 - accuracy: 0.9905 - val_loss: 0.0482 - val_accuracy: 0.9882\n",
      "\n",
      "Accuracy: 98.82%\n",
      "\n",
      "learning rate: 5.0e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 309\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 56us/sample - loss: 0.2504 - accuracy: 0.9302 - val_loss: 0.0794 - val_accuracy: 0.9784\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 49us/sample - loss: 0.0642 - accuracy: 0.9801 - val_loss: 0.0631 - val_accuracy: 0.9814\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s 83us/sample - loss: 0.0439 - accuracy: 0.9861 - val_loss: 0.0465 - val_accuracy: 0.9878\n",
      "\n",
      "Accuracy: 98.78%\n",
      "\n",
      "learning rate: 1.1e-04\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 64us/sample - loss: 0.5362 - accuracy: 0.8733 - val_loss: 0.1483 - val_accuracy: 0.9572\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 50us/sample - loss: 0.1423 - accuracy: 0.9581 - val_loss: 0.0895 - val_accuracy: 0.9730\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s 81us/sample - loss: 0.0942 - accuracy: 0.9724 - val_loss: 0.0687 - val_accuracy: 0.9806\n",
      "\n",
      "Accuracy: 98.06%\n",
      "\n",
      "learning rate: 5.5e-05\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 58us/sample - loss: 0.7263 - accuracy: 0.8272 - val_loss: 0.1854 - val_accuracy: 0.9470\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 51us/sample - loss: 0.1967 - accuracy: 0.9415 - val_loss: 0.1209 - val_accuracy: 0.9666\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 115us/sample - loss: 0.1333 - accuracy: 0.9604 - val_loss: 0.0909 - val_accuracy: 0.9748\n",
      "\n",
      "Accuracy: 97.48%\n",
      "\n",
      "learning rate: 3.5e-05\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 70us/sample - loss: 0.7778 - accuracy: 0.8057 - val_loss: 0.1853 - val_accuracy: 0.9496\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 54us/sample - loss: 0.1914 - accuracy: 0.9435 - val_loss: 0.1170 - val_accuracy: 0.9678\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 107us/sample - loss: 0.1281 - accuracy: 0.9622 - val_loss: 0.0848 - val_accuracy: 0.9778\n",
      "\n",
      "Accuracy: 97.78%\n",
      "\n",
      "learning rate: 3.6e-05\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 446\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 63us/sample - loss: 0.7959 - accuracy: 0.8041 - val_loss: 0.1977 - val_accuracy: 0.9454\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 53us/sample - loss: 0.1957 - accuracy: 0.9419 - val_loss: 0.1231 - val_accuracy: 0.9658\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 110us/sample - loss: 0.1334 - accuracy: 0.9598 - val_loss: 0.0973 - val_accuracy: 0.9734\n",
      "\n",
      "Accuracy: 97.34%\n",
      "\n",
      "learning rate: 6.6e-05\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 143\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 67us/sample - loss: 0.8583 - accuracy: 0.7593 - val_loss: 0.2149 - val_accuracy: 0.9410\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 50us/sample - loss: 0.2304 - accuracy: 0.9302 - val_loss: 0.1499 - val_accuracy: 0.9538\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s 86us/sample - loss: 0.1615 - accuracy: 0.9511 - val_loss: 0.1022 - val_accuracy: 0.9700\n",
      "\n",
      "Accuracy: 97.00%\n",
      "\n",
      "learning rate: 2.2e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 195\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 53us/sample - loss: 2.2477 - accuracy: 0.1821 - val_loss: 1.6547 - val_accuracy: 0.6054\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 49us/sample - loss: 0.8448 - accuracy: 0.7859 - val_loss: 0.4386 - val_accuracy: 0.8970\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 61us/sample - loss: 0.4028 - accuracy: 0.8893 - val_loss: 0.2623 - val_accuracy: 0.9310\n",
      "\n",
      "Accuracy: 93.10%\n",
      "\n",
      "learning rate: 4.2e-05\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 61us/sample - loss: 2.3073 - accuracy: 0.1076 - val_loss: 2.2982 - val_accuracy: 0.1126\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 49us/sample - loss: 2.2914 - accuracy: 0.1344 - val_loss: 2.2685 - val_accuracy: 0.1802\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s 79us/sample - loss: 2.1696 - accuracy: 0.3317 - val_loss: 1.9508 - val_accuracy: 0.5936\n",
      "\n",
      "Accuracy: 59.36%\n",
      "\n",
      "learning rate: 5.1e-06\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 59us/sample - loss: 2.0897 - accuracy: 0.5289 - val_loss: 1.6486 - val_accuracy: 0.7448\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 57us/sample - loss: 1.1361 - accuracy: 0.7831 - val_loss: 0.6805 - val_accuracy: 0.8716\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 118us/sample - loss: 0.5963 - accuracy: 0.8549 - val_loss: 0.4089 - val_accuracy: 0.9120\n",
      "\n",
      "Accuracy: 91.20%\n",
      "\n",
      "learning rate: 1.4e-06\n",
      "num_dense_layers: 4\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 70us/sample - loss: 2.3907 - accuracy: 0.0994 - val_loss: 2.3302 - val_accuracy: 0.0986\n",
      "Epoch 2/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "55000/55000 [==============================] - 3s 52us/sample - loss: 2.3122 - accuracy: 0.0994 - val_loss: 2.3043 - val_accuracy: 0.0986\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 121us/sample - loss: 2.3020 - accuracy: 0.1115 - val_loss: 2.3020 - val_accuracy: 0.1060\n",
      "\n",
      "Accuracy: 10.60%\n",
      "\n",
      "learning rate: 4.8e-04\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 225\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 59us/sample - loss: 1.5780 - accuracy: 0.4531 - val_loss: 0.3782 - val_accuracy: 0.9030\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 52us/sample - loss: 0.2985 - accuracy: 0.9146 - val_loss: 0.1567 - val_accuracy: 0.9564\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 118us/sample - loss: 0.1712 - accuracy: 0.9491 - val_loss: 0.1077 - val_accuracy: 0.9704\n",
      "\n",
      "Accuracy: 97.04%\n",
      "\n",
      "learning rate: 5.5e-03\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 69us/sample - loss: 0.1434 - accuracy: 0.9557 - val_loss: 0.0516 - val_accuracy: 0.9834\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 49us/sample - loss: 0.0457 - accuracy: 0.9858 - val_loss: 0.0441 - val_accuracy: 0.9874\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 110us/sample - loss: 0.0330 - accuracy: 0.9894 - val_loss: 0.0582 - val_accuracy: 0.9854\n",
      "\n",
      "Accuracy: 98.54%\n",
      "\n",
      "learning rate: 2.4e-03\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 5\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 63us/sample - loss: 1.1980 - accuracy: 0.5365 - val_loss: 0.3991 - val_accuracy: 0.8454\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 50us/sample - loss: 0.3059 - accuracy: 0.9074 - val_loss: 0.1823 - val_accuracy: 0.9488\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 4s 77us/sample - loss: 0.1815 - accuracy: 0.9497 - val_loss: 0.1492 - val_accuracy: 0.9606\n",
      "\n",
      "Accuracy: 96.06%\n",
      "\n",
      "learning rate: 6.6e-04\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 61us/sample - loss: 0.2145 - accuracy: 0.9392 - val_loss: 0.0638 - val_accuracy: 0.9830\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 51us/sample - loss: 0.0591 - accuracy: 0.9817 - val_loss: 0.0539 - val_accuracy: 0.9856\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 3s 62us/sample - loss: 0.0395 - accuracy: 0.9879 - val_loss: 0.0454 - val_accuracy: 0.9890\n",
      "\n",
      "Accuracy: 98.90%\n",
      "\n",
      "learning rate: 1.0e-06\n",
      "num_dense_layers: 5\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 64us/sample - loss: 2.2933 - accuracy: 0.2440 - val_loss: 2.2784 - val_accuracy: 0.4338\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 4s 68us/sample - loss: 2.2577 - accuracy: 0.5072 - val_loss: 2.2230 - val_accuracy: 0.5994\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 8s 154us/sample - loss: 2.1768 - accuracy: 0.6255 - val_loss: 2.0998 - val_accuracy: 0.7048\n",
      "\n",
      "Accuracy: 70.48%\n",
      "\n",
      "learning rate: 2.9e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 299\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 4s 69us/sample - loss: 0.1429 - accuracy: 0.9546 - val_loss: 0.0407 - val_accuracy: 0.9882\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 59us/sample - loss: 0.0452 - accuracy: 0.9855 - val_loss: 0.0391 - val_accuracy: 0.9882\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 132us/sample - loss: 0.0310 - accuracy: 0.9903 - val_loss: 0.0370 - val_accuracy: 0.9894\n",
      "\n",
      "Accuracy: 98.94%\n",
      "\n",
      "learning rate: 1.8e-04\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 62us/sample - loss: 0.3695 - accuracy: 0.8938 - val_loss: 0.0923 - val_accuracy: 0.9740\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 54us/sample - loss: 0.0903 - accuracy: 0.9729 - val_loss: 0.0664 - val_accuracy: 0.9808\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 126us/sample - loss: 0.0613 - accuracy: 0.9809 - val_loss: 0.0611 - val_accuracy: 0.9830\n",
      "\n",
      "Accuracy: 98.30%\n",
      "\n",
      "learning rate: 1.0e-06\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 512\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 9s 168us/sample - loss: 2.2758 - accuracy: 0.2655 - val_loss: 2.2455 - val_accuracy: 0.4236\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 8s 138us/sample - loss: 2.2145 - accuracy: 0.5053 - val_loss: 2.1717 - val_accuracy: 0.6046\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 128us/sample - loss: 2.1320 - accuracy: 0.6202 - val_loss: 2.0713 - val_accuracy: 0.6850\n",
      "\n",
      "Accuracy: 68.50%\n",
      "\n",
      "learning rate: 1.0e-02\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 5\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 60us/sample - loss: 1.5650 - accuracy: 0.3523 - val_loss: 1.2933 - val_accuracy: 0.4494\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 48us/sample - loss: 0.9217 - accuracy: 0.6472 - val_loss: 0.6613 - val_accuracy: 0.7856\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 5s 85us/sample - loss: 0.6513 - accuracy: 0.7897 - val_loss: 0.4982 - val_accuracy: 0.8736\n",
      "\n",
      "Accuracy: 87.36%\n",
      "\n",
      "learning rate: 1.4e-03\n",
      "num_dense_layers: 1\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 62us/sample - loss: 1.3383 - accuracy: 0.5226 - val_loss: 0.2304 - val_accuracy: 0.9322\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 53us/sample - loss: 0.2098 - accuracy: 0.9346 - val_loss: 0.1193 - val_accuracy: 0.9642\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 123us/sample - loss: 0.1266 - accuracy: 0.9606 - val_loss: 0.0846 - val_accuracy: 0.9750\n",
      "\n",
      "Accuracy: 97.50%\n",
      "\n",
      "learning rate: 4.6e-04\n",
      "num_dense_layers: 3\n",
      "num_dense_nodes: 105\n",
      "activation: relu\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 9s 168us/sample - loss: 0.3599 - accuracy: 0.8924 - val_loss: 0.0980 - val_accuracy: 0.9706\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 7s 124us/sample - loss: 0.0884 - accuracy: 0.9728 - val_loss: 0.0749 - val_accuracy: 0.9788\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 7s 124us/sample - loss: 0.0643 - accuracy: 0.9799 - val_loss: 0.0548 - val_accuracy: 0.9860\n",
      "\n",
      "Accuracy: 98.60%\n",
      "\n",
      "learning rate: 1.0e-03\n",
      "num_dense_layers: 2\n",
      "num_dense_nodes: 512\n",
      "activation: sigmoid\n",
      "\n",
      "Train on 55000 samples, validate on 5000 samples\n",
      "Epoch 1/3\n",
      "55000/55000 [==============================] - 3s 62us/sample - loss: 1.1924 - accuracy: 0.5680 - val_loss: 0.1600 - val_accuracy: 0.9518\n",
      "Epoch 2/3\n",
      "55000/55000 [==============================] - 3s 54us/sample - loss: 0.1456 - accuracy: 0.9542 - val_loss: 0.0808 - val_accuracy: 0.9764\n",
      "Epoch 3/3\n",
      "55000/55000 [==============================] - 6s 117us/sample - loss: 0.0892 - accuracy: 0.9727 - val_loss: 0.0612 - val_accuracy: 0.9806\n",
      "\n",
      "Accuracy: 98.06%\n",
      "\n",
      "CPU times: user 12min 22s, sys: 2min 47s, total: 15min 9s\n",
      "Wall time: 9min 35s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "search_result = gp_minimize(func=fitness,\n",
    "                            dimensions=dimensions,\n",
    "                            acq_func='EI', # Expected Improvement.\n",
    "                            n_calls=40,\n",
    "                            x0=default_parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimization Progress\n",
    "\n",
    "The progress of the hyper-parameter optimization can be easily plotted. The best fitness value found is plotted on the y-axis, remember that this is the negated classification accuracy on the validation-set.\n",
    "\n",
    "Note how few hyper-parameters had to be tried before substantial improvements were found."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fb8da0d14a8>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEYCAYAAACZaxt6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deZxcVZ338c836aQ7EAKEQIyABCQuqIh0FJA1YRHQeQIMbuAYBzCgMuI4OODgODgDz4DIuDwuiChEReKuuKCB2IiMoCYQSFjCIkSESIRJJoSls/2eP+6p7kqnqrvWrurc7/v1uq+6955zq351IfXrc8695yoiMDMzq8eoVgdgZmYjn5OJmZnVzcnEzMzq5mRiZmZ1czIxM7O6OZmYmVndnEzMrCKS3iPp1lbHYe3JycS2CpJOkbRQ0lpJKyTdIOmQVseVV5JulnRGq+Ow4eNkYiOepA8DnwH+LzAZeAnwRWBWK+MqJqmj1TGYNZOTiY1okrYH/h34QET8ICKejYj1EfGTiPhIqtMp6TOSnkjLZyR1prIjJP1Z0j9JWplaNX+fyg6Q9BdJo4s+70RJd6f1UZLOl/SwpKclfUfSxFQ2VVJIOl3Sn4BfSRot6XJJT0l6RNLZqU5H4btI+mqK4XFJFxU+u9DFJOlTklal448rimuipKvT91sl6UdFZW+RtFjSakm/lbTvIOczJH1Q0h9TnJdJKvk7IemNkv4g6X/T6xvT/ouBQ4HPp5bi52v4T2sjjJOJjXQHAV3ADwepcwFwILAf8FrgDcDHispfBGwP7AqcDnxB0o4R8TvgWWBmUd1TgG+l9X8ATgAOB14MrAK+MOCzDwdeCbwJeC9wXIpj/3RssWuADcDewOuAY4DirqIDgGXAJOCTwFclKZV9A9gGeBWwC/BpAEmvA74GnAnsBHwZuL6QTMs4EZieYpwFnDawQkqaPwM+l973v4CfSdopIi4AfgOcHRHjI+LsQT7LthYR4cXLiF2AU4G/DFHnYeD4ou03AY+m9SOA54GOovKVwIFp/SLga2l9O7Lkskfavg84sui4KcB6oAOYCgSwV1H5r4Azi7aPSnU6yLrneoFxReXvBHrS+nuAh4rKtknHvih97iZgxxLf/UvAfwzYtww4vMy5CuDYou33AwuKYrg1rf8d8PsBx94GvCet3wyc0er/P7wM3+J+XBvpngYmSeqIiA1l6rwYWF60vTzt63uPAcc+B4xP698CfivpfcBJwB0RUXivPYAfStpUdOxGssRQ8NiAOB4rU7YHMAZY0d/YYNSAOn8prETEc6neeGAi8D8RsYot7QHMlvQPRfvGsvn3H6j4Mweeq+LvsnzAvuVkrTvLIXdz2Uh3G9lf9AO7jIo9QfajWvCStG9IEXEv2Y/kcWzexQXZj+5xEbFD0dIVEY8Xv0XR+gpgt6Lt3Qe8Vy8wqei9JkTEqyoI8zFgoqQdypRdPCDGbSLiukHerziucudq4Dkt1C18d09HnjNOJjaiRcT/Ah8nG+c4QdI2ksZIOk7SJ1O164CPSdpZ0qRU/5tVfMy3gHOAw4DvFu2/ArhY0h4A6f0Hu4LsO8A5knZNP/znFX2PFcB84HJJE9Lg/kslHT5UcOnYG4AvStoxff/DUvFXgLPSxQSStK2kN0vabpC3/Eh6n93T9/52iTo/B16WLsnukPR2YB/gp6n8SWCvoWK3rYeTiY14EXE58GGyQfW/kv01fjZQuKLpImAhcDewBLgj7avUdWQD6b+KiKeK9n8WuB6YL+kZ4HayQfJyvkKWMO4G7iT7Qd5A1jUG8G6yLqh7yQbzv0c2HlKJvyMbr7mfbMznQwARsZBs4P/z6T0fIhv7GMyPgUXAYrJB9q8OrBARTwNvAf6JrKvxn4G3FJ2fzwInpyvLPlfhd7ARTBFujZq1Qrq094qIGNhd1DKSApgWEQ+1OhYbWdwyMRsmksZJOj51C+0K/BuDX9JsNmI4mZgNHwGfIOtuupPs0uKPtzQiswZxN5eZmdXNLRMzM6tbbm9anDRpUkydOrVk2bPPPsu22247vAFVyLHVxrHVxrHVZmuObdGiRU9FxM5bFLT6FvxWLd3d3VFOT09P2bJWc2y1cWy1cWy12ZpjAxZGid9Ud3OZmVndnEzMzKxuTiZmZlY3JxMzM6ubk4mZmdUtt5cG12L+Lffy5WtvZeXTa9hlpwmceeohHHPYPq0Oy8ys5ZxMKjT/lnu59Ir59PZmz1B68qk1XHrFfAAnFDPLPXdzVejL197al0gKens38OVrb21RRGZm7cPJpEIrn15T1X4zszxxMqnQLjtNqGq/mVmeOJlU6MxTD6Gzc/Mhps7ODs489ZAWRWRm1j48AF+hwiD7xZ//BRs3bmKnHbflA+8+3IPvZma4ZVKVYw7bh2lTs8ky//O8E5xIzMwSJ5MqdXWNAeCF3vUtjsTMrH04mVSpa6yTiZnZQE4mVepKg/AvDLjnxMwsz5xMqtTZmbVMet0yMTPr42RSpa7OQjeXWyZmZgVOJlXqGpu6uda5ZWJmVuBkUqX+bi63TMzMCpxMqjQuXRr8/AtumZiZFTiZVKkzdXP1upvLzKyPk0mVPABvZrYlJ5Mq9d9n4paJmVlBy5OJpImSbpT0YHrdsUSdGZIWFy0vSDohlV0j6ZGisv2aGa8H4M3MttTyZAKcDyyIiGnAgrS9mYjoiYj9ImI/YCbwHDC/qMpHCuURsbiZwfrSYDOzLbVDMpkFzE3rc4EThqh/MnBDRDzX1KjK6J/o0S0TM7MCRURrA5BWR8QOaV3AqsJ2mfq/Av4rIn6atq8BDgJ6SS2biOgtc+wcYA7A5MmTu+fNm1fyM9auXcv48eNLlj2x8lm++O37mDJpHB9456sq+5INNFhsrebYauPYauPYalNvbDNmzFgUEdO3KIiIpi/ATcDSEsssYPWAuqsGeZ8pwF+BMQP2Cegka9l8vJKYuru7o5yenp6yZY8+9lQcfNJl8Y6zrypbp5kGi63VHFttHFttHFtt6o0NWBglflOH5UmLEXFUuTJJT0qaEhErJE0BVg7yVm8DfhgRfQMWEbEirfZKuho4tyFBl+EBeDOzLbXDmMn1wOy0Phv48SB13wlcV7wjJaBCF9kJZC2epvGlwWZmW2qHZHIJcLSkB4Gj0jaSpku6qlBJ0lRgd+DXA46/VtISYAkwCbiomcH23bS4zi0TM7OCYenmGkxEPA0cWWL/QuCMou1HgV1L1JvZzPgGGjsmO2Xr1m1g06Zg1CgN58ebmbWldmiZjCijRsldXWZmAziZ1KDQ1eXJHs3MMk4mNSjMHOwbF83MMk4mNeifOdgtEzMzcDKpie81MTPbnJNJDTwAb2a2OSeTGnSN9b0mZmbFnExq0DdzsJ8Db2YGOJnUxN1cZmabczKpQae7uczMNuNkUoNCy6TXLRMzM8DJpCb995m4ZWJmBk4mNen0mImZ2WacTGowrm9uLrdMzMzAyaQmhTvgn/elwWZmgJNJTbrSRI+eNdjMLONkUgMPwJuZbc7JpAYegDcz25yTSQ26PGuwmdlmnExq0DedisdMzMwAJ5Oa9I2Z+GouMzPAyaQmftKimdnmnExq0Dc3l29aNDMDnExq0jdrsFsmZmaAk0lNOscWLg3eQES0OBozs9ZzMqnB6NGjGDtmNADr3NVlZuZkUqu+QXgnEzMzJ5NadXmyRzOzPhUnE0lvlbRdWv+YpB9I2r8RQUiaKOlGSQ+m1x3L1PukpHsk3Sfpc5KU9ndLWiLpoeL9zdTppy2amfWppmXyrxHxjKRDgKOArwJfalAc5wMLImIasCBtb0bSG4GDgX2BVwOvBw5PxV8C3gtMS8uxDYqrLHdzmZn1qyaZbEyvbwaujIifAWMbFMcsYG5anwucUKJOAF3pMzuBMcCTkqYAEyLi9sgurfp6meMbqmusJ3s0MytQpZe2Svop8ARwNPA64Hng9xHx2rqDkFZHxA5pXcCqwvaAep8CzgAEfD4iLpA0HbgkIo5KdQ4FzouIt5Q4fg4wB2Dy5Mnd8+bNKxnP2rVrGT9+/KAxX/2jZTz82DPMnjWNaS/ZvopvW59KYmsVx1Ybx1Ybx1abemObMWPGooiYvkVBRFS0AOOAE4G90/aLgKOrOP4mYGmJZRawekDdVSWO3xv4GTA+LbcBhwLTgZuK6h0K/HSoeLq7u6Ocnp6esmUF5/3nD+Lgky6LX9/+wJB1G6mS2FrFsdXGsdXGsdWm3tiAhVHiN7VjqCwk6RmyLibIWgRRGPdO+ydUks0itRzKfMaTkqZExIrUbbWyRLUTgdsjYm065gbgIOAbwG5F9XYDHq8kpnp4fi4zs35DjplExHYRMSEtW6w3KI7rgdlpfTbw4xJ1/gQcLqlD0hiywff7ImIFsEbSgamL7N1ljm8oP23RzKxfu9xncglwtKQHya4UuwRA0nRJV6U63wMeBpYAdwF3RcRPUtn7gauAh1KdG5odcJeftmhm1qeabq5S925EI1onEfE0cGSJ/QvJBtyJiI3AmWWOX0h2ufCwKUz26JmDzcwqSCYRsd1wBDLSuGViZtZvyGRSLN2ZPo3sfg8AIuKWRgc1EnR6AN7MrE/FyUTSGcA5ZFdLLQYOJLs8d2ZzQmtv41Iy6fUAvJlZVQPw55BNYbI8ImaQ3bi4uilRjQCFbq7n3TIxM6sqmbwQES8ASOqMiPuBlzcnrPbX2dcycTIxM6tmzOTPknYAfgTcKGkVsLw5YbW//gF4d3OZmVWcTCLixLR6oaQeYHvgF02JagToKjwHfp1bJmZmVV3NVRARv250ICNNpwfgzcz6VPNwrLmpm6uwvaOkrzUnrPbn+0zMzPpVMwC/b0T0Xb0VEavIrujKJc/NZWbWr5pkMqr4cbqSJlJjN9nWwLMGm5n1qyYZXA7cJum7afutwMWND2lkcDeXmVm/aq7m+rqkhfTf8X5SRNzbnLDaX2d6bG/vug1EBOkZL2ZmuVRVN1VKHrlNIMU6OkbT0TGKDRs2sX7DRsaOyW2Pn5lZ2zzPZETqu9fEg/BmlnNOJnXo6vKUKmZmUN2swTOBU8kmd1wK3A0sjYjeJsXW9rrGerJHMzOobszka8CHgDHAvsAJwKuAvZsQ14jgu+DNzDLVJJPlEfGjtP7dQWvmhC8PNjPLVDNmcoukf5Svge3ju+DNzDLVtEz2AV4DnCdpEdnTFhdHRG5bKf33mrhlYmb5Vs1Ni38LIGkc/YnlAHLc5eWWiZlZpuo77SLieWBRWnKtcGmwx0zMLO98n0kdCpcGO5mYWd45mdTBMwebmWUqSibK7N7sYEaaznRpsO8zMbO8qyiZREQAP29yLCOOWyZmZplqurnukPT6pkUyAvWNmaxzy8TM8q2aZHIAcLukhyXdLWmJpLvrDUDSREk3Snowve5Ypt4nJd0j6T5JnyvcPCnpZknLJC1Oyy71xlQpT/RoZpap5tLgNzUphvOBBRFxiaTz0/Z5xRUkvRE4mGxOMIBbgcOBm9P2qRGxsEnxlVXo5nr+BScTM8u3alomfwIOBWZHxHIggMkNiGEWMDetzyWbQHKgALqAsUAn2WSTTzbgs+vSmZ5n0utuLjPLOWVj6xVUlL4EbAJmRsQrU3fU/IioaxxF0uqI2CGtC1hV2B5Q71PAGYCAz0fEBWn/zcBOwEbg+8BFUeZLSZoDzAGYPHly97x580rGtHbtWsaPHz9k7A8/toarf/QAe+62Haef+PIh6zdCpbG1gmOrjWOrjWOrTb2xzZgxY1FETN+iICIqWoA70uudRfvuqvDYm8iegTJwmQWsHlB3VYnj9wZ+BoxPy23Aoals1/S6HTAfeHclMXV3d0c5PT09ZcuKLbn/8Tj4pMvived9s6L6jVBpbK3g2Grj2Grj2GpTb2zAwijxm1rNmMl6SaPJupyQtDNZS2VIEXFUuTJJT0qaEhErJE0BVpaodiJwe0SsTcfcABwE/CYiHk+f8YykbwFvAL5exfeqWf99Jh4zMbN8q2bM5HPAD4FdJF1MNgj+nw2I4XpgdlqfDfy4RJ0/AYdL6pA0hmzw/b60PQkg7X8LWYtnWPQ9A95jJmaWc9XMGnxtmnr+SLJxixMi4r4GxHAJ8B1JpwPLgbcBSJoOnBURZwDfA2YCS8haRr+IiJ9I2hb4ZUoko8m6077SgJgqMs4TPZqZAdU9A/7SiDgPuL/EvppFxNNkCWrg/oVkA+5ExEbgzBJ1ngW66/n8enT6SYtmZkB13VxHl9h3XKMCGYn6urk8N5eZ5dyQLRNJ7wPeD+w14I737YD/blZgI0FHxyhGjxIbN25iw4aNdHSMbnVIZmYtUUk31/FkA9vLgL8p2v9MRPxPU6IaISTR2TmG555fxwu9GxjvZGJmOVVJN9dLgfVkyWQN8ExakDSxeaGNDF2FcRM/B97McqySlskVwAJgT7JH9aqoLIC9mhDXiNE3pYrHTcwsx4ZsmUTE5yLilcDVEbFXROxZtOQ6kYAvDzYzg+ruM3lfmo9rGtmki4X9tzQjsJGicHnw804mZpZj1dxncgZwDrAbsBg4kGyOrJnNCW1k6HI3l5lZVfeZnAO8HlgeETOA1wGrmxLVCNLlGxfNzKpKJi9ExAsAkjoj4n5geOZdb2Odfg68mVlVswb/WdIOwI+AGyWtIptLK9e6+mYOdjeXmeVXNQPwJ6bVCyX1ANsDv2hKVCNI4dG9vs/EzPKsmpZJn4j4daMDGan6komfA29mOVbNmImV0DW2cAe8u7nMLL+cTOrkAXgzsxqSiaRt0+N7jf5uLg/Am1meDZlMJI2SdIqkn0laSfZwrBWS7pV0maS9mx9m+/J9JmZmlbVMeshmDv4o8KKI2D0idgEOAW4HLpX0ribG2Nb6u7ncMjGz/Krkaq6jImKLP7vTs0y+D3w/PYM9l8YVurl8abCZ5VglswavB5D0WUkarE4eFbq5nvelwWaWY9UMwD8DXC9pWwBJb5KU68f2Qn83V68vDTazHKvmDviPSToFuFnSOmAtcH7TIhsh+u4z8QC8meVYNVPQHwm8F3gWmAKcFhHLmhXYSNHlAXgzs6q6uS4A/jUijgBOBr4tKdfPMoH+h2P1umViZjlWTTfXzKL1JZKOI7ua643NCGykGNc30aNbJmaWX5XctFjuCq4VwJGD1cmDTk/0aGZW2U2Lkv5B0kuKd0oaCxwkaS4wuynRjQBjx4xGgvUbNrJh46ZWh2Nm1hKVdHMdC5wGXCdpT7JH9XYBo4H5wGci4s7mhdjeJNHVOYbnX1jPunUb6Bg3ttUhmZkNu0paJpdGxBeBo4E9yLq29o+IPSLivfUmEkkTJd0o6cH0umOZepdKWpqWtxft31PS7yQ9JOnbqcU0rDp9ebCZ5VwlyeSw9PqbiFgfESsiYnUDYzgfWBAR04AFlLh3RdKbgf2B/YADgHMlTUjFlwKfjoi9gVXA6Q2MrSJdnobezHKukmSyQNJtwIsknSapW1JnA2OYBcxN63OBE0rU2Qe4JSI2RMSzwN3AsWngfybwvSGOb6r+mYN9RZeZ5VMlc3OdC7wL2AjsCfwrsFTSPZK+3YAYJqcrwwD+AkwuUecusuSxjaRJwAxgd2AnYHVEFH7F/wzs2oCYqtLlyR7NLOcUEZVVlF4WEQ8UbY8HXh0Rt1dw7E3Ai0oUXQDMjYgdiuquiogtxk0kXQC8FfgrsBL4A/BN4PbUxYWk3YEbIuLVZeKYA8wBmDx5cve8efNKxrt27VrGjx8/1Nfqc9X37+fRJ9Zy2okvY6/dJgx9QB2qjW04ObbaOLbaOLba1BvbjBkzFkXE9C0KIqKiBegETgH+Bfh4Yan0+EHedxkwJa1PAZZVcMy3gOMBAU8BHWn/QcAvK/nc7u7uKKenp6dsWSkf/o/vxsEnXRa/XfhwVcfVotrYhpNjq41jq41jq029sQELo8RvajXTqfyYbHxjA9n8XIWlXtfTf5/K7PQ5m5E0WtJOaX1fYF9gfvpiPWTTu5Q9vtm6xhbugnc3l5nlU8XTqQC7RcSxTYjhEuA7kk4HlgNvA5A0HTgrIs4AxgC/STfarwHeFf3jJOcB8yRdBNwJfLUJMQ7Kkz2aWd5Vk0x+K+k1EbGkkQFExNOkaVkG7F8InJHWXyC7oqvU8X8E3tDImKrlyR7NLO+qSSaHAO+R9AjQSzZeERGxb1MiG0HGuWViZjlXTTI5rmlRjHB9kz16zMTMcqqaKeiXNzOQkazvpkXPHGxmOVXJFPS3ptdnJK1Jr4VlTfNDbH8egDezvBuyZRIRh6TX7ZofzshUmOjRd8CbWV5V8wz46WQ3LE4tPs4D8G6ZmJlVMwB/LfARYAngp0AV6Z/o0S0TM8unapLJXyPi+qZFMoL1TfTolomZ5VQ1yeTfJF1F9syR3sLOiPhBw6MaYQrJ5Hm3TMwsp6pJJn8PvIJsapNCN1cAuU8mvgPezPKummTy+oh4edMiGcE8AG9meVfNrMG/lVRyfqy86yo8A96XBptZTlXTMjkQWOy5ubbU6QF4M8u5apJJM6af3yqM6yp0c7llYmb55Lm5GmDsmMId8BvYtCkYNUotjsjMbHhVM2ZiZYwaJU+pYma55mTSIL6iy8zyzMmkQXyviZnlmZNJg3SNLTwgyy0TM8sfJ5MG8WSPZpZnTiYN0tXle03MLL+cTBqk0M3lyR7NLI+cTBrEA/BmlmdOJg3iS4PNLM+cTBrEA/BmlmdOJg3SmcZMen1psJnlkJNJg3iyRzPLMyeTBuns6+Zyy8TM8sfJpEH67oB3y8TMcqjlyUTSREk3Snowve5Ypt6lkpam5e1F+6+R9IikxWnZb/ii7+cBeDPLs5YnE+B8YEFETAMWpO3NSHozsD+wH3AAcK6kCUVVPhIR+6Vl8XAEPZCftmhmedYOyWQWMDetzwVOKFFnH+CWiNgQEc8Cd9NmT350y8TM8kwR0doApNURsUNaF7CqsF1U5xjg34CjgW2A3wNfiIjLJV0DHET2XPoFwPkR0Vvms+YAcwAmT57cPW/evJIxrV27lvHjx1f1PZY9uppv/OQhpr1kArNnvayqY6tRS2zDxbHVxrHVxrHVpt7YZsyYsSgipm9REBFNX4CbgKUlllnA6gF1V5V5jwuAxcCNwLXAh9L+KYCATrKWzccriam7uzvK6enpKVtWzh1L/xQHn3RZfOBj11V9bDVqiW24OLbaOLbaOLba1BsbsDBK/KZW/Az4ekTEUeXKJD0paUpErJA0BVhZ5j0uBi5Ox3wLeCDtX5Gq9Eq6Gji3ocFXyN1cZpZn7TBmcj0wO63PBn48sIKk0ZJ2Suv7AvsC89P2lPQqsvGWpcMQ8xY6fWmwmeXYsLRMhnAJ8B1JpwPLgbcBSJoOnBURZwBjgN9k+YI1wLsionDZ1LWSdibr6loMnDXM8QPFLRNfzWVm+dPyZBIRTwNHlti/EDgjrb9AdkVXqeNnNjXACvXPGuyWiZnlTzt0c20VCsnEEz2aWR45mTRI59j+Afho8eXWZmbDzcmkQUaPHsXYMaOJgHXrN7Y6HDOzYeVk0kCdHjcxs5xyMmmgrrG+18TM8snJpIE82aOZ5ZWTSQP5Lngzyysnkwbqu9fElwebWc44mTRQ370mbpmYWc44mTSQp1Qxs7xyMmmgwmSPz7tlYmY542TSQIWWibu5zCxvnEwaqH+yR3dzmVm+OJk0UKcvDTaznHIyaSBfzWVmeeVk0kC+z8TM8srJpIH6Lg1+wS0TM8sXJ5MG6io8B36dk4mZ5YuTSQN1+qZFM8spJ5MG8gC8meVVR6sD2JosXfYEALfd8Qh/e+aVnHnqIRxz2D595fNvuZcvX3srK59ewy47TdisfLCy4vInn1rD5OseqOq96y3fmmMzs8ZQXp9XPn369Fi4cGHJsptvvpkjjjiiqvebf8u9XPLFX272yN7Ozg7OO+sYjjlsH+bfci+XXjF/s2edFMqBsmVDHdvs8q05tsJ/t2YluqHKN0t0k4b3sx2bY6v1DytJiyJi+hb7nUy2VEsy+dszr+TJp9aULOvoGMWGDZuqDbGiY5tdvjXG1tU5hnf8TTdPrVrLL2+5j/XFfwCM7eDs2UdwxEEv4+bbHuDzc2+mt+hS70aVA017b8fm2CqKbcAfVpVyMhmg0cnk0JM/RU5PpZmNUJMnTeD7X55T1THlkokH4Btkl50mlNw/edJ29Mz7RyZP2q5s+WBlQx3b7PKtMbYJ47uYffKBJcsKdpgwrqnlrfxsx9ac8lZ+dq2xrXy6dG9KLZxMGuTMUw/puzS4oLOzgzNPPZQxY0Zz5qmHli0frGyoY5tdvjXG9qHTZ/Ledx7C5Enl/gCYwE+v/kBTy1v52Y7NsRWU+yO4FqMvvPDChr3ZSHLllVdeOGdO6ebdo48+ytSpU6t6v5fusTNTdp7A/Q8/yXPP9zJ50gTOOW1GX3/kYOXVHPvsc9W9t2MrH9uO24/j9sWPsHFj/9hKZ2cH55w2g5fusXNTy6fv+5KWfbZjc2wDj63GJz7xiRUXXnjhlQP3e8ykhFrGTIaLY6tNudja+eoax+bYRtLVXERESxfgrcA9wCZg+iD1jgWWAQ8B5xft3xP4Xdr/bWBsJZ/b3d0d5fT09JQtazXHVhvHVhvHVputOTZgYZT4TW2HMZOlwEnALeUqSBoNfAE4DtgHeKekQkq9FPh0ROwNrAJOb264ZmY2UMuTSUTcFxHLhqj2BuChiPhjRKwD5gGzJAmYCXwv1ZsLnNC8aM3MrJS2GTORdDNwbkRsMZAh6WTg2Ig4I23/HXAAcCFwe2qVIGl34IaIeHWZz5gDzAGYPHly97x580rGsnbtWsaPH1/vV2oKx1Ybx1Ybx1abrTm2GTNmlBwzGZa5uSTdBLyoRNEFEfHj4YgBICKuBK6EbAC+3GDxSBxIbgeOrTaOrTaOrTbNim1YkklEHFXnWzwO7F60vVva9zSwg6SOiNhQtN/MzIbRSJk1+A/ANEl7kiWLdwCnRERI6gFOJhtHmQ1U1NJZtGjRU5KWlymeBDxVf9hN4dhq49hq49hqszXHtkepnS0fM5F0IvD/gJ2B1cDiiHiTpBcDV0XE8ane8cBngNHA1yLi4rR/L7JEMhG4E3hXRPTWGdPCUqnVYMMAAAdZSURBVH2C7cCx1cax1cax1SaPsbW8ZRIRPwR+WGL/E8DxRds/B35eot4fya72MjOzFmn5pcFmZjbyOZmUtsW8M23EsdXGsdXGsdUmd7G1fMzEzMxGPrdMzMysbk4mZmZWNyeTASQdK2mZpIcknd/qeIpJelTSEkmLJZWeP3/4YvmapJWSlhbtmyjpRkkPptcd2yi2CyU9ns7d4nSpeSti211Sj6R7Jd0j6Zy0v+XnbpDYWn7uJHVJ+r2ku1Jsn0j795T0u/Tv9duSxrZRbNdIeqTovO033LEVxTha0p2Sfpq2G3/eSk0lnNeF7B6Wh4G9gLHAXcA+rY6rKL5HgUmtjiPFchiwP7C0aN8nSY8HAM4HLm2j2C4km/ut1edtCrB/Wt8OeIBsJuyWn7tBYmv5uQMEjE/rY8geO3Eg8B3gHWn/FcD72ii2a4CTW/3/XIrrw8C3gJ+m7YafN7dMNldyduIWx9SWIuIW4H8G7J5FNnMztHAG5zKxtYWIWBERd6T1Z4D7gF1pg3M3SGwtF5m1aXNMWoI2mDV8kNjagqTdgDcDV6Xtpsy27mSyuV2Bx4q2/0yb/GNKApgvaVGaAbndTI6IFWn9L8DkVgZTwtmS7k7dYC3pgismaSrwOrK/ZNvq3A2IDdrg3KWumsXASuBGsl6E1ZHNywct/Pc6MLaIKJy3i9N5+7SkzlbERjZzyD+TPYAQYCeacN6cTEaWQyJif7KHhH1A0mGtDqicyNrPbfPXGfAl4KXAfsAK4PJWBiNpPPB94EMRsaa4rNXnrkRsbXHuImJjROxHNqHrG4BXtCKOUgbGJunVwEfJYnw92XRP5w13XJLeAqyMiEXN/iwnk82Vm524LUTE4+l1JdkUNO02jcyTkqYApNeVLY6nT0Q8mf7BbwK+QgvPnaQxZD/W10bED9Lutjh3pWJrp3OX4lkN9AAHkWYNT0Ut//daFNuxqdswIpsr8Gpac94OBv6PpEfJuu1nAp+lCefNyWRzfbMTp6sb3gFc3+KYAJC0raTtCuvAMWSPPG4n15PN3AxVzOA8HAo/1MmJtOjcpf7qrwL3RcR/FRW1/NyVi60dzp2knSXtkNbHAUeTjekUZg2H1p23UrHdX/THgcjGJIb9vEXERyNit4iYSvZ79quIOJVmnLdWX2XQbgvZ5JIPkPXHXtDqeIri2ovs6rK7gHtaHRtwHVmXx3qyPtfTyfpiFwAPAjcBE9sotm8AS4C7yX64p7QotkPIurDuBhan5fh2OHeDxNbycwfsSzYr+N1kP8ofT/v3An4PPAR8F+hso9h+lc7bUuCbpCu+WrUAR9B/NVfDz5unUzEzs7q5m8vMzOrmZGJmZnVzMjEzs7o5mZiZWd2cTMzMrG5OJmZmVjcnEzMzq5uTieWGpJB0edH2uZIubMD7Ti1+dkozSfqgpPskXVvn+6wttW5WKycTy5Ne4CRJk1odSDFlKv23+H7g6MimxDBrG04mlicbgCuBfyzeObBlUWixpP33pyfmPSDpWklHSfrv9ETE4on7OlL5fZK+J2mb9F7vSk/hWyzpy5JGF33mMklfJ5tuY/cBMX1Y0tK0fCjtu4JsGowbJG32HVL5u9N053dJ+kba96P0yIJ7hnpsQZr/7Wfp+KWS3l6izg8kXSTpFkl/knTUYO9p+eFkYnnzBeBUSdtXWH9vsinXX5GWU8jmsDoX+Jeiei8HvhgRrwTWAO+X9Erg7cDBkU1PvhEoblFMS8e8KiKWF3ZK6gb+HjiA7Il975X0uog4C3gCmBERny4OUtKrgI8BMyPitcA5qei0iOgGpgMflLTTIN/1WOCJiHhtRLwa+EWJOq8hexbGYekz3EIywMnEciay53N8HfhghYc8EhFLIpt+/R5gQWQT2i0BphbVeywi/jutf5Ms4RwJdAN/SA9OOpKsZVGwPCJuL/GZhwA/jIhnI3uC3w+AQ4eIcybw3Yh4Kn3PwpMmPyjpLuB2stbPtEHeYwlwtKRLJR0aEf9bXJhaW9sDhUQ2Blg9RFyWEx1DVzHb6nwGuIPsGROQdX8V/2HVVbTeW7S+qWh7E5v/+xk4Y2qQPRt8bkR8tEwcz1YRc9UkHQEcBRwUEc9JupnNv9tmIuIBSfuTzRR8kaQFEfHvRVX2ARZFxMa0vS/t9xgEaxG3TCx30l/t3yGbmh7gSWAXSTulR6u+pYa3fYmkg9L6KcCtZFPKnyxpFwBJEyXtUcF7/QY4QdI26dk1J6Z9g/kV8NZCN5akiWStiFUpkbyCrMusLEkvBp6LiG8ClwH7D6jyGrJp6Qv2JZt23cwtE8uty4GzASJivaR/J3u+w+PA/TW83zKyRyl/DbgX+FL6Ef8YMD9drbUe+ACwfJD3ISLukHRNigfgqoi4c4hj7pF0MfBrSRvJnq9xJnCWpPtSfKW61Iq9BrhM0qYU6/tKlP+uaPvVuGViiZ9nYmZmdXM3l5mZ1c3JxMzM6uZkYmZmdXMyMTOzujmZmJlZ3ZxMzMysbk4mZmZWt/8PgN5vN5CxO6QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_convergence(search_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best Hyper-Parameters\n",
    "\n",
    "The best hyper-parameters found by the Bayesian optimizer are packed as a list because that is what it uses internally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.01, 2, 104, 'relu']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_result.x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can convert these parameters to a dict with proper names for the search-space dimensions.\n",
    "\n",
    "First we need a reference to the search-space object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "space = search_result.space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can use it to create a dict where the hyper-parameters have the proper names of the search-space dimensions. This is a bit awkward."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.01,\n",
       " 'num_dense_layers': 2,\n",
       " 'num_dense_nodes': 104,\n",
       " 'activation': 'relu'}"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "space.point_to_dict(search_result.x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the fitness value associated with these hyper-parameters. This is a negative number because the Bayesian optimizer performs minimization, so we had to negate the classification accuracy which is posed as a maximization problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.9904"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_result.fun"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also see all the hyper-parameters tried by the Bayesian optimizer and their associated fitness values (the negated classification accuracies). These are sorted so the highest classification accuracies are shown first.\n",
    "\n",
    "It appears that 'relu' activation was generally better than 'sigmoid'. Otherwise it can be difficult to see a pattern of which parameter choices are good. We really need to plot these results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(-0.9904, [0.01, 2, 104, 'relu']),\n",
       " (-0.9894, [0.0029398096826104927, 2, 299, 'relu']),\n",
       " (-0.9892, [0.000695826471438557, 1, 365, 'relu']),\n",
       " (-0.989, [0.000658408209046353, 1, 512, 'relu']),\n",
       " (-0.989, [0.0034329866293724173, 2, 451, 'relu']),\n",
       " (-0.9882, [0.0027527776962813647, 2, 441, 'relu']),\n",
       " (-0.9878, [0.0004989932212151087, 2, 309, 'relu']),\n",
       " (-0.987, [0.009700568764470742, 3, 132, 'relu']),\n",
       " (-0.986, [0.00045720538478821585, 3, 105, 'relu']),\n",
       " (-0.9854, [0.005470139269128146, 1, 512, 'relu']),\n",
       " (-0.983, [0.0001756750312956145, 3, 512, 'relu']),\n",
       " (-0.9806, [0.00010996497783044355, 1, 512, 'relu']),\n",
       " (-0.9806, [0.0010101832379944083, 2, 512, 'sigmoid']),\n",
       " (-0.9778, [3.542177009701199e-05, 4, 512, 'relu']),\n",
       " (-0.975, [0.0014121597194479596, 1, 512, 'sigmoid']),\n",
       " (-0.9748, [5.4968307266680435e-05, 2, 512, 'relu']),\n",
       " (-0.9746, [7.662555005436298e-05, 5, 154, 'relu']),\n",
       " (-0.9734, [3.623853563736315e-05, 5, 446, 'relu']),\n",
       " (-0.9708, [0.0006699631867581338, 3, 230, 'sigmoid']),\n",
       " (-0.9704, [0.0004806874905820532, 2, 225, 'sigmoid']),\n",
       " (-0.97, [6.592244748528267e-05, 4, 143, 'relu']),\n",
       " (-0.97, [0.0005519055319736135, 3, 277, 'sigmoid']),\n",
       " (-0.9606, [0.0024078003512664754, 3, 5, 'relu']),\n",
       " (-0.9592, [0.0002839701110319199, 2, 512, 'sigmoid']),\n",
       " (-0.931, [0.00021682795230749897, 2, 195, 'sigmoid']),\n",
       " (-0.9256, [1.152958307942762e-05, 5, 182, 'relu']),\n",
       " (-0.9226, [1.068205878028229e-05, 1, 496, 'relu']),\n",
       " (-0.9176, [0.00037183580449927443, 3, 72, 'sigmoid']),\n",
       " (-0.912, [5.101545871674443e-06, 3, 512, 'relu']),\n",
       " (-0.8736, [0.01, 3, 5, 'relu']),\n",
       " (-0.725, [1e-05, 1, 16, 'relu']),\n",
       " (-0.7048, [1e-06, 5, 512, 'relu']),\n",
       " (-0.6904, [0.0001581796478320478, 4, 112, 'sigmoid']),\n",
       " (-0.685, [1e-06, 2, 512, 'relu']),\n",
       " (-0.5936, [4.158851683068185e-05, 1, 512, 'sigmoid']),\n",
       " (-0.106, [1.42502793530235e-06, 4, 512, 'sigmoid']),\n",
       " (-0.106, [0.0028996992551655475, 4, 156, 'sigmoid']),\n",
       " (-0.106, [0.0034427213718442543, 3, 5, 'sigmoid']),\n",
       " (-0.106, [0.006781678732829231, 4, 466, 'relu']),\n",
       " (-0.0978, [0.0009844064941977042, 5, 122, 'sigmoid'])]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sorted(zip(search_result.func_vals, search_result.x_iters))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plots\n",
    "\n",
    "There are several plotting functions available in the `skopt` library. For example, we can plot a histogram for the `activation` parameter, which shows the distribution of samples during the hyper-parameter optimization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAARu0lEQVR4nO3de5BlVXmG8eflYiBAKRQTgoRxkKAGL4wwIIgShGjhJQFLhWAEjOiIirfCVFGmiBg1QkSpSlRwiARN0EAEFMWgFAFBE4SZERgQ8YKgEoQhGGTEcP3yx9kdm2a659Az+xym1/Or6jp7r337uuvMO7tXr71OqgpJUjs2GHcBkqTRMvglqTEGvyQ1xuCXpMYY/JLUmI3GXcAwtt5661qwYMG4y5Ck9cqyZcvurKp5U9vXi+BfsGABS5cuHXcZkrReSXLL6trt6pGkxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMasF0/uro0Fx14w7hL0OHXzCS8fdwnSWHjHL0mNMfglqTEGvyQ1xuCXpMb0FvxJtk9ySZLvJrk+yTu79uOT3Jrk6u7rZX3VIEl6tD5H9TwIHFNVy5NsASxLclG37eSqOqnHa0uSptFb8FfVbcBt3fI9SW4AtuvrepKk4Yykjz/JAuC5wLe7pqOTXJvk9CRbTnPM4iRLkyxduXLlKMqUpCb0HvxJNgfOAd5VVb8ETgF2BBYy+I3go6s7rqqWVNWiqlo0b96jPjJSkjRLvQZ/ko0ZhP6ZVXUuQFXdXlUPVdXDwGnAHn3WIEl6pD5H9QT4NHBDVX1sUvu2k3Z7JXBdXzVIkh6tz1E9ewOHASuSXN21vRc4NMlCoICbgTf3WIMkaYo+R/V8E8hqNn21r2tKktbMJ3clqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSY3oL/iTbJ7kkyXeTXJ/knV37VkkuSvKD7nXLvmqQJD1an3f8DwLHVNXOwJ7A25LsDBwLXFxVOwEXd+uSpBHpLfir6raqWt4t3wPcAGwHHAh8ptvtM8BBfdUgSXq0kfTxJ1kAPBf4NrBNVd3Wbfo5sM0oapAkDfQe/Ek2B84B3lVVv5y8raoKqGmOW5xkaZKlK1eu7LtMSWpGr8GfZGMGoX9mVZ3bNd+eZNtu+7bAHas7tqqWVNWiqlo0b968PsuUpKb0OaonwKeBG6rqY5M2nQ8c0S0fAXyprxokSY+2UY/n3hs4DFiR5Oqu7b3ACcDZSY4EbgEO7rEGSdIUvQV/VX0TyDSb9+/rupKkmfnkriQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5Ias8bgT3LiMG2SpPXDMHf8L15N20vXdSGSpNGYdlrmJG8B3go8Ncm1kzZtAXyr78IkSf2YaT7+zwH/BnwYOHZS+z1VdVevVUmSejNt8FfV3cDdwKFJNgS26fbfPMnmVfWTEdUoSVqH1vgJXEmOBo4Hbgce7poLeE5/ZUmS+jLMRy++C3h6Vf1338VIkvo3zKienzLo8pEkzQHD3PHfBFya5ALgvonGqvpYb1VJknozTPD/pPt6QvclSVqPrTH4q+r9oyhEkjQaw4zquYTBKJ5HqKr9eqlIktSrYbp63jNpeRPgVcCD/ZQjSerbMF09y6Y0fSvJlT3VI0nq2TBdPVtNWt0A2A14Ym8VSZJ6NUxXzzIGffxh0MXzY+DIPouSJPVnmK6eHUZRiCRpNIbp6tkYeAuwT9d0KfCpqnqgx7okST0ZpqvnFGBj4JPd+mFd2xv7KkqS1J9hgn/3qtpl0vq/J7lmTQclOR14BXBHVT2razseeBOwstvtvVX11cdWsiRpbQwzSdtDSXacWEnyVOChIY47AzhgNe0nV9XC7svQl6QRG+aO/y+AS5LcxGBkz1OAP1/TQVV1WZIFa1WdJGmdG2ZUz8VJdgKe3jXdWFX3zXTMGhyd5HBgKXBMVf1idTslWQwsBpg/f/5aXE6SNNm0XT1JXpfkMICquq+qrq2qa4GDk7x2ltc7BdgRWAjcBnx0uh2raklVLaqqRfPmzZvl5SRJU83Ux/924LzVtJ8LHDObi1XV7VX1UFU9DJwG7DGb80iSZm+m4N+4qlZNbayqXzEY3vmYJdl20uorgetmcx5J0uzN1Me/aZLNuqD/f0m2YIgPZEnyeWBfYOskPwPeB+ybZCGDKSBuBt48y7olSbM0U/B/GvhCkqOq6haAbpTOJ7ptM6qqQ6c5pyRpjKYN/qo6Kckq4LIkm3fNq4ATquqUkVQnSVrnZhzOWVWnAqd23TtU1T0jqUqS1JthHuAy8KUeLTj2gnGXoMexm094+To/5zBTNkiS5hCDX5Ias8bgT/LbSY5Lclq3vlOSV/RfmiSpD8Pc8f8jcB+wV7d+K/DB3iqSJPVqmODfsar+FngAoKruZTBLpyRpPTRM8N+fZFMGT9vSzc2/NrNzSpLGaJjhnO8DLgS2T3ImsDfw+j6LkiT1Z5j5+C9KshzYk0EXzzur6s7eK5Mk9WLa4E+y65Sm27rX+UnmV9Xy/sqSJPVlpjv+aT8khUF//37ruBZJ0gjMNEnbi0ZZiCRpNNbYx59kE+CtwAsY3OlfDpxaVf/bc22SpB4MM6rns8A9wN93668F/gl4TV9FSZL6M0zwP6uqdp60fkmS7/ZVkCSpX8M8wLU8yZ4TK0meByztryRJUp+GuePfDfiPJD/p1ucDNyZZAVRVPae36iRJ69wwwX9A71VIkkZmmCd3b0myJbD95P19gEuS1k/DDOf8AIO5eX5EN1EbPsAlSeutYbp6DmYwNfP9fRcjSerfMKN6rgOe1HchkqTRGOaO/8PAd5Jcx6R5+KvqT3qrSpLUm2GC/zPAicAK4OF+y5Ek9W2Y4L+3qv6u90okSSMxTPBfnuTDwPk8sqvH4ZyStB4aJvif273uOanN4ZyStJ4a5gEu5+WXpDlkmDt+krwceCawyURbVf11X0VJkvqzxnH8SU4FDgHezuDD1l8DPGWI405Pckc3DHSibaskFyX5Qfe65VrULkmahWEe4Hp+VR0O/KKq3g/sBTxtiOPO4NETvB0LXFxVOwEXd+uSpBEaJvh/3b3em+TJwAPAtms6qKouA+6a0nwgg+cC6F4PGrJOSdI6Mkwf/1eSPAn4CLCcwYie02Z5vW2q6rZu+efANtPtmGQxsBhg/vz5s7ycJGmqYUb1fKBbPCfJV4BNqurutb1wVVWSmmH7EmAJwKJFi6bdT5L02Ezb1ZNk9yS/O2n9cOBs4ANJtprl9W5Psm13vm2BO2Z5HknSLM3Ux/8p4H6AJPsAJwCfBe6muxOfhfOBI7rlI4AvzfI8kqRZmqmrZ8Oqmvjj7CHAkqo6h0GXz9VrOnGSzwP7Alsn+RnwPgb/eZyd5EjgFgZz/UuSRmjG4E+yUVU9COxP94fWIY4DoKoOnWbT/o+hPknSOjZTgH8e+EaSOxkM6bwcIMnvM+jukSSth6YN/qr6UJKLGYzZ/3pVTYys2YDBU7ySpPXQjF02VXXFatq+3185kqS+DfPkriRpDjH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGbDSOiya5GbgHeAh4sKoWjaMOSWrRWIK/86KqunOM15ekJtnVI0mNGVfwF/D1JMuSLF7dDkkWJ1maZOnKlStHXJ4kzV3jCv4XVNWuwEuBtyXZZ+oOVbWkqhZV1aJ58+aNvkJJmqPGEvxVdWv3egdwHrDHOOqQpBaNPPiTbJZki4ll4CXAdaOuQ5JaNY5RPdsA5yWZuP7nqurCMdQhSU0aefBX1U3ALqO+riRpwOGcktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1ZizBn+SAJDcm+WGSY8dRgyS1auTBn2RD4BPAS4GdgUOT7DzqOiSpVeO4498D+GFV3VRV9wP/Ahw4hjokqUkbjeGa2wE/nbT+M+B5U3dKshhY3K2uSnLjCGprwdbAneMu4vEgJ467Ak3D9+gka/k+fcrqGscR/EOpqiXAknHXMdckWVpVi8ZdhzQd36P9G0dXz63A9pPWf69rkySNwDiC/ypgpyQ7JHkC8KfA+WOoQ5KaNPKunqp6MMnRwNeADYHTq+r6UdfRMLvP9Hjne7Rnqapx1yBJGiGf3JWkxhj8ktQYg79BSVaNuwbNfUn+oe+n8pN8NcmTVtN+fJL39Hnt9dnjdhy/1k6SMPgbzsPjrkVtqqo3juAaL+v7GnORd/xzSJIF3eR3nwWuA45LclWSa5O8fzX775vkK5PWP57k9SMsWXNEks2SXJDkmiTXJTkkyaVJFnXbj0zy/SRXJjktyce79jOSnJLkiiQ3de/J05PckOSMSec/NMmK7twnTmq/OcnW3fJfdtf4JvD00f4E1i8G/9yzE/BJ4N0MpsfYA1gI7JZkn3EWpjntAOC/qmqXqnoWcOHEhiRPBo4D9gT2Bp4x5dgtgb0YvGfPB04Gngk8O8nC7vgTgf0YvJd3T3LQ5BMk2Y3BM0ELgZcBu6/z73AOMfjnnluq6grgJd3Xd4DlDP6x7TTOwjSnrQBenOTEJC+sqrsnbdsD+EZV3VVVDwD/OuXYL9dgXPkK4PaqWtF1UV4PLGAQ4pdW1cqqehA4E5h6E/NC4LyqureqfokPhc7IPv6551fda4APV9WnZtj3QR75n/8mvVWlOa2qvp9kVwZ32x9McvFjOPy+7vXhScsT6xsBD6ybKjXBO/6562vAG5JsDpBkuyS/M2WfW4Cdk/xWNzJi/1EXqbmh6465t6r+GfgIsOukzVcBf5hkyyQbAa96jKe/sjt+6+7zPA4FvjFln8uAg5JsmmQL4I9n9Y00wjv+Oaqqvp7kD4D/HAzwYRXwOuCOSfv8NMnZDP4Q/GMG3ULSbDwb+EiShxncob8FOAmgqm5N8jcMAvwu4HvA3dOdaKqquq37pL5LGPwme0FVfWnKPsuTnAVcw+A9ftXaf0tzl1M2SOpdks2ralV3x38egzm6zht3Xa2yq0fSKByf5Gp+89vlF8dcT9O845ekxnjHL0mNMfglqTEGvyQ1xuCXOt08Mc+ftH5UksNnea7Xd2PbJ9Z7n6lSGpZ/3JU6SY4HVlXVSevgXJcC76mqpWt7Lmld845fc16SLyZZluT6JIu7tgOSLO9mk7w4yQLgKODdSa5O8sKJOd2TPCPJlZPOtyDJim75r7oZUK9LsiQDrwYWAWd259p0ykyV0800uSrJh7qarkiyzeh+SmqJwa8WvKGqdmMQxu/oAvU04FVVtQvwmqq6GTgVOLmqFlbV5RMHV9X3gCck2aFrOgQ4q1v+eFXt3s1IuSnwiqr6ArAU+LPuXL+eONcaZprcDLiiq+ky4E3r/kchGfxqwzuSXANcAWwPLAYuq6ofA1TVXUOc42wGgQ+PDP4XJfl29xvAfgymE57JTDNN3g9MfD7CMgYzU0rrnMGvOS3JvsAfAXt1d9LfAa6exanOAg5O8jSgquoHSTZh8NkHr66qZzP4LWJtZjh9oH7zR7eHcC4t9cTg11z3ROAXVXVvkmcw+DCQTYB9JrpukmzV7XsPsMXqTlJVP2IQxsfxm7v9iZC/s5sF9dWTDpnuXMPMNCn1yjsKzXUXAkcluQG4kUF3z0oG3T3nJtmAwWyOLwa+DHwhyYHA21dzrrMYTDm8A0BV/U+S0xjMP/NzHjkj5BnAqUl+zeDTpeiOWeNMk1LfHM4pSY2xq0eSGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMb8H9TJcWeajngrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plot_histogram(result=search_result,\n",
    "                         dimension_name='activation')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also make a landscape-plot of the estimated fitness values for two dimensions of the search-space, here taken to be `learning_rate` and `num_dense_layers`.\n",
    "\n",
    "The Bayesian optimizer works by building a surrogate model of the search-space and then searching this model instead of the real search-space, because it is much faster. The plot shows the last surrogate model built by the Bayesian optimizer where yellow regions are better and blue regions are worse. The black dots show where the optimizer has sampled the search-space and the red star shows the best parameters found.\n",
    "\n",
    "Several things should be noted here. Firstly, this surrogate model of the search-space may not be accurate. It is built from only 40 samples of calls to the `fitness()` function for training a neural network with a given choice of hyper-parameters. The modelled fitness landscape may differ significantly from its true values especially in regions of the search-space with few samples. Secondly, the plot may change each time the hyper-parameter optimization is run because of random noise in the training process of the neural network. Thirdly, this plot shows the effect of changing these two parameters `num_dense_layers` and `learning_rate` when averaged over all other dimensions in the search-space, this is also called a Partial Dependence plot and is a way of visualizing high-dimensional spaces in only 2-dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEPCAYAAAC6Kkg/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de5QcZZ3/8fe3O5PMTAi5MAQjoBHI8HMAgxLQQHRDFFaFg6uyLqK7q7KyuKD4U5F1LyieVdd1158CR5FVj4BGUBQWWVQCuUBIiEwwQDKBIdxCQnCYXCfJXPry/f1RNZmeTs9MV6arLzOf1zl9uuupp6q+XdPT336epy7m7oiIiESRqHQAIiJSe5Q8REQkMiUPERGJTMlDREQiU/IQEZHIlDxERCSy2JOHmb1gZk+a2Tozay0w38zsOjPbZGZPmNlb4o5JRERGZ0KZtnO2u3cOMe89wJzw8Vbg++GziIhUqWrotnofcIsHHgGmmdmsSgclIiJDK0fycOA+M1trZpcWmH808FLO9JawTEREqlQ5uq0WuPtWM5sJLDGzp9z9wagrCRNPf/I5DaBx+kSmzmosYaiye9t+9u/sOzAd5z5OWPaQl921rYd9OwbinDxjItNm1Re1bJJo292xrZe9O1IHpqfMmEDTaycWua2RL/+THGY/DBVrosB6k2ZDrieTcxmiLAP1Mjm/HzOe8zqs01+WWy/bX+YWri93HWGZW059K/jsec/B6+A53bmbbNe+A+WJKZOZMH16WInBz4Dllw0371DK8suzXrh8UP2ciWxunby/Xf50doT5nveZyK8PeLbw56aLnZ3ufmTBmRHFnjzcfWv43GFmdwJnALnJYytwbM70MWFZ/npuAm4CMDOvq09y/lfeTPPZ6uEqpfZl27jz6lZSPRni3sdTkj2HvGzb0g4WX7WOVE+WuvoEF157Mi2LZha17NTk/kjbenzpdn70uY309WSZWJ/gk19/A6e9c3pRy05JdI9YZ1pi6HgOT/QOsd50gbKhOxK6cr5MurID//Z7spMOvN6Vbcyp0xA8Z4KEvDuTMy8s25sJlu1KDyTtfekgqe5NDax3f7oOgJ7wuTsVPPemgjhS6eSBuum+4PXeNU/R+f3b8L4UNrGOGR+7kMktJwNgqSDZWHog6STCskT4eyJ39xwoS+WUpQ8uSx6oN/BlnMyd3ztQnkhlC5f3ZXPqZAZe9+S87hv8t7O8aXr6Bk/35U33Dp7Odh/8GfPewp+bJenbXyw44xDEmjzMbDKQcPeu8PW5wFfzqt0NXGFmtxEMlO92923Drbdx+kQljpg0nz2L939zHs+t7uC4+TOrdh+3LJrJxd86lfZVnTSf2VR04jgUcxcdwSXffiNtD++k5azpnPbOybFtSwKNb2mh6VMX0f3kJupPaqbx1BboG3k5KZ+4Wx5HAXda0JyeACx299+Z2WUA7n4jcC/wXmATsB/4+EgrnTqrsWq/1MaC5rNn1cT+bVk0M9akkWvuoiOYu+iIcKr4FlNXtqGo1occrPEtLTS86eRKhyFDiDV5uPtzwNwC5TfmvHbg8jjjEBGR0qqGQ3VFRKTGKHlIxfQPvIpI7VHyEBGRyJQ8REQkMiUPERGJTMlDJCKN1Ygoecg4lHu2tEi+bN3Ql3mRAUoeIiISmZKHiIhEpuQhFaXxA5HapOQhIiKRKXmISEXVTciMXEmqjpKHiIhEpuQhIiOKegMtqSybNGnkSqOk5CEiIpEpeYiMM7m3sB3qNrciI1HyEBGRyJQ8REQksrIkDzNLmtkfzeyeAvM+Zmavmtm68PF35YhJqodOFBSpPbHewzzHlcBG4PAh5t/u7leUKRYRERml2FseZnYMcB7ww7i3JSIi5VGObqvvAF8EssPU+aCZPWFmd5jZsWWISURERiHW5GFm5wMd7r52mGq/AWa7+5uAJcDNQ6zrUjNrNbPWfTt1eKGISCXF3fI4C7jAzF4AbgMWmdlPcyu4+3Z3788GPwROK7Qid7/J3ee5+7zJ0+M/e1JERIYWa/Jw9y+5+zHuPhu4CFjq7h/NrWNms3ImLyAYWBcRkSIkGhoqst1yHW01iJl9FWh197uBz5jZBUAa2AF8rBIxiYhI8cqWPNx9ObA8fH1NTvmXgC+VKw6pTl2ZeqYkeyodhogUSWeYi4hIZEoeIiISmZKHjEu7M42VDkGkpil5iIhIZEoeIiISmZKHiIhEpuQhIiKRKXmIiEhkSh4iImWUnViRC3uUnJKHVA3dUVCkdih5iIhIZEoeIiISmZKHiIhEpuQhIiKRKXmIiEhkSh4iIhKZkoeIiESm5CEiIpEpeUhV0YmCIrWhLMnDzJJm9kczu6fAvElmdruZbTKzNWY2uxwxiYjIoSvXRVauBDYChxeYdwmw091PMLOLgG8Cf1XKjbcv28Zzqzs4bv5Mms+eVcpVS4W0Le2gfVUnzWc20bJoZsnW+/jS7bQ9vJOWs6Yzd9ERJVtvsVYu2cejK/dz+oJGFpwzuezbl+Js37aBXa8+w7Qj5zBzRktZt93R+wKdqS001R3DzEmzy7rtXLG3PMzsGOA84IdDVHkfcHP4+g7gnWZmpdp++7Jt3Hl1K623Pc+dV7fSvmxbqVYtFdK2tIPFV61j9c83s/iqdbQt7SjJeh9fup0ffW4jK372Mj/63EYeX7q9JOst1sol+7j2yj9x5y17uPbKP7Fyyb6ybl+Ks+PlDTy9djHbnl/F02sX09nRVrZtd3Q/x7quB9jcs4F1XQ/Q0ftC2badrxzdVt8Bvghkh5h/NPASgLungd3AQT/5zOxSM2s1s9Z9O3uL3vhzqztI9WQASPVkeG51ab5opHLaV3WS6gk+TqmeLO2rOkuy3raHd9IXrrevJ0vbwzuHrBvH2MyjK/fT2+0A9HY7j67cX/JtyOjt+lM72UwKgGwmxc7O9rJtu7N3M1nSwbZJ05naUrZt54s1eZjZ+UCHu68d7brc/SZ3n+fu8yZPn1T0csfNn0ldfRKAuvokx80vXReHVEbzmU3U1Qcf3br6BM1nNpVkvS1nTWdiuN6J9QlazppekvUW6/QFjUxqCBrdkxqM0xc0lnX7UpxpRzWTSNYBkEjWMb2puWzbbpr0OhLhaEOCCTTVHVO2beeLe8zjLOACM3svUA8cbmY/dfeP5tTZChwLbDGzCcBUoGT9Bc1nz+L935ynMY8xpGXRTC7+1qklH/OYu+gILvn2Gys25rHgnMl8+btH5Y15FN/KlgHZCZBIx7PuGa89iRNPu/jAmEdTGcc8ZjYcx6lT3lkVYx7m7uXZkNlC4Avufn5e+eXAKe5+WThg/gF3/9Bw63rtSdP9ktsWxharVNaUZE9ZtjM1ObpuoWLjnJLoHnb+tMTQcRyeKJw8phT4ZpySGLojoSubzZse+N24Jxu05HdlG3PmNwTPOd1zuzONg8r2ZoLlutIDdfalJwbzUgO9A/vTwa/0nvC5OxU896YGYkilg96BdF/wnE0F78XTA+/J+oLXlgpaZ5YeGBpNhGWJvoH32L+L+ssSqQLzcsqSucumgu/FZO78Xs+Zny1c3pfNqZMZeN0z8DqoN/D3s768v2VP3+Dpvrzp3rxpINt98GfMew/+7CxJ377W3ecdNOMQVOQ8DzP7qpldEE7+CDjCzDYBnwP+sRIxyfjT/2UoItGV7X6I7r4cWB6+vianvAf4y3LFISIio6czzKXq6Cxzkeqn5CEiIpEpeYiISGRKHiIiElnRycPMjjezSeHrhWb2GTObFl9oIiJSraK0PH4FZMzsBOAmghP7FscSlYhIGWTrKh1B7YqSPLLhtafeD1zv7lcBOl1bRGQcipI8Umb2YeBvgf77cihvi4iMQ1GSx8eB+cDX3P15M3sDcGs8YYmISDUr6gxzM0sC/+zuH+kvc/fnCW7cJCIi40xRLQ93zwCvN7OJMccjAugsc5FqF+XaVs8BD5vZ3cCBW5y5+7dLHpXIOLIr2zjslXVFqlGU5PFs+EgAU+IJR0REakHRycPdrwUws0Z3188kEZFxLMoZ5vPNrA14Kpyea2bfiy0yERGpWlEO1f0O8OeEt4h198eBd8QRlIhIqels8tKKdGFEd38pryhTsKKIiIxpUQbMXzKzMwE3szrgSmBjPGGJiEg1i9LyuAy4HDga2AqcGk4PyczqzewPZva4mW0ws2sL1PmYmb1qZuvCx99FeQMiIlJ+UVoennuGeZF6gUXuvjdsraw0s9+6+yN59W539ysirltERCokSsvjETP7pZm9x8ysmAU8sDecrAsfHjVIERGpLlGSRzPBfTz+BnjGzL5uZs0jLWRmSTNbB3QAS9x9TYFqHzSzJ8zsDjM7doj1XGpmrWbWum9nb4SwpVbpEiUi1avo5BG2Ipa4+4eBTxJcmv0PZrbCzOYPs1zG3U8FjgHOMLOT86r8Bpjt7m8ClgA3D7Gem9x9nrvPmzx9UrFhi1RcV7ah0iGIlFyUkwSPMLMrzawV+ALwaaAJ+DxF3FHQ3XcBy4B355Vvd/f+psQPgdOKjUlktHZnGisdgkhNitJttRo4HPgLdz/P3X/t7ml3bwVuLLSAmR3Zf59zM2sAziE8Qz2nTu7dCC9Ah/+KiFS9KEdbnejuBQe73X2o+3rMAm4O7weSAH7h7veY2VeBVne/G/iMmV0ApIEdwMcixCQiIhUQJXk0mdkXgZOAAyOZ7r5oqAXc/QngzQXKr8l5/SXgSxHiEBEpi0wdJFOVjqI6Rem2+hlBl9MbgGuBF4BHY4hpRBkv6khhERGJSZTkcYS7/whIufsKd/8EMGSrI257M5MOPEREpLyidFv1N962mdl5wMvAjNKHFF1uAjksqXNARETiFiV5/JuZTSU4NPd6giOv/m8sUY1CfyJREhERiU+UOwneE77cDZwdTziloyQiIhKfEZOHmV3PMNejcvfPlDSiElOXVm3rytQzJdlT6TCkCvkEx9I6eKZSiml5tMYeRZmoNSIiUhojJg93L3itqXxmdr27f3r0IcVPSUREZHQi3YZ2BGeVcF1locN8RUQOTSmTR03SuSIiItGN++TRT0lERKR4pUweY+KwByUREZGRRU4eZjbUDRC+O8pYqooSiIjI0KLcDOpMM2sjvB+Hmc01s+/1z3f3n5Q+vMpSK0REpLAoLY//B/w5sB3A3R8H3hFHUNVGCUREZLBI3Vbu/lJeUaaEsVQ1JRDJ15WpH7mSyBgVJXm8ZGZnAm5mdWb2BcbZLWPVjVUZ+pIWqT5RksdlwOXA0cBW4NRwetxRAhGR8S7KVXU7gY9EWbmZ1QMPApPCbd3h7l/OqzMJuAU4jWA85a/c/YUo26mEvZlJuryJiIxbUY62+g8zOzzssnrAzF41s4+OsFgvsMjd5xK0VN5tZm/Lq3MJsNPdTyAYlP9mlDdQSXG3QNqXbeN3X3+c9mXbYt2OVK/lS7r5xjU7Wb6kO/Ztrbl/Dz/4ylbW3L+n4Py2pR3c9W9tBT+PL67YzKr/eIStD7140LxXH36Wp69byo7Vz5Q85jjt2ryeFx/5NTu3rC96mc5X22h/+n/ofLUtxsii68hu5bH0gwAnlGqdUbqtznX3PcD5BPcvPwG4argFPLA3nKwLH/mXd38f0H/xxTuAd5pZzZxwGFcCaV+2jTuvbqX1tue58+pWJZBxaOWSffzjp3dw+83Bc5wJZOWSffznlZv531t38J9XbmbtAzsHzX986XYWX7WO1T/ffNDn8cUVm1n2Lw+y8ZdPs+qapYMSyCsrn2fDv93L1rsep/3rv6mZBLJr83qeW/FTXn3qYTat/GlRCaSzo40NG25j69ZH2LDhNl7dXh1Dwh3ZrTyRXcV2tgFMLdV6oySP/i6u84BfuvvuYhYys6SZrQM6gCXuviavytHASwDunia42dQRBdZzqZm1mllr967q6i6KI4E8t7qDVE9wMFuqJ8NzqztKvg2pbo+u3E9Pd/Bbq6fbWf1QfPc1eXTlfnp7gm319jhPrhz879328E5SPVng4M/j1jUvkwk/q5meNNvWbDkw79VHN5PtTQOQ7U2za+0Lsb2HUtr9cjvZTHDn7Wwmxe5t7SMus7OznWw2XCabYvvOTbHGWKzt/gpOtuTrjZI87jGzpwjGJh4wsyOBET/N7p5x91OBY4AzzOzkQwnU3W9y93nuPq9hWvUNWJf6SKzj5s+krj4JQF19kuPmzyzZumWw3ZmhLppQWacvaKS+IWiE1zcY898e31Fnpy9oZFK4rUn1xikLBv9AbTlrOnX1wddF/ufx6Le+lmT4WU3WT2DWW485MO/I019HYlLwuzMxaQLTTpsd23sopamvbSaRrAMgkaxj6qzmEZeZ3tRMIhEuk6jjiOkl6yEalSPsNVgMlzE09yFvEnhwZbMZwG53z4SXKTnc3V+JsPw1wH53/8+cst8DX3H31WY2AXgFONKHCeyolhl+8eJzi4673Eo1kN6+bBvPre7guPkzaT57VknWWavivpvg1OT+Q1qu2LimJIbvcpqWKLz9xx7YxeqHepj/9noWntOQs750gW0M/QXRlc3mTQ8cK7MnG/zoWblkHw8/1MepC6bQsuiooF7OYdKrl+ylfVUnx75tFs1nzzrwY6krXc+LKzazdc3LNJ3+eo5+++vZmxr4IfXiis3sWPsiU958HDPmz6E7FXzB9qYGYkilg+ST7gues6ngvXh64D1ZX/DaUgO92v13EkyEZYm+gffYv4sGlaUGz8stSw6q5+zavJ7dL7cz/ahmph9zMslez5k/sD9zy3dsXc+OHc8wY8YcZk47caB+z+BT4hJ9AwFYX97fsqdv8HRf3nRv3jSQ7T748+W9A99DHdmtbMk+y3a27Xb3aQdVPgRRk8eZwGxyjtJy91uGqX8kkHL3XWbWANwHfDPnfuiY2eXAKe5+mZldBHzA3T80XBzVnjxAN5oqtfGaPA5PFP4cRU0eMDiBFEoeALuyjeH8IFHlJo/+Flp/WW7y6LcvPTGYl5M89qeDZNETPtdK8uiX7J9fRPJI9GVz6gwkjEonj35L0revdfd5B804BEUfqmtmtwLHA+sYOLPcCQ6zHcos4GYzSxJ0kf3C3e8xs68Cre5+N/Aj4FYz2wTsAC6K/jaqjw7lFZGxrOjkAcwDWobrTsrn7k8Aby5Qfk3O6x7gLyPEUTOUQGS8Oqyud1DrQ8aeKKMo64HXxBWIiIjUjigtjyagzcz+QHDyHwDufkHJoxpD1Pooja5MfezjHiJSvCjJ4ytxBTHWKYGIjG/Z+uRBg+a1Lsq1rVaY2euBOe5+f3iobjK+0MYWJRARGUuiXNvqkwSXD/lBWHQ0cFccQY1VuhqviIwVUQbMLwfOAvYAuPszgE57jkgJRETGgijJo9fdD5ydEp4NXvwZhnKAEoiI1LooyWOFmf0T0GBm5wC/BH4TT1giIlLNoiSPfwReBZ4E/h64F/iXOIIaD9T6EJFaFuVoqyzw3+FDSkBHYEm1m5Ls0T3kpaARk4eZPckwYxvu/qaSRjTOKIGISC0qpuVxfvh8efh8a/j8UTRgLiIyLo2YPNz9RQAzO8fdcy9yeLWZPUYwFiKjoNaHiNSaKAPmZmZn5UycGXF5GYYG0EemvncRYNLESkcARLu21SXAj82s//6Uu4BPlD4kERGpdlGOtloLzO1PHu6+O3e+mf2tu99c4vjGFXVfiUitiNzt5O678xNH6MoSxDPuqftKZIBNyI5cqcb5xCgdQNWjlGMWNnIVEREZC0qZPA46bNfMjjWzZWbWZmYbzOyg1omZLTSz3Wa2Lnxck19nvFHrQ0SqXSnbS4VaHmng8+7+mJlNAdaa2RJ3b8ur95C7n19g+XFL4x8iUs1K2fJ4OL/A3be5+2Ph6y5gI8F9QEREpIYV3fIws2nA3wCzc5dz98+Ez1eMsPxs4M3AmgKz55vZ48DLwBfcfUOxcY1lan2ISLWK0m11L/AIwVV1Ix0CYWaHAb8CPuvue/JmPwa83t33mtl7Ce5OOKfAOi4FLgWYMqsxyuZFRKTEoiSPenf/XNQNmFkdQeL4mbv/On9+bjJx93vN7Htm1uTunXn1bgJuAjiqZca4uaaWWh/lsTvTyNTk/kqHIVIzoox53GpmnzSzWWY2o/8x3AJmZsCPgI3u/u0h6rwmrIeZnRHGtD1CXGOejr4SkWoTpeXRB3wL+GcGDst14LhhljkL+GvgSTNbF5b9E/A6AHe/EbgQ+JSZpYFu4CJ3HzctCxGRWhQleXweOCG/O2k47r6SEU4edPcbgBsixDEuqfsq0JWpZ0qyp9JhiIx7UbqtNgHqFBaRcSkzSRfRyBWl5bEPWGdmy4ADP4H7D9WV+Kn1ISLVIkryuCt8iIjIOBflkuy63HoVUOtDRKpBlDPMn6fAxQ/dfbijrUREZAyK0m01L+d1PfCXwLDneUg81PoQkUor+mgrd9+e89jq7t8BzosxNhERqVJRuq3ekjOZIGiJ1OYtsMYAtT5EDpadCIm+SkcxPkT58v8vBsY80sALBF1XIiIyzkQ5SfA9BNepeoDg3h1bgYviCEqKo2teiUilRD3PYxfBJdR1fQgRkXEsSvI4xt3fHVskckg09iEilRCl22qVmZ0SWyQiNagrU1/pEEQKsknxdmtHSR4LgLVm9rSZPWFmT5rZE3EFJjIUfWFXlykT1Is9HkXptnpPbFHIqKjrSqYkovwOFBm9KNe2ejHOQEREpHbo58oYocN2ZSxJ1GUrHYKMQMlDREQiizV5mNmxZrbMzNrMbIOZXVmgjpnZdWa2KRyIf0uhdcnI1PoQkSG5c2QJL2Yb97Wp0sDn3f0xM5tCcLTWEndvy6nzHmBO+Hgr8P3wWURESmQOO9kBbyjV+mJNHu6+DdgWvu4ys43A0UBu8ngfcIu7O/CImU0zs1nhsgV1vbKfZ5dv5fiFR8cZvoxhbUs7aF/VSfOZTbQsmlnpcA7Z/ff1sPLBXs5Z2MC7zy18CPPv7uth+Yoe3vr2ibxriDpRtC/bxto7XiDjCU754PGx/h/uf6yN7ic3UX9SM5NbTo5tO8Xa8fIGdv2pnQmJiWTSvUw7cg4zZ7QcmN/5ahs7djzDjBlzaDqyhUQqU8FoA9O8B8NZkFnPmhKu14Lv7PiZ2WzgQeBkd9+TU34P8O/uvjKcfgC42t1bh1mXT6hP8p5vzFcCKWA8HLY7JXno5xa0Le1g8VXrSPVkqatPcPG3TqVl0UymJvfHFsuURPew86clCm/78EThv+WURJr77+vhyit20dMNDQ3wg+/NOCiB/O6+Hv7+H3bQ3Q31DfDdG6bxrnPr6coO/G7ckx3o7tyVbQSgK9twoKz/vJrdmUbalnbw08+tI5MKBrQTdQnO+48zmbngeAD2pSceWG5vKljv/nQdAD3hc3cqeO5NDcSQSicBSPclB5Zf8xSd378N70thE+to+sRHaJx7EpayA3UsHbxO5JT1X1U3kR48HdQbPC+3LDmo3sD3YjKcv3PLejat/CnZTGqgXrKON869mKaZLXR2tLFx3WKy2RSJRB0nnXQRM6edOFC3ZyCRJPpyAgAsd7on77LAfQUuE9w7uCzbXfjzdVzPK3w/cx9ZgmtKvQNodbeClSMqy4C5mR0G/Ar4bG7iiLiOS82s1cxaAdI9GTY/8kopw5Rxon1VJ6me4Msv1ZOlfVVnhSM6NCsf7KUn/M7o7oblKw5OYstX9ND/vdLTHSwzGu2rOg8kDoBsKhvb/2H3+mfwvuCL2vtSdG9sD17XlecHb77d29oHJQ6AbCbFzs4grp2d7WSzwfxsNsWOHc+UPcZ8z9p0/jXxdvZhJe9mij15mFkdQeL4mbv/ukCVrcCxOdPHhGWDuPtN7j7P3ecBTKhP8rq3vSaOkGueBs6H13xmE3X1wUe/rj5B85lNFY7o0Cx4xyTqwwZCQwMs/LODu6QW/lk9DWGd+oZgmdFoPrOJZN3A10aiLhHb/2HDyXOwiUErxSbW0fDG5li2U6yps5pJJOsGlSWSdUxvCuKa3tRMIhHMTyTqmDFjTtljLGRN4rX8gqMxCtxHfBRi7bYyMwNuBna4+2eHqHMecAXwXoKB8uvc/Yzh1ts4o97fdc3p6rIaxljvuhpNtxUUHvOotW4rOPQxj0PttgJYe//Og8Y8utJBnVJ2W2VTiSHHPPq7rsrZbQWw+/n1w4557Ni6fsgxj0p0W3lv8Pm5Nf0bjmQ/zdC7yb0k1/eJO3ksAB4CngT627r/BLwOwN1vDBPMDcC7gf3Ax4cb7wA4qmWGX7z43NjiHguUPA7NoSSQSiaPgemROxG6stmc14eePPqnc1u4cSUPAE8Hz9Y38B4rlTySvR7Ozx5UFmxr8MmN1ZA8Ep7la5kHuTl5Ctdn7l/b33szWnEfbbUSGHZwJjzK6vI44xiPdL0rkWgyEwcnkLEiawm+NGFhyderM8ylJunKujJm1U8cuU4VUPIQEalmE6szmSh5jGE66qp65I4hiIwFSh4iUhGT6tIjV5KqpeQxxqn1ITL2JRrK37JV8hARkciUPEREJDIlj3FAXVciUmpKHiIiEWXr9NWpPSAiIpEpeYwT6roSkVJS8hARkciUPEREJDIlj3FEXVciUipKHlKzdGVdkcpR8hARkciUPMYZdV2JSCkoeYiISGSxJg8z+7GZdZjZ+iHmLzSz3Wa2LnxcE2c8IiJxyk4cP7/HY72HOfAT4AbglmHqPOTu58cch4iIlFCsadLdHwR2xLkNiU7jHiIyWtXQxppvZo+b2W/N7KRKByMiIiOLu9tqJI8Br3f3vWb2XuAuYE6himZ2KXApwJRZjeWLUERkGJlJRrLXKx1G2VW05eHue9x9b/j6XqDOzJqGqHuTu89z93kN09TtMlrquhKR0aho8jCz15iZha/PCOPZXsmYRERkZLF2W5nZz4GFQJOZbQG+DNQBuPuNwIXAp8wsDXQDF7n7+Gv/iYjUmFiTh7t/eIT5NxAcyisVsDczicOSvZUOQyRW2YmQ6Kt0FGNPNRxtJSJD2JPV2JRUJyUPERGJTMlDREQiU/IY52r9kN1S39Njd0bnEIkUQ8lDREQiU/IQEZHIlDyk5ruuRKT8lDxEZNzJVvqqfmOAkoeIiESm5CEiIpEpebbQIfoAAAikSURBVAigcQ8RiUbJQ0REIlPyEBmlUp+oKFILlDxERCQyJQ85QOMeIlIsJQ8REYlMyUNERCJT8hARkciUPGSQWhz30NFOIuUXa/Iwsx+bWYeZrR9ivpnZdWa2ycyeMLO3xBmPiIiURtyXB/sJcANwyxDz3wPMCR9vBb4fPkuZPLt8K5sfeYWJh9XRtzfF6972Gua+s6nSYR2ytqUdtK/qpPnMJloWzaxoLGsf2MmTK3dzyoKpnPbO6RWNpVJeWfk82/6whQmNk+jdm2LaabNpmNcS2/b2ta2n+6mnaTz+RA6fc3Js24lTR9czdO7eRFPD65k5+YRKhzOkWJOHuz9oZrOHqfI+4BZ3d+ARM5tmZrPcfVuccUng2eVb+e2XVpPuyRwo2/A/z9PwzXk0nz2rgpEdmralHSy+ah2pniytd27h4m+dWrEEsvaBndzw2Wfp68my4o5OrvjO8Sw8Z3x1r72y8nn+eO3vyfSmD5R1/P5JZn9xAlPf1lzy7e1rW8+fFt+Kp1LseewPJD741xx2Ym0lkI6uZ1i39S6ynmbL3vWcynlVm0As+N6OcQNB8rjH3Q/6K5rZPcC/u/vKcPoB4Gp3by1Q91Lg0nDyZKBgV1gJTQV2x7xsMfWGqhOlPL+sf/pYoNC3awfw0ghxRVWO/XlCWLdf/vsYbj3F7s9i9m8T0MDgfVvqfXqo+zPKciPVHWl/Hs7In6+R9m8T0FlMsBz8eS52n5fjs1lM3UL7rAPYM8RyUf7X+53o7lOKjHd47h7rA5gNrB9i3j3AgpzpB4B5RayztQxx3xT3ssXUG6pOlPL8sgLT42J/Dje/2P1Z5P6t2v0ZZblS78/x/Nkczf6s1v/1Sh9ttZXg10K/Y8KyavCbMixbTL2h6kQpzy8bzXs7VNWwP4ebX+z+LGb/lsOhbjPKcqXen+P5s1lM3Zr6X690t9V5wBXAewkGyq9z9zOKWGeru88rcajjlvZnaWl/lo72ZWmVcn/GOmBuZj8HFgJNZrYF+DJQB+DuNwL3EiSOTcB+4ONFrvqmkgc7vml/lpb2Z+loX5ZWyfZn7C0PEREZeyo95iEiIjVIyUNERCJT8hARkcjGXPIws4SZfc3Mrjezv610PLXOzBaa2UNmdqOZLax0PLXOzCabWauZnV/pWGqdmb0x/FzeYWafqnQ8tc7M/sLM/tvMbjezc0eqX1XJY6gLKZrZu83s6fACiv84wmreR3C+SArYElestaBE+9OBvUA943h/lmhfAlwN/CKeKGtHKfanu29098uADwFnxRlvtSvR/rzL3T8JXAb81YjbrKajrczsHQRfVLf0nxdiZkmgHTiH4MvrUeDDQBL4Rt4qPhE+drr7D8zsDne/sFzxV5sS7c9Od8+a2VHAt939I+WKv5qUaF/OBY4gSMSd7n5PeaKvPqXYn+7eYWYXAJ8CbnX3xeWKv9qUan+Gy/0X8DN3f2y4bcZ9Vd1IvPCFFM8ANrn7cwBmdhvwPnf/BnBQ0z88n6QvnMzkzx9PSrE/c+wEau9mHyVSos/mQmAy0AJ0m9m97p6NM+5qVarPprvfDdxtZv8LjNvkUaLPpwH/Dvx2pMQBVZY8hnA0gy9utoXhL9v+a+B6M3s78GCcgdWoSPvTzD4A/DkwjeDy+jIg0r50938GMLOPEbboYo2u9kT9bC4EPkDwo+beWCOrTVG/Oz8NvAuYamYnhCdyD6kWkkck7r4fuKTScYwV7v5rgoQsJeLuP6l0DGOBuy8Hllc4jDHD3a8Driu2flUNmA+hmi+eWIu0P0tH+7K0tD9LK9b9WQvJ41Fgjpm9wcwmAhcBd1c4plqm/Vk62pelpf1ZWrHuz6pKHuGFFFcDJ5rZFjO7xN3TBFfe/T2wEfiFu2+oZJy1QvuzdLQvS0v7s7QqsT+r6lBdERGpDVXV8hARkdqg5CEiIpEpeYiISGRKHiIiEpmSh4iIRKbkISIikSl5iIhIZEoeMqaY2d4ybOMyM/ubuLczxLY/ZmavrcS2RXLpJEEZU8xsr7sfVoL1JN29Ipf0H27bZrYc+IK7t5Y3KpHB1PKQMcvMrjKzR83sCTO7Nqf8LjNba2YbzOzSnPK9ZvZfZvY4MD+c/pqZPW5mj4Q3xMLMvmJmXwhfLzezb5rZH8ysPbwVAGbWaGa/MLM2M7vTzNaY2bxhYs3f9jVh7OvN7CYLXAjMA35mZuvMrMHMTjOzFeH7+b2ZzYpnb4oMpuQhY5IF92CeQ3BDnFOB08K7rUFw17TTCL6IP2NmR4Tlk4E17j7X3VeG04+4+1yCe8N8cojNTXD3M4DPAl8Oy/6B4I6WLcC/AqeNEHL+tm9w99PDu8I1AOe7+x1AK/ARdz8VSAPXAxeG7+fHwNeK20MiozPm7uchEjo3fPwxnD6MIJk8SJAw3h+WHxuWbye48+SvctbRB/TfKnYtwe08C/l1Tp3Z4esFwHcB3H29mT0xQrz52z7bzL4INAIzgA3Ab/KWORE4GVgS3ASOJLBthO2IlISSh4xVBnzD3X8wqDC4+9y7gPnuvj8cQ6gPZ/fkjTWkfGBQMMPQ/y+9RdQZyYFtm1k98D1gnru/ZGZfyYlx0NsBNrj7/EPcpsghU7eVjFW/Bz5hZocBmNnRZjYTmErQnbTfzP4P8LaYtv8w8KFw2y3AKRGW7U8UnWH8F+bM6wKmhK+fBo40s/nhdurM7KRRRS1SJLU8ZExy9/vM7I3A6rBLZy/wUeB3wGVmtpHgy/eRmEL4HnCzmbUBTxF0O+0uZkF332Vm/w2sB14huKlPv58AN5pZNzCfILFcZ2ZTCf6fvxNuSyRWOlRXJAZmlgTq3L3HzI4H7gdOdPe+CocmUhJqeYjEoxFYZmZ1BGMT/6DEIWOJWh4iZWRma4BJecV/7e5PViIekUOl5CEiIpHpaCsREYlMyUNERCJT8hARkciUPEREJDIlDxERiez/A0c1qzlH/2FGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plot_objective_2D(result=search_result,\n",
    "                        dimension_name1='learning_rate',\n",
    "                        dimension_name2='num_dense_layers',\n",
    "                        levels=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We cannot make a landscape plot for the `activation` hyper-parameter because it is a categorical variable that can be one of two strings `relu` or `sigmoid`. How this is encoded depends on the Bayesian optimizer, for example, whether it is using Gaussian Processes or Random Forests. But it cannot currently be plotted using the built-in functions of `skopt`.\n",
    "\n",
    "Instead we only want to use the real- and integer-valued dimensions of the search-space which we identify by their names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_names = ['learning_rate', 'num_dense_nodes', 'num_dense_layers']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then make a matrix-plot of all combinations of these dimensions.\n",
    "\n",
    "The diagonal shows the influence of a single dimension on the fitness. This is a so-called Partial Dependence plot for that dimension. It shows how the approximated fitness value changes with different values in that dimension.\n",
    "\n",
    "The plots below the diagonal show the Partial Dependence for two dimensions. This shows how the approximated fitness value changes when we are varying two dimensions simultaneously.\n",
    "\n",
    "These Partial Dependence plots are only approximations of the modelled fitness function - which in turn is only an approximation of the true fitness function in `fitness()`. This may be a bit difficult to understand. For example, the Partial Dependence is calculated by fixing one value for the `learning_rate` and then taking a large number of random samples for the remaining dimensions in the search-space. The estimated fitness for all these points is then averaged. This process is then repeated for other values of the `learning_rate` to show how it affects the fitness on average. A similar procedure is done for the plots that show the Partial Dependence plots for two dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAGECAYAAAAodGdgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeXxcdbn48c8z2bemaZLuTVPaspUu0LAvsonsiyIuIItKQVEQVOCKXou/670IKqJevRRQAUFBQCiILLLKTveFAi3d9yZt2izNMjPP749zJpmks5xkZpLJ5Hm/XmlmzpzlO0l6nvluz1dUFWOMMSYaX38XwBhjTHqzQGGMMSYmCxTGGGNiskBhjDEmJgsUxhhjYrJAYYwxJiYLFMYYY2KyQBGBiDT2wTWuFpFLU32dKNe+XERG98e1jTEDj9iEu32JSKOqFifhPFmqGkhGmZJ5bRF5Ffieqs7r21IZYwYiq1HEISLfF5H3RWSJiNwatv1JEZkvIstFZFbY9kYR+YWILAaOdp//VEQWi8g7IjLC3W+2iHzPffyqiPxMRN4TkY9F5Hh3e6GIPCoiH4jI30XkXRGpiVHW7tf+T7fsy0RkjjguBGqAh0RkkYgUiMhMEXnNfT/Pi8io1Pw0jTEDkQWKGETkNGAycAQwA5gpIie4L39VVWfi3HSvFZFyd3sR8K6qTlfVN9zn76jqdOB14Mool8tW1SOA7wA/drd9E9ilqgcDPwJmxily92v/VlUPV9VDgALgbFV9DJgHXKyqMwA/8BvgQvf9/AH4qbefkDFmMMju7wKkudPcr4Xu82KcwPE6TnC4wN0+zt1eBwSAx8PO0QY84z6eD3w6yrWeCNun2n18HHAXgKouE5Elccrb/doniciNQCEwDFgOPN3tmAOAQ4AXRQQgC9gS5zrGmEHEAkVsAvyPqt7dZaPIicCpwNGq2uy2+ee7L7d06xto186OoADRf+atHvaJp+PaIpIP/A6oUdUNIjI7rIxd3g6wXFWP7uU1jTEZzpqeYnse+KqIFAOIyBgRGQ6U4jQJNYvIgcBRKbr+m8BF7rUPBqb24NhQUKh1y39h2GsNQIn7+COgUkSOdq+TIyJTEiq1MSajWI0iBlV9QUQOAt52m2UagUuA54CrRWQFzo32nRQV4XfA/SLyAfAhTtPRbi8Hqmq9iNwDLAO2Au+Hvfwn4P9EZC9wNE4Q+bWIlOL8TfzKvZYxxtjw2HQmIllAjqq2iMhE4F/AAara1s9FM8YMIlajSG+FwCsikoPTl/BNCxLGmL5mNYoBSETeBfK6bf6Kqi7tj/IYYzKbp0AhIgVAlap+lPoiJaaiokKrq6v7uxgGmD9/fq2qVvZ3OYwxiYnb9CQi5wA/B3KBCSIyA/iJqp6b6sL1RnV1NfPmWWaKdCAi6/q7DMaYxHkZHjsbZ2ZyPYCqLgImpLBMxhhj0oiXQNGuqt2HZFrHhjHGDBJeRj0tF5EvA1kiMhm4FngrtcUyxhiTLrzUKL4NTMFJMfEwzoSv76SyUMYYY9JH3BqFqjYDt7hfxhhjBpm4NQoReVFEhoY9LxOR51NbLGOMMenCS9NTharWh56o6i5geOqKZIwxJp14CRRBEakKPRGR8SQ46klEhrk1lZXu97Io+1WJyAsissJd5a06kesaY4zpOS+B4hbgDRF5UET+jLNoz38keN2bgZdUdTLwkvs8kgeAO1T1IJy5HNsTvK4xxpge8tKZ/ZyIHEbnmgvfUdXaBK97HnCi+/h+4FXgpvAd3PUXslX1RbccjQle0xhjTC94XbgoD9gJ7AEODls3urdGqGpouc2twIgI++wP1IvIEyKyUETucNNuG2OM6UNecj39DPgCzkI2QXez4jRBxTruX8DICC91GWarqioikfo8soHjgUOB9cAjwOXAfRGuNQuYBVBVVdX9ZWOMMQnwMjP7fJzFclrj7hlGVU+N9pqIbBORUaq6RURGEbnvYSOwSFVXu8c8idP8tU+gUNU5wByAmpoaSy9ijDFJ5KXpaTWQk+TrzgUucx9fBjwVYZ/3gaEiEkpTfTLwQZLLYYwxJg4vNYpmYJGIvISTxgMAVb02geveBjwqIl8D1gEXAYhIDXC1qn5dVQMi8j3gJXEWrJ4P3JPANY0xxvSCl0Ax1/1KGlWtA06JsH0e8PWw5y8C05J5bWOMMT3jZXjs/QNphTtjjDHJ5SXX0znAIuA59/kMEUlqDcMYY0z66u0Kd/ulsEzGGGPSSG9XuAtG3NMYY0zGsRXujDHGxNTTFe7+gpPGw1a4M8aYQcJWuDPGGBNT1EAhIk8TY90JVT03JSUyxhiTVmLVKH7ufv8sTnK/P7vPvwRsS2WhjDHGpI+ogUJVXwMQkV+oak3YS0+LyLyUl8wYY0xa8NKZXSQiHfMmRGQCUJS6IhljjEknXobHXg+8KiKrAQHGA1eltFTGGGPShtelUCcDB7qbPuzp2hTGGGMGLi81CoCZQLW7/3QRQVUfSFmpjDHGpA0vS6E+CEzESQwYcDcrYIHCGGMGAS81ihrgYFW1JUaNMWYQ8jLqaRnOPApjjDGDkJcaRQXwgYi8R9elUG1mtjHGDAJeAsXsZF9URIYBj+B0kK8FLlLVXd32OQm4M2zTgcAXVfXJZJfHGGNMdHGbntwZ2muBHPfx+8CCBK97M/CSqk4GXnKfd7/uK6o6Q1VnACcDzcALCV7XGGNMD3lZCvVK4DHgbnfTGCDRT/XnAfe7j+8Hzo+z/4XAP91MtsYYY/qQl87sa4BjcdahQFVXAsMTvO4IVd3iPt4KjIiz/xdx1sKISERmicg8EZm3Y8eOBItmjDEmnJc+ilZVbRMRAEQkmxjpx0NE5F9EHi3VZV0LVVURiXo+ERkFTAWej7aPqs4B5gDU1NTYMF5jjEkiL4HiNRH5AVAgIp8Gvgk8He8gVT012msisk1ERqnqFjcQbI9xqouAv6tqu4eyGmOMSTIvTU83AzuApTjJAJ8FfpjgdecCl7mPLwOeirHvl4jR7GSMMSa1vCQFDIrI/cC7OE1OHyVhlvZtwKMi8jVgHU6tARGpAa5W1a+7z6uBccBrCV7PGGNML3nJ9XQW8H/AJzhpxieIyFWq+s/eXlRV64BTImyfB3w97PlanFFWxhhj+omXPopfACep6ioAEZkI/APodaAwxhgzcHjpo2gIBQnXaqAhReUxxhiTZrzUKOaJyLPAozh9FJ8H3heRzwKo6hMpLJ8xxph+5iVQ5APbgE+5z3cABcA5OIHDAoUxxmQwL6OeruiLghhjjElPXnI97S8iL4nIMvf5NBFJdB6FMcaYAcJLZ/Y9wH8A7QCqugQn95IxxphBwEugKFTV97pt86eiMMYYY9KPl0BR686dUAARuRDYEvsQY4wxmcLLqKdrcDKzHigim4A1wMUpLZUxxpi04WXU02rgVBEpAnyqapPtjDFmEIkZKETkAGAWznrVACtEZI6qfpzykhljjEkLUfsoRORo4FWcdB1zcEY/NQGvishRfVI6Y4wx/S5WjeI/gS+p6qth254UkZeBHwNnpLJgxhhj0kOsUU8TuwUJAFT1NWC/lJXIGGNMWokVKGJ1WjcluyDGGGPSU6ymp3Ei8usI2wVbTMgYYwaNWIHi+zFem5fIRUVkGPAIUA2sBS5S1V0R9rsdOAun5vMicF0SlmE1CWppD1Df3M6elnYaWvw0tfppbvPT0h6k1R+gzR+kPWC/JmMyRdRAoar3p/C6NwMvqeptInKz+/ym8B1E5BjgWGCau+kNnFTnr6awXINac5ufzfV72VTfwub6vWzZ3cL2PS1sb2hlR0MrdY2t1DW10eoP9ndRjTF9yMvM7FQ4DzjRfXw/zs3/pm77KM5aGLk4zV05OOtimF5SVXY1t7Omtom1tU2sq2tibV0z63c2s2FnM3VNbV32F4HyojyGl+RRWZLH5BHFlBflUlaUS2lBDkPycyjJz6Y4L5uC3CwKcrLIy8kiN8tHTpZQ9rN+eqPGmKTqr0AxQlVD+aK2AiO676Cqb4vIKzh5pQT4raqu6MMyDlhNrX7W1jWxtraZNbWNrN7RxOraJtbUNrF7b3vHfj6B0UMLqBpWyGlTRjC2rJCxZQWMGVrAyNJ8RgzJJyfLSzowY0wmS1mgEJF/ASMjvHRL+BNVVRHZp0FbRCYBBwFj3U0visjxqvrvCPvOwplBTlVVVaJF7zeqSntACQSV9mCQQMD9HlT8AcUfVPyBYMc+bYEgW3e3uEGhiXV1zaypa2JHQ2uX844qzWdCRRFnTxvFhIoiJlQUUV1RxLiyQnKzLRAYY2KLGihE5De4GWMjUdVrY51YVU+Nce5tIjJKVbeIyChge4TdLgDeUdVG95h/AkcD+wQKVZ2DM3ucQ6Yfpss373Zutu4N1R8IOjfZYLDjhtse6HYD7njN3TcQfpx2ey3YdVvHOcJfC3aUIfycgbBrd3x39w8m0P9bWZLHhPIiTty/kuqKIqrLi6iuKGRCRRGFuf1VcTTGZIJYd5CERjbFMRe4DLjN/f5UhH3WA1eKyP/gND19CvhVvBOv3N7AWb9+I2kFzfYJ2VlCjs9HVpaQ7fN1bHO+dz7P8vnIcR8X5mZ37uMem+Nz98kKHe8cmxU6v0/IyercJ8s9v3OckJPVdZ/sLGF4SR7V5UUU5VkwMMakhvTHaFMRKQceBaqAdTjDY3eKSA1wtap+XUSygN8BJ+DUbJ5T1RvinXvylOn6m0ee77jR7nNz9XXepPfZpyMYOF9ZPkFEUviTyGwiMl9Va/q7HMaYxMQNFCJSiTMi6WCcUUgAqOrJqS1a79TU1Oi8eamsDBmvLFAYkxm89GQ+BKwAJgC34kyQez+FZTLGGJNGvASKclW9D2hX1ddU9atAWtYmjDHGJJ+XHtDQwPstInIWsBkYlroiGWOMSSdeAsV/iUgp8F3gN8AQ4PqUlsoYY0za8LJm9jPuw93ASaktjjHGmHQTa8Ldjap6e7SJd/Em3BljjMkMsWoUobxKNtbUGGMGsVhpxp92Hzar6t/CXxORz6e0VMYYY9KGl+Gx/+FxmzHGmAwUq4/iDOBMYEy3JVGHAP5UF8wYY0x6iNVHsRmnf+JcYH7Y9gZseKwxxgwasfooFovIMuAzKV4W1RhjTBqL2UehqgFgnIjk9lF5jDHGpBkvM7PXAG+KyFygKbRRVX+ZslIZY4xJG14CxSfulw8oSW1xjDHGpBsvKTxu7YuCGGOMSU9xA4W7cNGNwBQGwMJFxhhjksvrwkUfksSFi0RkmIi8KCIr3e9lUfb7mYgsc7++kMg1jTHG9E5/LVx0M/CSqk4GXnKfd+GufXEYMAM4EvieiAxJ8LrGGGN6yEug6LJwkYgcSuILF50HhOZm3A+cH2Gfg4HXVdWvqk3AEuD0BK9rjDGmh7wEivCFi74H3EviM7NHqOoW9/FWYESEfRYDp4tIoYhU4KyFMS7B6xpjjOmhWLme8oGrgUnAGOA+VfW8cJGI/AsYGeGlW8KfqKqKSKT1Ll4QkcOBt4AdwNtAIMq1ZgGzAKqqqrwW0RhjjAeius892nlB5BGcZqd/A2cA61T1uqRcVOQj4ERV3SIio4BXVfWAOMc8DPxZVZ+NtV9NTY3Om2dLaKQDEZmvqjX9XQ5jTGJiDY89WFWnAojIfcB7SbzuXOAy4Db3+1PddxCRLGCoqtaJyDRgGvBCEstgjDHGg1iBItSJjar6RSSZ170NeFREvgasAy4CEJEa4GpV/TqQA/zbve4e4BJVtfTmxhjTx2I1PQXozO0kQAHQ7D5WVU3Loaoi0gB81N/lSIIKoLa/C5GgA1TV0r4YM8DFSjOe1ZcFSaKPMqFdXETmDfT3ISLWWWRMBvAyPNYYY8wgZoHCGGNMTJkYKOb0dwGSJBPeRya8B2MGvaid2cYYYwxkZo3CGGNMElmgMMYYE5MFCmN6QESqRWRZf5ejt0Rktoh8r7/LYQYWCxTGGGNiskBhBiT3k/0KEblHRJaLyAsiUiAir7qpYBCRChFZ6z6+XESedFdUXCsi3xKRG0RkoYi8IyJR11gRkZkislhEFgPXhG3PEpE7ROR9EVkiIle52090y/GYiHwoIg+Jm4tGRG4TkQ/c/X/ubqsUkcfd87wvIsfGKMtsEfmDe/7VInJt2Gs3hK0I+Z2w7beIyMci8gZwQNj2iSLynIjMF5F/i8iB7vbPu+dYLCKv9/BXYzKRqtqXfQ24L6Aa8AMz3OePApcArwI17rYKYK37+HJgFVACVAK7cfKKAdwJfCfGtZYAJ7iP7wCWuY9nAT90H+cB83CWDD7RPf9YnA9jbwPHAeU46WVCow2Hut8fBo5zH1cBK2KUZTZO6v089/3V4eRFmwksBYqAYmA5cGjY9kJgiPsz+J57rpeAye7jI4GX3cdLgTHhZbSvwf0VKymgMelujaouch/Pxwkesbyiqg1Ag4jsBp52ty/FyU68DxEZinOzDH2yfhAn7T7AacA0EbnQfV4KTAbagPdUdaN7jkVu2d4BWoD7ROQZ4Bn3uFOBg8MSbw4RkWJVbYzyPv6hqq1Aq4hsx1n46zjg7+qsBomIPAEcjxOo/q6qze72ue73YuAY4G9h181zv78J/ElEHgWeiFIGM4hYoDADWWvY4wBO4ko/nU2q+TH2D4Y9D9K7/wsCfFtVn++yUeTECGXLVicL8xHAKcCFwLdw1p/3AUepaovH6+5z7l6U3QfUq+qM7i+o6tUiciRwFjBfRGaqal0vrmEyhPVRmEyzFqe5BZybcUJUtR6oF5Hj3E0Xh738PPANEckBEJH9RaQo2rncT/Gl6iy+dT0w3X3pBeDbYfvtc/P24N/A+e7SwUXABe62193tBSJSApzjvq89wBoR+bx7TRGR6e7jiar6rqr+J87qkrYE8SBnNQqTaX6Os9bJLOAfSTrnFcAf3CV7wxfPuhenSWmB21m9Azg/xnlKgKfcZYYFuMHdfi3wvyKyBOf/5Os4yxB7pqoLRORPdC4wdq+qLoSO1SoXA9uB98MOuxj4vYj8EKef46/ufneIyGS3jC+528wgZik8jDHGxGRNT8YYY2KypidjXCLyv0D3OQx3qeof+6EsVwDXddv8pqpeE2l/Y1Ip45qeCgsLde/evRTnVdLYuqNje0n+CLys+62qNLRs6/Fx0QRzskCVxobN0ctcNgZECIbCtipl2UG2bNkCQM640QC0b+g8x/ADhwKw/cP6jm3VBxfg83UtazCorP1gb8fz8Qfms+7DlqjPI50j4vvqdt5Ix61e1lyrqpVxT+ZRRUWFVldXJ+dkmzfD6NHJOdcgM3/+/KT+Xk36y7gaRXl5OTt2NFDzqe+y5KO/sHPdYoaNn86kEy71fI5Vrz/AznWLGbL/DKrOinxc85BWfHl5EV+LZOtfHqBxqTPkv7DGGbLfPG8JhTXTqJx1yT77X+cv4oZvXkPJMVMY/d2LANj8i0dpeGs5I0+azIzZZzC1dDNzb3yHj17YyAGnjeXc24/qco7Di1cDcOe1q3n72XqOPnMo1/96v7jPvfrRN7dHvTbAHTMeW+f5ZB5UV1czb16SVle1QNFrIpLU36tJfxlXo6iasJ+WXvBtWtzPO8HWnt3QQ2IdF7rpR7vJRz1nizP83Zef1/E89Dhcfvlevu2bzF1Ny/AVdL4+oaIOf3Mb2YW5Hdumlm6mrdlPbmHsmN99n3jPvYp13B0zHpuvSVz3u6amRpMWKJ5+Gs45JznnGmREJKm/V5P+Mq5GgU9oOMgftiGLAP6ou0cX+bhgS2tHzaB53hJyv1Pf5WbuzV4mVMSev5S7ZwITxzUCXSfnhgcJgKW73U/Fuz1ctvs+8Z571dvj+tO554IqLe0BVm1v5JAxpf1dImPSVlIChYiUAeNUdUkyzpdYYZT88r3x9/Mg2s28/aTJbH1lJSNPmhzxZt4f1tSW93cRBqSnF2/mpseX8O4PTqWypOc1T2MGg14HChF5FTjXPcd8YLuIvKmqN8Q8MMV8onE/rSdqxuwz8N94SpdP98m+Ubf5sj2ds6WuIKnXHWzqm9sJKmyq32uBwpgoEqlRlKrqHhH5OvCAqv7YnVna7/rs03Vz31wmVjDIqe2/1sP8HfH3SVt33w1AS3sAgK2798K4of1ZImPSViJ3mWwRGQVcBNySpPIkLKi9H8qaToJ+Hy27IgeI3gaHVN/YA22tZOUOkE/ls2YB0OJ3AsWW3V7z8Rkz+CQSKH6CkxTtTVV9X0T2A1Ymp1i9F6vpaSC14/uyg+SX741Ym2ivcDrZexowWuKMfE8kkKz/xwPs+XhRzCHFaUUEVGltDwKw1QKFMVH1OlCo6t+Av4U9Xw18LhmFSpVk9130ReAJ75jvHjRCASOWngSTeIEkmmBrK3s+dkaC7fl4Ec1f+kKvhiTH09jam9FrsVmNwpj4EunM3h/4PTBCVQ8RkWnAuar6X0krXZrr68DTfTSXl45sL8HEq0jzPnJqs/Hl5VE8dQaNSxdRPHVGSoIEdPYnJFNHjWKPBQpjokmk6eke4PvA3QCqukREHgYGTaDoqe6T5boLDzy5eyYwYUicQFSRrJJ11T1gBfe2svV3T9Hw1vIuM8UBZ3FPYMzsCwjuPdOdU9KQknK1+YPJO9nZZwPQ4remJ2PiSSRQFKrqe93yICW/baCHfChTS6PnVeovsVJtRFLY1N5v7yP8unNvfIeVL2zseN7w1nJO//lCT7O470hyudoCSQwUTzuroHaOempBVRPK62VMpkokzXitiEwEFMBdN3hLUkqVYdqa/Xzk3mw/emEjbc39Hk89CS93yAGnje1Vqo/eEpFZIjJPRObtbW1P3ond9B2tbo2iLRBkZ1Nb8s5vTAZJ5H/8NcAc4EAR2QSsAbwnPkqRINKZ1qIfhX8qzy3M5oDTxnbUKPryRpuI7uU+fXZNn5ddVefg/J2RP2qyJu1T/zPPAF37PbbsbqG8eIAM7zWmDyUy6mk1cKq7Pq9PVVPTMD1AdQ9WE275LOOud/oolnrIjTQ5kBMx4PV1c9S5tx9F2+zeJQxMNgV2NLQyfEh+0s7Z6g8ytDCH+uZ2tu5usZxPxkTQ4//9IhIxRUfoU56q/jLBMmWsWB3ZXvVbbSlNEv+t39mc3EDRHmB8eRH1zfVssZFPxkTUmz6KEverBvgGMMb9uho4LHlFM2ZfG3YlKW+Km16/pT3A2LICsnzCtggjn1Zs2cPGZF3TmAGqx4FCVW9V1VuBscBhqvpdVf0uMBOo8nIOEckSkYUi8oz7fIKIvCsiq0TkERHJdbfnuc9Xua9X97S8JrOsr0tOZmDmzAGcpqfCnCyGl+TtM+muqdXPF+e8w+3PfZScaxozQCUy6mkEED5MpM3d5sV1wIqw5z8D7lTVScAu4Gvu9q8Bu9ztd7r7mUEqx+dj/c4kfbq/6irAqVHk52QxsjSfrXu6BqHH5m9k9952Gw1lBr1EAsUDwHsiMltEbgXeBf4U7yARGQucBdzrPhfgZOAxd5f7gfPdx+e5z3FfP0VsoPuglZvtY0OyAoWr1R8kP8fHqNL8LjWKQFC57401ADSkIHWIMQNJIqOefioi/wSOxxmQcoWqLvRw6K+AG3H6OcCZ21uvqqH/jRtx+jxwv29wr+cXkd3u/rXhJxSRWcAsgIrKCj67Z+Cv0lgWKBrw7+O5JJ8vNzuJNQpA3RXu8rKzGDmkgFc/2tEx6e7FD7axfmczpQU5NLQkcf6GMQNQomMeA0AQJ1DEnTYrImcD21V1voicmOC1O4SPta+ePEGfGJKkdZX70Wf31JAJ7yOZcrN8bGto6WguSsjcubQHlKBCfo6P0oIcmtsC7GnxU1qQw73/Xs3YsgKOmVjOKx8N5IU3jElcr5ueROQ64CGcjEPDgT+LyLfjHHYscK6IrAX+itPkdBcwVERCQWsssMl9vAkY514vGygFUrt8nUlbudmCuqvRJWzmTFrdzLF52U4fBTipPOat3cm8dbv46rETGFqYazUKM+gl0kfxNeBIVf2xqv4ncBRwZawDVPU/VHWsqlYDXwReVtWLgVeAC93dLgOech/PdZ/jvv6yqjuu0Qw6udnOn2tSmp/GjKHFzRybn+PrCBRrapu48fEljBySz0WHj6MkL5uW9iDtycwzZcwAk0jTk+A0PYUE3G29cRPwVxH5L2AhcJ+7/T7gQRFZBezECS4ZId4M6/5MCpgsSe+jyMqiGZLWod1Ro8jJYqQ7ie8nTy9n8+4WHvr6kRTnZVOS7/wXaWjxM6wo8QmTxgxEiQSKPwLvisjf3efn03mDj0tVXwVedR+vBo6IsE8L8PneFG6g32STra05PdJwJCI7S8jL9rG+LjmBIlSjyMv2McINFJt3t/DVYydw7CQnh3tJfg4AjRYozCCWyKinX4rIazj9DuB91FNKpWua8f7U0xTn6axqWGFyZmdfeWVHQsD8nCxys50hssV52dx4+gEduxW7NYo91k9hBrFEP2Iuwkktng0gIlWquj7hUpmk2SfFeZok+OutqmGFrN+ZhM7sOXNoXbcLcGoUAPdeVkNlcV6XEVXhTU/GDFaJLIX6beDHwDY6+ycUmJacovVOtDTjg7WWMVBTnEczemgB89fvSvxEM2fS+sjzAB2BYcrofTPHDnGbngbTyKetu1s6OveNgcRqFNcBB6jqgBiu2t9rVPRnoEqnVOGJGlaUS31zO/5AkOysBAbtLVjQsWhRrDkZg61GsXTjbs757Rs8/a3jmDrWUq4bRyJ3jg2kTfLp9NfTQBVtPYqEZMBvq7zY6VDe1dxOZUliiwyF+ihCTU+RlAyyGsWSTfUArK1rskBhOiQSKFYDr4rIP4DW0EZbj8KkUnmRExzqmloTCxSjRtHi7+zMjqY4z/kv0jhI8j2t3tEEwK5mS4RoOiUy4W498CKQS+caFSUxjzAmQaEhqjsbE7yRbd5Ma9jw2Ghys33kZfsGTdPTJzsaAahL9OdrMkoiw2NvjfW6iPxGVeOl9DCmRyrcpqfaRFN/z55Ny6edSf/x8kaV5OewZ5AFCkutbsKlsnfz2Pi7mJ5K59FbfTFgoLNG0RpnzzhuvZXWk74COCk8YhmSnz0o+iha2gNs3OUMPd5pTU8mzMAfBtPH+upGnYzfRikAACAASURBVIwUHn09G7t7eZOdwgNgaGEuPknOJ97OmdnxahTZg6LpaW1dU2iF2MSb9kxGybhAYTOzHZk0Gztclk8oK8xNvOkJaPEHyMkSsnyxU5SV5A+ONSk+2e50ZI8tK7DObNNFIp3Z8dhKdP1kn9nYzYl/Gk7GOZJlWFFu4p94582jtT0YtzYBzsinwVCjCPVPzBxfRp31UZgwCdcoRKRQVSMl37kr0XOb3ok0G/vw4tU9Osf7jft1PE632kl5cS51TQn2UeDUKOL1T4DT9DQYhseu3tHImKEFjBlawK6mto7V/oxJJIXHMTjrXhcDVSIyHbhKVb8JoKp/SkoJTY+EAsLhvxtOS1M5+UVZOFNeeneelqYAd4TVTqbesZKlOjlp5Y0nfJnbqqoqwJlLsWLrnsROXFND6yOLPNUonKanzA8Un+xoYuLwYoYV5eIPasdqf8YkUqO4E/gMzuJCqOpiETkhKaUyvRZeEwCgsfNhT2oV4ec54LS6jhpFXwYJ6LrMbU1NjYJTo0hKZ7Y/QF4PahSBoMbtzxioVJVPdjRy0fhxnSPLmtosUBggwaYnVd3QrWoaiLav6Xs9bW6Kemya5YoK5XtqDwTJSSDfU2t7gHxPNYrO2dmZeuPcuqeF5rYAE4cXUxYWKCZUFPVzyUw6SCjXk9v8pCKSg5MkcEVyimW88BIIWpoCbvNTgtcpjvzaPjWYPlBeFMr31Mbwkl5mOf3xj2n1Bz3XKMDJ95SpgSKUumNiZVFH2hKbdGdCEgkUV+N0WI8BNgEvANcko1DGm/cb94sZLO68djVvP1vP0WcO5fpfe7uh98eNv6fKi918T40JBIrZs2n5v7c81ihCiQEzt58iNOJpUmVxR1bdXRYojCuRFB61wMVJLIvphWg39rZmP28/uwCAt5+t54gfVqVN01GiwtvQe230aFp/8CjFRfF/JuFNT5nqk+2NlORlU1mSx143q64NkTUhvW7gFZHbRWSIiOSIyEsiskNELklm4UzvhYbIAhmxYFG4UL6nhG5kW7bQ0h7wPOoJMjvV+OraJvarLEJEKMzNJj/HZ5PuTIdE7h6nqeqNInIBsBb4LPA68OdkFKy3oq1wF08mzubOpAWLwg0LpRpPMN9TS3vQ8zwKyOymp7rGNkYP7WzGG1aYaxlkTYdE7iChY88C/qaquwfy5Jz+XgEvJNkBK9OCBMDQgpzE8z0ddhitfq81CudnmMkZZBta2ynJ71wlYFhxrtUoTIdE7iLPiMiHwF7gGyJSCbQkp1iDVyhgRVvhLhNrPj3l8wnDinKpTeQT7/z5tNz6grcaRV7mNz01tvg7AiJAWWGu9VGYDol0Zt8sIrcDu1U1ICJNwHnJK1r6S+VNOxnZYzPZsKJcdiaSxmPWLFpHXBB3LQpw0pBn+yRjm55UlYZugaK8KJe1dU39WCqTThJtlzgQqBaR8PM8kOA5E2LZYyPr65TjqVZelJdY09M999By03kxV7cLERFndnaGBoqW9iD+oFKc1zlHpKwol11NmVuDMj2TSK6nB4GJwCI6Z2Qr/RwojCM8MKRbUr9kGFacy4rNCeZ7AvI81Cggs1ONh95X9xpFY6vfcz+OyWyJfMSsAQ5WDS11YtJFeGA4fXZN15TjGTIKqqIoOW3oXmoUkNmLFzW480PCA0VoZNmupnZGllqgGOwSWY9iGTAyWQUxydF9LQogI+dTDCvKY/deJ99Tb9R++AkQf73skExekyL0vroGCqcZKhnp3M3Al8hdowL4QETeAzr+mlT13GgHiMg4nKapETjNVHNU9S4RGQY8AlTjzMm4SFV3iTPe9i7gTKAZuFxVFyRQ5owXaS2KTJxPUe5OutvV1MbwIT1P46HzFwAFngNFSX4OG3dFWnZl4OtseursowivURiTyJ1jdi+O8QPfVdUFIlICzBeRF4HLgZdU9TYRuRm4GbgJOAOY7H4dCfze/W5iiBQYMilIQGdiwLpeBorKiz8PNz3juelpSCY3PVmNwsTR66YnVX0N59N/jvv4fSDmp31V3RKqEahqA0622TE4w2rvd3e7HzjffXwe8IA63gGGisio3pZ5MMm0wNBdKN9TorOHvdcoMneVu8aOQBGpRmFzKUxiuZ6uBB4D7nY3jQGe7MHx1cChwLvACFXd4r60FadpKnTODWGHbXS3DXrptIZ1f+jIIJvgJ17vndk5NLb6ycSxG3vcpqdQenGA0oIcJNHZ7yZjJPKx8xrgCJwbPaq6UkSGezlQRIqBx4HvqOqe8NQfqqoi0qP/jeHLZVZUVjB98xk9OTwtFbSXRn0f9RubaNnTRv6QXIaOTeeFZR5L2ZlHDHECxYadves3WPmTX0BTDzqz87MJBJXmtgBFeZlVWws1PYUHiiyf2Oxs0yGRv/hWVW0L3eTdSXdxb/DuIkePAw+p6hPu5m0iMkpVt7hNS9vd7ZuAcWGHj3W3dRG+XOZ+k6t18eh/9vIt9UwqJ7FN33wGkd5HW7Ofu87srLhd99b5Gd/MFElJfg6ThxezYH19r45ff+HFcP88Tyk8nOt1JgbMxEBRnJe9zzKvZYU5lu/JAIkFitdE5AdAgYh8Gvgm8HSsA9xRTPcBK1T1l2EvzQUuA25zvz8Vtv1bIvJXnE7s3WFNVP2qvyaxRRrV1FfSJXFiyMzxZTy3fCvBoOLr4VrWpxw80u3M9j7qCZwRQiNLe7lYUppqbG3v0pEdUl6Ul1g+LZMxErnL3Ax8DVgKXAU8C9wb55hjga8AS0VkkbvtBzgB4lER+RqwDrjIfe1ZnKGxq3CGx16RQHl7JNZN0d/c1mWuwsLrK8guzE3q9aMlBQSYcMtnGXd9G9mFuSzdndTLDiiHjS/jr+9vYHVtE5OGR1mrNQ6vNYph7u+3trGNySPi7DzAhGoU3U0aUcxTCzfZ7GyTUFLAIHCP++X1mDeAaB/9Tomwv9LD5VV7ux5FT2QX5jLypMlsfWUlI0+anPQg4cWG5lFO6BzEZo4vA2D+up29DhReb4AjS50+kW17Mi9BcveEgCGnHDich99dz3trdnL85EoAlm8exJ9MBrEeBwoRWUqMvghVnZZQiQaIGbPPwH/jKR1BYk1teVLP3+bLjnrOlrqCpF4rXE7twGl/36+iiKGFOcxft4svHF7Vo2PXH3My4L1GMcKdq7E1IwNFO0MjfNg5ZmIFedk+XlqxvSNQ/PezK/q6eCYN9OaucLb7PfRJ/0H3+yV46MzuK8m+cUeVgk/1LXUFBMt8tOzaNyDk1GaTE+GYcPk7kl+mQFsrWbl5yT9xAkSEmVVlzF+3q8fH/uOnc+C5D3s0M7s4L5utuzMwULT6GTescJ/tBblZHDupgpc+3MaPzzmYVdsbeXNVXT+U0PS3HgcKVV0HICKfVtVDw166SUQW4PRd9JugSt8FiSQJ1RCCLa348iPfjLt/0u9NMCis7V0cX/X6A+xct5hh46cz6YRLe3WO3gof+lxVtW+t4bDxZbz04XZ2NbVRVuS9CfAzN1/Jz078LrlZ3qcSjRiSN6iangBOPnA4L3+4nVXbG3ng7XXkepx3YjJLIr91EZFjw54ck+D5Bq388r3smPNnNlz7I3bMSd2S480V0vHlVaC9lZ3rFgOwc91iAu19m9JBVeeoao2q1lRWVu7zeqifYuGGntUq9nv3FXKzfT0aLTWyND9jm57CZ2WHO+UgZ2rUk4s28fiCjZwzLb1Gvpm+kUiD9NeAP4hIqfu8Hvhq4kUafPy7GmietwSA5nlLCF667824vcLfpVbRUplYE5P3YJHPkP1nsOfjRQwbP52snK41np4EnVSYPnYoWT5h/rpdnHxgz4YjeZ2VHTJiSD7vfJJZTS/tgSAt7UFKoswNGVVawMGjhnD3a6vxB5XLjhnPLyPuaTJZIqOe5gPTQ4FCVbsMhxCRy1T1/ogHmw6bf/EoDW8tJ6uslMCu3RTWTIva/NRdS7cP2KnomwCoOutSAp/+Alm5eWk30KogN4spo4f0qp/Ca/9EyMgh+WxvaO3VvI101RghIWB3pxw0nA+27OHQqqFMGzu0r4pm0kjCTUWqurt7kHBdl+i5M11wbysNby0HILBrN2Pu+CGVsy7p9flaKrt+JVMvO7L7pCny8OphLFhfT12j92axG/660POIp5CRpfn4g0ptBmVU7UjfEaXpCeAzU0YiAl87bkJfFcukmVT+R86Mj1wp0lJXQFvzUAprnNHEhTXTyC4dEnX/SMNW83fE/uoPoSC19l8PgJP0MeW+ePg42vxBHnp3vedjZr7wtx5PIgsNkd2+J/0DRUt7gLc/qeO3L6/k0fc3RM18uyfCMqjdHTKmlLduPpmzrX9i0ErloPm0GSqbLiLNf6icdQnBSyOPdooWHLrrzWimQHvrPv0NvRXeT5G/wxlK27h0EX01l3fyiBJOPKCSB95ey6wT9vPUpHTxH/6bv/z69B5dZ2RoLsXuFg4ZUxpn7/7z/PKtXPuXhbT6O1f/+/Hc5Zx/6Bhmn3twlwAZaS2KSEaVpm7ujkl/qQwUVqNwxZsgFx4kQsFBSrr++LoHiHjBoXBb9JXJli9+mB3bllA5YhpTpn855nm6ax6xbxPFvmXJZdj46ZS6o6X6wpXH78fF977LU4s2eZ5819MaRSjHU1+PfGoPBBEg28NQ3u0NLdz0+BImDS/mhk/vT031MFZtb+Shd9fxl/fWUzO+jM/NHNuxf6imUZIXb3aOGcxSGSjeTOG5B5T88r2AtxnV7RWRmwhaKiFnU+ekt+YKiRksIt3QwalJ7NjmjLDasW0JDcO+mLSaRbiTZpzB2j4MFMdMLOegUUO4999ruKhmHOGp66PpaR9FRXEeWT7p07kUTa1+PnXHq9Q2tlKSl82Bo0p4+MqjyIkQNFSVHzyxlL1tAe764qEdaU1mji/jsKqhLNpQz4PvrOsSKBo8ND0Z0+u/DhEZClyKs851x3lU9Vr3+7cSLVxv+ESZUJH8IYzJmMQXChheiC9A1gENHc9Do6MKa6Z1dHg3RDk2pzabYGsrvrxIASCf4uUzaFy6iOKpM9g9LTmZUCUYpKC9sxZz3JuLWJuUM3u8vghXHj+BGx5dzGsf7+DEA2IvjfKTK/+H/B7WKLJ8QmVxXq9mZ+/e205pQc8/tT+3bCu1ja1cdvR4djW3M3fxZt5YVctJEd7f4ws28a8V2/nhWQftk/tKRLjkyPH85JkPWLZpd0fTmdemJzO4JfLX8SzwDk722GCcffuMD2Vq6eaO58lKEJiK4BNL7p4JTBjiXNPf3MZH7uio5nlLmPz/VsdMRLjoL//sSFg4Y/a+ix9NOvYE/M1HuedYG7csXoKkLxDkysde4+rHXic76Pw5pG7qYGRnTxvNf/1jBU8s2BQ3UCwfNZnyHtYoAEb0YtLdh1v3cPav3+AXF03nvBk9W6Dx8QUbqRpWyOxzp9AeUF5fuYMnF27aJ1AEgsr/PLuCw6vL+OqxkUcnfW7mWG5//kP+/M46bvucM4giVKMotkBhYkjkryNfVW9IWklSJDxoDCSFTe2dZS+FDWFrUBw6qjbqcW3Nfp57ZSUAW19ZyQE/nRJ5zYoe9MWGyhEv6P7jmmmsO66c7/+/5xm+o9H7BZIkN9vHZ6aM5KlFm2hpD8Ts1H7kJxdyw18X9vgaI4fksaa2qUfH/PmddfiDym9eXsU500Z7noOxcVczb6+u4zun7I+IkJstnDV1FI8v2Ehja9fU4Is21FPX1Mat502Jev7SghzOmz6GpxZt5j/OPIjSghwaWv3kZvssjbiJKZHhsQ+KyJUiMkpEhoW+klYy08W5tx/FdW+dH3eRpNDCRkDSFzaaWro57pecILx/cf+Ntz9r6iia2wK8+lH88cF5vahRjByS36Omp6ZWP08u3Mzo0nxWbW/kXyu2eT727ws2oQqfPayzFvLZw8bQ0h7kheVbu+z76kfbyfIJx0+KPYHmK0ePZ297gCcWOOupNLT4GWK1CRNHIoGiDbgDeBuY737NS0ahTGReb/peg0qqTHl5E235WdRB9KpPihy13zCGFeXyj6XRF0IMBJ1BAAU5Pb9BjijNZ0+Ln71tAU/7P714M42tfu78wgzGDSvgd69+grPMSmyqyhMLN3HUfsO6ZHY9rKqMccMK+PvCrisCv/zhdmZWlVFaGLsf5JAxpRw8agj/XOYEGichoI14MrElEii+C0xS1WpVneB+7ZesgpnE9Nc62kO27SVvb4Df/Plk1jqrFfap7Cyn+emlFdtoaY98M1+ysZ6Hp3+GGVU9T0cxsofrUjz83nr2H1HMEROGMeuEiSzaUM87q3fGPW7B+l2sqW3ic4eN7bJdRLhgxhjeXFXLdrcM2/a0sHzzHk46MHa/TMiR+w1jycZ62gNBNyGg1ShMbIkEitDypCaNtDVHHl7bV3yBIL958GS2Teq/CWlnTws1P20HYPueFtoDneMtXv+4llvO+DbHT6ro8bnDJ93Fs2zTbpZs3M2Xj6hCRPj8zLFUFOfxf699EvfYvy/cREFOFmdMHbXPa+cdOoagwl/f3wDAa24z20kHesvbclhVGS3tQT7c0kBjlGVQjQmXSKBoAhaJyN0i8uvQV7IKZnpu7o3vcNcxTzL3xnf6rQz1o4vw5/dvx+iRE5zmp3v+vYav3PcuR/z3S/zihY87Xn995Q5efOiGHq1fETLCnXTXfS5FS3tgnxrMX95bT162jwvcWkF+ThYXzhzLm6tqaYqSUgPAHwjy3LKtnHzQ8Ig38YmVxZw+ZSS/eXklizbU88pH2xlVms8BI0o8vYfDwpaQjbUWhTEhiQSKJ4GfAm/R2UcxPxmFMj3X1uznoxecDsqPXtjY7zWL/pSd5eOMQ0Yyf90uVm1v5MCRJfzlvfW0tAfYvbedRRvqmbTh4/gniiBS01N7IMgX5rzDFX98v2Nbmz/IM0u2cPohI7vMnzhuUgX+oPLemujNT++t2UltYxtnR6hNhNz2uakML8nnWw8v4I2VtZx4wHBPkwwBRpfmM2JIHgvW18dci8KYkETSjKdlCvEg0uu5EwN1KC10jnYKDaHtrz6KdHHj6Qdy5tRRHDlhGO+t3cmX73mXZ5ZsoSg3q6MzuzeK8rIp6bYk6pzXV7N4Qz0Aq7Y3MGl4Cf9euYPde9s5b0bXv8Wa6jJys328uao2ap/CM0u3UJCTFXMuyNDCXH775UP5/P+9jT+onHSA93TBIsJhVWUsWL/LahTGk0RmZq8hQuK/gdyhnazJeckwOZDTUR6vAezc24+ibba/z4NEOv3cQkoLcjjW7YM4er9yJlYW8ed31nHQqBJK8rLRUaN6nYxsRGk+76/dyfY9LdTvbeeuf63khP0reWtVLY+8v4FbzjqYpxZtZmhhDsd1G66an5PFzKoy3oyyAJI/EOT5ZVs55aDhFOTGbsI7tKqM/zznYOa8vrrjvXp1WFVZx8inaIsWGROSyF9ITdjjfODzgM2jSIEe34gjrQ4yiIkIF7vpK1Ztb+TYSeXI5t7XHq88fgI/enI5J//iNSqKcynOz+aXF03nlr8v5YkFm/jWyZN58YNtnH/omIhrTB87qZyfv/AxdY2tlBd3TbPy7pqd1DW1cVaMZqdwlx5dzaVHV/f4PRw2vnPElzU9mXh63UehqnVhX5tU9VfAWUksmzFJ87mZY8nP8dHY6ueE/Sth9uxen+sLh1fxwvUnMHN8GWvrmrn13ClUFOfxxcOrqGtqcxLztQc4d3rkAH+M++n/7dVOrWJz/V6eX76VXU1t/GPpFgpzYzc7JcOU0aXkuokFrenJxJNI09NhYU99ODUM+4szaam0IIdzp4/m0XkbOWFyJRx1a0LBorqiiD9dcTi1jW1Ulji1ghP2r2TkkHz+sXQLI4bkccSEyBXsaWNKKcnL5s1VdXxq/0ouufddVtc2IQJZIpx+yMi4zU6Jys/JYsqYISxcX295nkxcifyF/ILOPgo/Tna5zydaIGNS5abTD+SUg0Z0memcCBHpCBLgZJe9cOZYfvvKKs6eNpqsKDmXsrN8HLnfMN76pJbv/20J63Y2c/uF09hS38LCDbv6bMnRw6rKWLi+3pqeTFyJBIozgM/RNc34F4GfJFgmEyYZ6c2No7w4j89MGZnSa3z5yCreWFXLl46IvXjSMRMr+NeK7ayra+aWMw/ioppxKS1XJIdXD+O+N9ZQUdzz+SRmcEkkUDwJ1AMLgL5d8iuGoEpG3FzbfNkZ8T7S1rzUpCUbPbSAJ685Nu5+x012+ilOnzKSrx/fP0kUPzNlBI9edTRTRqfvsq4mPSQSKMaqas8WHc4Qwb2t+Ap6tiqcl9XtulyjzEfLLlunWERmAbMAqqq8LXE6EOw/ooSHrzySQ8eVeZ4ol2wiErUfxZhwiQSKt0RkqqouTVppBoBIK831pdCa2oOFqs4B5gDU1NT0fqZcdzU14CGLayodM7HnuaaM6Q+J3HWOAy53J961AgKoqk5LSsnSUHBvKw1hK80FL23Fl5/89aa7ixYc8uMvuZCwvC0tXdbUDrS39mSN7URSxBhj0kSindmDiq8gj5JjptDw1nJKjplC4Zgg4KyD3dOmJS8SCRCFtYl/Wl7zwv3s2LaEyhHTmDL9yyxf/HCX57EsX/wwwKEJF8IY0+/EyyIqA4mINAAfpfgyPlK/TngF/bDwTxgfXW/0i4HpYc8XEv1n0HGsqiatAV5EdtAPa1zE0d+/p1hSVbbxquo9uZQZ8DKxwfsjVa2Jv1t6E5F5A/19iEhShxal480pnX9P6Vw2M7BYG7IxxpiYLFAYY4yJKRMDxZz+LkCSZML7yIT3EE86v8d0LpsZQDKuM9sYY0xyZWKNwhhjTBJZoDDGGBOTBQpjzKAnItUisqwfr3+5iPy2v64fjwUKY4zJcCKS0Jw5CxTGmLThfrJfISL3iMhyEXlBRApE5FURqXH3qRCRte7jy0XkSRF5UUTWisi3ROQGEVkoIu+ISNT0uCIyU0QWi8hi4Jqw7VkicoeIvC8iS0TkKnf7iW45HhORD0XkIXFT/4rIbSLygbv/z91tlSLyuHue90Ukfv5557hzRORd9z38S0RGiIhPRFaKSKW7j09EVrnXiHgdEZktIg+KyJvAgyIyRUTeE5FFbjkne/29WKAwxqSbycD/quoUnDVvPhdn/0OAzwKHAz8FmlX1UOBt4NIYx/0R+LaqTu+2/WvAblU93D3nlSISWjTkUOA7wMHAfsCxIlIOXABMcZOi/pe7713Ane55PgfcG+d9hLwBHOW+h78CN6pqEPgzcLG7z6nAYlXdEec6BwOnquqXgKuBu1R1Bs7S1Rs9licjU3gYYwa2Naq6yH08H2cVzVheUdUGoEFEdgNPu9uXAhGzWYvIUGCoqr7ubnqQzkSnpwHTRORC93kpTvBqA95T1Y3uORa5ZXsHZ/G2+0TkGeAZ97hTgYPD1hsZIiLFqtoY5/2MBR4RkVFALrDG3f4H4CngV8BXcQJd1Ou4j+eq6l738dvALSIyFnhCVVfGKUcHq1EYY9JNa9jjAM4HWj+d96v8GPsHw54H6d2HYcGpacxwvyao6gvRyqaqfuAI4DHgbOA593UfTs0gdJ4xHoIEwG+A36rqVOAq3PerqhuAbSJysnu9f3q4TlPopKr6MHAuTsrrZ93zeGKBwhgzEKwFZrqPL4yxnyeqWg/Ui8hx7qaLw15+HviGiOQAiMj+IlIU7Vzup/dSVX0WuJ7OLMsvAN8O22+Gx+KVApvcx5d1e+1enCaov6lqoCfXEZH9gNWq+mucmonntYMGRKBwO6mWup0wqVns2BiTzn6Oc/NeiJM+PRmuAP7XbUIKT4d/L/ABsMAdMns3sWsmJcAzIrIEp3/hBnf7tUCN23H8AU4fgRezgb+JyHz2TRM/Fyims9mpJ9e5CFjmvt9DgAc8lmdgpPBwRzjUqGq65v03xpiUc0d+3amqx/flda0z2xhjBgARuRn4Bl2byfrm2gOkRrEG2AUocLeqWlZMY4wnIvK/QPc5DHep6h8j7Z/islwBXNdt85uqek2k/dPFQAkUY1R1k4gMB17EGZHwetjrs4BZAHl5eTMrKyspKEjaCpwR7d3b+XPzcq3w/XPzfUjYIarQ1hLs+jrZIP4u5whEWVU0GNbVtM+vM/wYdf7R9vaOTb7sHDqaZ7sdK9H+NoLe/mbWbVxbm+iqdOG/29zcvJmjRo5GRcAH6nPK4ZOu5ckSb6vUtrdnof6wn4Uvh45fjCrBYOdrkpsDAtrauS30e9Juv6cslADx/yayuv/AXd2PDWjXrsRo72+f/dRHIMK+WRKEbn9zkpuDqHQubutz/3Qk7GesEGj1s3HjxqQucVtRUaHV1dXJOt2+Nm+G0aNTd/4MMX/+/Kj/XwdEoAgnIrOBRlX9eaTXx40bpzMOreV3vx+a0nJ88xv1PPN0C2efk+/pWuH7f+mX+w42+PV1q3j32Z0ceeYwrr1rEvmfXEHLxK4feN7YE3ki5ZKdnf8JNu/oVpbtnSMJ87c7/7c3/v0BGlYsYuiEGex3Sud8pOKtXW94BRsaIr+Z9Vsib+/m+V33zk/mUpylOcP1iMOvpWFCAY1jfLQd4YwAPGjkti77nVzxoafz3fOHs6j9wx/YVreMkXkTOeSwy2mYUABAyZq9LP34EbbVLWNE+SEU/ugKssc00fT7B9j88ipmnD6cy++cyqi1F7Gl+tEu5z2z+AOebTw47vXPLP4g4vbux75ce6Cn99d9v3Prj2Tu0Hf32S90/J+uX8qi57Yz+uRJ5J37DQo3CcWbnEjROMZH8xgle4wzujL0M375xtdofHtZUgNFTU2NzpuXwjEqFig8EZGo/1/Tvo/CHZbmU9UG9/FpwE+i7V9QICkPEgC/+/1Q7vh5kKIibwPHwvf/9959X7/2rkm0/HeA/KKsJJd0X2MvuJTgWV+gqD4v5ddKd1P3/wJTVx5Jti8Hf4TXDgqcVBw2VAAAIABJREFUT3ZWHp+422pu/QzH3j6GvKK0/68T1+V3TuVTv1rGj3d8hqXzvR0z6oaLWPn5ZQtTW7Ikmz/fAkWCBsJf+wjg7+6sw2zgYVV9LvYhfcNrkPC6f18EiRBfrgWJkGxfTvTXsvb9OWVCkAgpKvLBjh4f5q1tL12ce26ENlnTE336Fy8iE4GNqtoqIifiTPh4wJ38EpGqrqZzAosxxpg+1tcT7h4HAiIyCWc933HAw31cBmOMMT3Q14Ei6OZFuQD4jap+HxjVx2Uwxgwmd9/d3yUY8Po6ULSLyJdw8peEMixGbyA2xphEzZrV3yUY8Po6UFwBHA38VFXXuDneH+zjMhhjBhNJ7ZyqwaDPOrNFJAu4RVU7pp+r6hrgZ31VBmOMMT3XZzUKNyXueBHJ7atrGmOMSVxfDwhfDbwpInPpuqDGL/u4HMaYweLss/u7BANeXweKT9wvH04O90Hn33ur+rsIxgwuTz8dfx8TU58GClW9FUBEClW1uS+vbYxJH+HJHquqUvzh6ZxzLFgkqE9HPYnI0e4KTB+6z6eLyO/6sgzGmP6nqnNUtUZVayorE0owHN8zz8Tfx8TU18NjfwV8BqgDUNXFwAl9XAZjjDE90OdrZqvqhm6bAhF3NMYYkxb6ujN7g4gcA6iI5OCs9LSij8tgjBlMLHNswvq6RnE1cA0wBtgEzHCfG2NMasyxlZMT1deBQlX1YlUdoarDVfUSVa3r4zIYYwaTq67q7xIMeH0dKN4Rkb+JyBkiPUvAIiJZIrJQRGwIgzHG9KG+DhT746xDcSmwUkT+W0T293is9WcYY0w/6NNAoY4XVfVLwJU46cbfE5HXROToaMeJyFjgLODenl6zqSl9Vm2MVpaWJm8Dv9qb25NZnB7xB9ri76P9V77eCLS1Rt7e3rndH4i8T6YKf+8xDKy1YOfO7e8SDHh9vRRqOXAJ8BVgG/BtYC5Op/bfgAlRDv0VcCM9TPvxzW/U88zTLZx9Tj6/+/3QXpc7GUJlOfJMP9feNalj+6+vW8W7z+7kyDOHddnenbPfe0z8dBWn3XZ8XxS5w+K1j7N19weMLD2Y6dWfi7xP40tsbV/DyJwJTC8+pU/L1xt/un4pHz73ErvKD+HQrOM6tq9840F2rl/MiPJDANhWt4wR5YdQyBX9VdQ+s2HuA+z5aBHDqqYz6guXRdxnzdU/h4G2NPHMmf1dggFPtA+HjonIxzjrT/xRVTd2e+0mVd0n5biInA2cqarfdNfZ/p6qnt1tn450AJWVFTMfeuhHBILK8mX+jn2mHJJNlq9/8tJ3L0v1wYWIT9CgsvaD5n22A/haywnmOf383ferOKCsYz+AvYHOhLzt7VldL+7v3E/8Xd+/r1sFwNfe9W/B1xZAURr2bu3YVlIwEmnzd9lPURoCnWMSSrLKEYRZ118+X1VrSED477asdNjMn932WwJ5PoK5QrDQqYkV5HQtT0n23pjn1CBsXNHQ8XxIdgUgaF42DU1bIh6TO3YMkqsU5Pi7nD+nbRjtuTu77Fua1cLuQH7c91aa1RJxe/djG/wFXZ5He3/d9ysNFLE7q2mf/cKPL81qYWNr2f9v79zD4yqr/f/55tIkTUNKaQslBVtL6QWwhUZQLooIiFxFrv5ELHKooHLT/hCPiHAeLyhohXItIgXBIxWFA5xSQG4CQrEtLS0tSgsVWnqj0EvSJk0y6/yx9ySTSWaSNJk9M8n6PM9+Zu93v5f1zjuz117vfvdabKstZseq1c3ppUOrsBJBcWAFlxU3EmtqYsvyjUydOhUz67E/U3V1tc2bN6+nqmuL5EtkO4GklP/XqE3IMZZCM7WnJEIOA06WdDxQCuwi6T4zOyeh7AyCZx/su+/eNnb8zwG45ZYWi+LMs7NrUcRlOeT4QVxySovl8KebEyyKhPTSFedRN+ruNvlGHbM3x36utUXx+od7Nu+/X5PUz/UtF53S9a3/22XrW2cdsLb1BbfsveBi2saieLftxTRTFkXi2FYWD7VZ965g68gyaqoK2HFwDQDjhqxrVeaowW92WO990xezcM56dk+wKBpHVbFg9YPtWxQ/Oo+iqlrGDVnXqv5hK89kzYhZreo+cMBSZteM71CGAwcsbTc9uewzH4ztVP+S85286RAeGTi3Tb7E8gcOWMqtK7/M4hUfZ+Od97WyKLZVGUVVgaKJf8ezz/c4Y32RqBXFYElXAPsRXPQBMLOjUhUwsx8APwBIsCjOSZU/kVtvG8j1N8QoL4/8BfSUsiwoGNEq/ZIb96HuZ02Ulhe2XzAh37M/GEFx/+gjx04YcRr7NZ1EUWHqUCITBnye/ayBIuVHZNvJ0w7g9jFHM/zVRljRcic9+vCv0dRwJgNXBXfS45q+RFFhCSuyJWiE7HXyuZStPIPC4hJqUuQZeftU3jrj6kWRCuZknaivoPcTOAQcCVwLrAT+kckGc0FJxEklS0dKIk42lEScdEqiOU+eKIk4hf1K2k8vbkkvKmw/T28lse9paOw4Sw5xwQXZliDvifoqupuZ3QU0mNnzZvYNIKU1kYyZPZf8fMJxHCct/mZ2t4laUcQfn66RdIKkA4FBEcvgOE5fwlc9dZuon1H8RFIl8D1gOrALcHnEMjiO05dYsCDbEuQ9UUe4i7vf2Ax8Lsq2HcdxnJ0jEkUhaTqQciGzmV0ShRyO4/RBhg3LtgR5T1QWRQbfpskfXtie4djAjuO05f33sy1B3hOJojCzezqTT9J0M7s40/I4jtOHuOaaYHN2mtx5ySDgsGwL4DhOL+Paa7MtQd6Ta4rCcRzHyTFcUfQy9hyyKdsiOI7Ty8g1RZEd966O4/ReMumZto+QFUUhqX+KUzdGKojjOI7TIZEqCkmHSlpK4BgQSRMk3Ro/b2Yzo5THcZzsIGmKpHmS5m3YsCGzjVV3KySKQ/QWxTTgC8BGADNbBHwmYhkcx8kyZjbDzKrNrHrIkCHZFsfpgMinnszsvaSkzgWMdhzHcbJC1E4B35N0KGCSioFLgWURy+A4Tl/ixz/OtgR5T9QWxYXAt4EqYDUwMTxOiaRSSa9KWiTpDUn+9ozjOJ3H38ruNlF7j/0A+GoXi9UDR5lZTWiFvCjpcTN7peclzF1e3DI62yI4Tn6y557u76mbRL3q6ZeSdpFULOlpSRskpY1/bQHxEL7F4ZbSE21PUVsby3QTO0XDtoa055u27wAgVlcffNa3/uwqjU070h53QI/eiMSs64+z6mvbj9rZXnpj0859R11pN5Fc/Y11gqinrLvHmjXZliDviXrAjzWzKySdShAv+8vA34D70hWSVAjMB/YBbjGzuZkU8lsXbeKxR+s48aRSbr1tYCab6hJPXvkCK556l1HH7M2x1x3R5vybP3mEjc//k8JBlTR9uLnlc2AlTZs2UzFuIsNPPbfT7S1a+WfWbl7KHpXjmTDitJbj4pFMGPD5tGWf3/TfABO62MW01DR9yDNzr+WTI6/rVP6Zly9m4Zz1TDxuKJOnHdAmfZcxmxk26DQAFm5+irWvrmDQ2gmMPvxr3ZLz0os+4vHHnm/TbrJsl81ZzxdPLOXG23btVntR8s6FN0APj6uT+8gs4zfnLY1JS8xsf0m/BR40szmSFplZp354kgYCDwEXm9mShPQpwBSAIUMGT7r//h/ttIxNMeONJS13g/vtX0RhQc+8ML411q/TeQvqdyNWsrGlbEM/PvjnR83Hg8fsikK5tjf1w2Ixapev77De0t2roCAwJAuSjJOChoTfgsWo2dJirleU7s7WunUtx4W7oRQv0hsxtjZ9yNSpUzGzbn15iWNbWVk56Uc/+hFllXtAWTGx/oGFUVbc+u69omg7FoNVy7Y2pw0fV4EKaJNeUT4M1TewpfGD5rT+u1ZRtKP1/6J+UAEUxygrbqSiaHtzevGOQTT0+7Cl7ynaTSQ5z/j9i+NDAsDmptJW+bc2lrXpX3sk56tsKmdzYW2bfInlKwvrWFW/K9u39aOgQRSE/Y71E7Fig+LA6ikrbiTW1MSW5Rt7ZFwTqa6utnmZfHt60iSYPz9z9fcSJM03s3ZfOonaonhM0pvAduAiSUOAus4WNrNNkp4FjgOWJKTPAGYA7Lvv3jZ2/M+7JeQtt7RYFGee3XMWRVfiUZSuOI+6UXc3H8/dMponb0qwKD7XYlG8/uGeALz5aNcsirIkvTJgbesL7r9emNldi6LbJI6tJLviiiv55NnXUVNVwI6DgxnJcUPWtSpz1OA3AbhveoJFcULLnX08fZcxEzl40GkUrVgdWBT1Kxi0d2BRVLzT+mK84qz+FFXVMm7Iuub6AYatPJM1I2a1yvv0bS/x+GN1bdpNJC7DF08s5eSzWlsUs2vGtzp+5oOx7fYvmeR8J286hEcGtjW+E8sfOGApt678MotXfJz+q8WA1YFiqKkqYFuVUVQVKJr4dzz7/N+323ZO40qi20T9MPtKSb8ENptZk6Ra4JR0ZUJl0hAqiTLgGOAXmZTz1tsGcv0NMcrLc8sV1rHXHUHD1Q0U9y9u9/zYq06m6Xs7WFczlFhdPQWlJcTeEwUlJcTq6+m/ubTdcqmYMOI09ms6iaLCfq2PV2/soCR8duBXeOKj3y7qUoMdMKBwEJ86+HK2dpwVgMnTDqD+J42UlBe1m37vA6fAi8GFcGLlMdSN+Arb9+n+jcGNt+3Kkdfs26bdZBk++5slOfcb64iRt0/lrTOu7tFxzThTpsCMGdmWIq/JxkOpscAISYlt35sm/zDgnvA5RQEwKyH2dsbo6T9wT0W3S6Uk4hSW9YMaKCgtAaCgpPVnV4kriVTHHdDxE90uUKDCLpdJdbFuL72ocOe+o660m0i+KYkEenRcM86dd7qi6CaRKgpJvwdGAQtpeSPbSKMozOx14MDMS+c4juO0R9QWRTUw3qJ8gu44juN0i6ht3yXAHhG36ThOX2b16mxLkPdEbVEMBpZKepXgjWsAzOzkiOVwHKevMH9+8Ha2s9NErSiuibg9x3H6OiefDD7b3S2iXh77vKSPAaPN7K9hpLuuL2VxHMdxIiNqX08XAA8Cd4RJVcDDUcrgOI7jdI2oH2Z/GzgM2AJgZm8BQyOWwXGcvsQdd3Scx0lL1Iqi3sya3Y+GL9355KHjOJljypRsS5D3RK0onpf0n0CZpGOAPwGPRiyD4zh9CfWY/8I+S9SK4kpgA7AY+CYwG7gqYhkcx3GcLhD1qqcYcGe4OY7jOHlAJIpC0mLSPIsws09EIYfjOH2QE0/MtgR5T1QWRXykvh1+xp3an4M/zHYcJ5M86o9Bu0skisLM/g0g6RgzS/QE+31JCwieXTiO00dIjFxYtdfHeHfjtoy1NeT/nc6GPzzYgTwZaz6n2Nl+Ru3CQ5IOM7OXwoNDif6BuhMRRdmJd9IpSj/a0XEmJ2MkRi4sGTbaPnP9sxlra+WTj5PJ+vsCUf+Rzwd+J6kyPN4EfCNdAUl7EcSr2J1gmmqGmd2YUSl7mJ4KWpRvjIC9si1DKo786b+YPr7jfE7mGb5rGb88Y0LmGvgF/CpN/dma+4462kJHrZ2VJm5o1Kue5gMT4orCzDYnnpf0dTO7J6lYI/A9M1sgqQKYL+kpM1sajdTOzlDVtJkPYFC25WiPge9sY8z/rmPvYRtooH+2xenz7Nq/H6dNGp7RNjJdf2/grDTnsjLtY2abk5VEyKXt5F1jZgvC/a3AMgIfUXlPXW1Tx5lS0LCtoXm/aXvbaZRYXT2xuvo26c3nd6Q+l0xj047mLRUyo9QamrfP7ni70/VnnJhRXNvYvO37+DoAvrjsNUqb6imNNVAaa0AW63LV7RWpre16Pcll62szH2003kbjti5Pw3Ut+Hq2cc+x3SbX5pDTPmqRNIIgLOrcVHm2bze+ddEmbr1tYM9K1sPcdOly5s7+kEOOH8QlN+7TpbJPXvkCK556l1HHBFNaK556l90+O4axVwVhPdbf9EdqX1kMQPnECQydfG6r8qseupetyxYycOREPv751ueSWbTyz6zd3GK87VE5ngkFh7bJJ4zT617nK3ULKQyN3Ju61KvMIYOD7n6Xg29fSUFTy0Xj4pfmcDFzaEL8oXwS99G1KcKZly/mnKO/yH3TFzN52gHNaZfNWc/E46w5rbNcetFHPP5YHQP32MSmtfVMPG5ol+voLDMvX8zCOeuZNqyAtWvupHziBEYc8/UOy7111o8B9suIUJlixgx349FNlEtRSSUtMLODUpwbADwP/NTM/pJ0rnkFxeDBgyddeeWV7Ld/EYUFubGUYWusX6tjixkrl7as8hgxvj9KkrWgfjdiJRubj2uaSprLfvDPj9ptp3yfwL9i7fL1rdL7Da9CEmoUxGLUrWuJ+FW2WxVSYFgWNLT+LWhHI1u3r23TTkXhbiiFTi+zBvaIbaWIGFOnTmWeWbcGIXFsd60cNOkX191MU0kBsX4i1j+wyMqKW999VxRtb7eu4m1NVK7a3qqfjSpkbUEF21UMJcU0lQTfRWF9a6ugflABFMcoK26komg7FoNVy7YyfPhwVq1axfBxFUCQFmf4uAqUxmavLKxr3o/FYOmShjZ5ho+roCZW1qn+bW1sna+yqZzNhbVt8g0o2N5KzjilQ6sobAyGK9ZPxIoNioPvoay4kcb6BmpWfsTUqVOxbo5rItXV1TZv3ryeqq4tklsVnUDSfDOrbu9cXlgUkoqBPwP3JysJaL2CYq+99rLnnr+KM8/OHYuivYfZf7o5waI4pa1FUbriPOpG3d18/MqW0c37T96U3qJY+OCcdi2K0vXB17vqofvatSgGrG19wS17b2unLYpETq97nf/Y/mraPJ0lcWwri4farHtXsHVkGTVVBew4uAaAcUPWtSpz1OA3U9Z30Mv/5ogbljcf3/GxL/Dw9pEANI6qYuvI4GJb8U7ri/GKs/pTVFXLuCHrmuu/b/pizjn6Cu776y+ZfMIBzWkL56wPrIET0lsDBw5o/Zht+s1xi6KkxaI44QCe+WBsp/qXnO/kTYfwyMC2xvdRg99slnOPYQWsXRNrtigGrA4UQ01VAduqjKKqQNHEv+NHzrw1bZ+c3kmuKYqXkhMkCbgLWGZmv+6ogrIy5fy0E8AlN+5D3c+aKC3vetymY687goarGyjuXwzAaxcPprCsxWoZesnZxKacCkDBlso25Yefei6xE86ifFNJh21NGHEa+zWd1HxcVNgP3l2TtsyhO1ZSRyEbafqgUx2KkFF/3UBDaQFLTtidcf+zgcM/XMbDZSN3qq7J0w5gj7crWimEydMO4LO/WcLztn+X67vxtl352Q0xnrf9qa9tpKQ8c3/PydMOoP4njZy6+7+4bOmJLFs2FjoRWnr0A9fy1hlXv5ExwZycJFJFIWkgcC4wIrFtM7sk/PxOO8UOA74GLJa0MEz7TzObnVlpM8/OKIk4cSUBtFIScQpKQyWwpf3yBf06VhJxigrb1p+K3WK1lNHApbucwsotf/l3pwtGQPm6Ovpta+KBBz7Jxn0HcG3FQfx61r0Maqrlw8Lynaqzvaml8vICqNlJGcOymVQSceJtFPXv/PiG1HWcJYd45JFsS5D3RG1RzAZeIfAe26mlIWb2Ih085HZyh0KLcWnFKexQrhmrUNBo/PGBappKAwW9fMgwLt7/m+y2YkWWJXMyyqRJ2ZYg74n631xqZt+NuE0nQtYXVmRbhJRsrSprk7ajsJgNOSyz0wNUVfnD7G4S9XsUv5d0gaRhkgbFt4hlcBzHcbpA1BbFDuB64Ie0vFFuwMcjlsNxHMfpJFEriu8B+5hZzq2GcRynl3LBBdmWIO+JeuppOZA5f8KO4zjJzJiRbQnynqgtilpgoaRngWZnQ/HlsY7jOD3OpEkwf362pchrolYUD4dbn6Gvuhh3nJxhwYJsS5D3RO1mPNmFuOM4jpPjRP1m9ju0Ez/DzHzVk+M4mWHYsGxLkPdEPfWU6JmwFDiDHA1u4zhOL+H997MtQd4T6aonM9uYsK02s98AJ0Qpg+M4fYxrrsm2BHlPpIpC0kEJW7WkC8k9D7aO4/Qmrr022xLkPVFfpH9FyzOKRmAlwfST4ziOk6NErSi+CJxGazfjZwP/FbEcjuM4TieJ+s3sh4GTgAYCj/01BC/hOfnE3r6KxMkjMhlmtY8QtUUx3MyO60oBSb8DTgTWm+1E2DDHcXKOxFjoe+/tL6XmOlFbFH+XlD6QcFtmAl1SLo7j5DZmNsPMqs2sesiQIZltrLq64zxOWqJWFIcD8yX9U9LrkhZLej1dATP7G/BhNOJ1jtraGLW1nQrQ14a62qaM5O2Ixi0pYqKGNDXUpz2fsl5r2Kly3WFnZe2t1Nc2ZryNxm07Eg/7Z7xBJ6fIxsPsvOZbF23isUeDkMEnnlTKrbcN7HTZmy5dztzZH3LI8YO45MZ90uZd/149Fx03v1N5O+Lda/6Lpk2bKaoYyOiLr25z/u2n72XTOwsZMuwTjJt0TqfrXVTzNGsb3mGP4pFMGPD5bsnYWRb/6wHWvbyEXcZMZPeDvxxJm7nMzMsXs3DOeiYeN5S9fzg2I22s+fUs3np5CXsetQ/vP7McYFxGGnJyFlkehAiUNAJ4LNUzisT5TmB/YEmGRCkADkxKe43Oxf9OLpuuXFfydkQRMCHheBHB0uTuttWZcmPMrFtxRpPGdixQ3kGbUTMYyFZ8lY7GoCdka+83j5n1WBx7SRuAf/dUfe2QzTHqSTLdj4+ZWbvzgL1CUSTlnWdmeT8p2Rv60Rv60BG53Mdcli1Kesv3kM1+RP2MwnEcx8kzcl5RSPpv4GVgjKRVks7PtkyO4zh9iZz3s2RmX+likd4S97A39KM39KEjcrmPuSxblPSW7yFr/ciLZxSO4zhO9sj5qSfHcRwnu7iicJw0SNpL0rOSlkp6Q9KlYfogSU9Jeiv83DVMl6SbJC0PXyo9KMPyFUp6TdJj4fFISXPD9h+Q1C9MLwmPl4fnR2RSrlwg1djlI8njHDWuKBwnPY3A98xsPPAp4NuSxgNXAk+b2Wjg6fAYgpdKR4fbFOC2DMt3KbAs4fgXwDQz2wf4CIgv/jgf+ChMnxbm6+2kGrt8JHmcI6VPKQpJBZJ+Kmm6pK9nW56dRVK5pHmSTsy2LDuLpC9JujO8yz022/KkwszWmNmCcH8rwZ+1CjgFuCfMdg/wpXD/FOBeC3gFGCgpI+52JQ0niBD52/BYwFHAgynkisv7IPD5MH+vJc3Y5RXJ45wN8kZRSPqdpPWSliSlHxf6jlou6cpU5UNOAYYTuDlflSlZU9FDfQD4PjArM1J2TE/0w8weNrMLgAuBszIpb08RTtccCMwFdjezNeGptcDu4X4V8F5CsVVk7uL0G+AKWt7G3g3YZGbxN+8T226WKzy/OczfJ0gau3wjeZwjJ+eXxyYwE7gZuDeeIKkQuAU4huBP8Q9JjwCFwM+Tyn8DGAP83czukPQgwZRBlMyk+32YACwFSiOQNxUz6WY/zGx9uH9VWC6nkTQA+DNwmZltSbwZNzOTFOnywdCaXG9m8yUdGWXb+Uby2GVbnq6QK+OcN4rCzP7WzgO4g4HlZvY2gKQ/AqeY2c8JYli0QtIqIO4Gs+dcs3aSHurDkQT+jsYD2yXNNrNI7zR6qB8CrgMej08P5CqSigkuNPeb2V/C5HWShpnZmnBqKa74VgN7JRQfHqb1NIcBJ0s6nuCmYRfgRoKprqLQakhsOy7XKklFQCWwMQNy5RQpxi6faDPOku4zs8577+wB8mbqKQVdNfP/AnxB0nTgb5kUrAt0qQ9m9kMzuwz4A3Bn1EoiDV0di4uBo4HTJV2YScG6Q6jQ7gKWmdmvE049AsSfc30d+J+E9HPD1U+fAjYnTFH1GGb2AzMbbmYjCMIJP2NmXwWeBU5PIVdc3tPD/L36Jao0Y5c3pBjnSJUE5JFF0ROY2TZaVoHkNWY2M9sydAczuwm4KdtydILDgK8BiyUtDNP+k8AamqXApcy/gTPDc7OB44HlwDbgvGjF5fvAHyX9hMCb7F1h+l3A7yUtJ4jvcnbEcmWDdsfOzGZnUaa8JN8VRVRmfibpDX2A3tOPVpjZi0Cq1UFtgnCEd+nfzqhQbdt8Dngu3H+bYBowOU8dcEaUcmWbDsYu70gc56jJ96mnfwCjw5eM+hHcJT2SZZm6Sm/oA/SefjiOk0TeKAq140U2fGD3HeAJgjXSs8zsjWzKmY7e0AfoPf1wHKdzuFNAx3EcJy15Y1E4juM42cEVheM4jpMWVxSO4zhOWlxROE4XkDQi2cdVPiHpGklTsy2Hk1+4onAcx3HS4orCyUvCO/tloavyNyQ9KalM0nOSqsM8gyWtDPcnS3pYQZChlZK+I+m7CoLBvCJpUJq2JklaJGkRCS/TKQgmc72kfygIUvTNMP3IUI4HJb0p6f7QnQSSrlMQSOd1STeEaUMk/Tms5x+SDksjyzWh997nJL0t6ZKEc9+VtCTcLktI/6Gkf0l6kcAxZjx9lKQ5kuZLekHS2DD9jLCORZJyxdWNk03MzDff8m4DRhAEppkYHs8CziF4c7U6TBsMrAz3JxO41agAhhC42b4wPDeNwLNoqrZeBz4T7l8PLAn3pwBXhfslwDxgJHBkWP9wgpuxl4HDCdx6/5OWZekDw88/AIeH+3sT+CZKJcs1wN/D9gYTOPYrBiYBiwkcRg4A3iBwqx1P70/gOHA5MDWs62lgdLh/CIEfIcL8VYky+ta3t3x34ZERJNWY2YAMt3EhsM3M7u0wc8+3PRl40szej7rtHuYdM4v78JlPoDzS8awFAWy2StoMPBqmLwY+0V4BSQMJLpbxO+vfE0SxAzgW+ISkuBO+SoLIdjuAV81sVVjHwlC2V4A64C4FIS3jYS2PBsarxXX5LpIGmFlNin78r5nVA/WS1hPEwjgceMjMasM2/wIcQaCoHrLAzxkKXL/HXW8fCvwpod2S8PMlYKakWQRp/C+/AAAEF0lEQVSONJ0+jiuKDCKp0MzadWduZrdnq22Cu+slQL4rivqE/SagjMDKiE+pJsfsSMwfSziOsXP/BQEXm9kTrRIDV/DJshWZWaOkgwl8RJ1O8Cb7UaG8n7LAH1NnaFP3TsheQBDkaGLyCTO7UNIhBFHV5kuaZGa93iW5kxp/RtEBkv5/whz0tQnpD4dzu29ImpKQXiPpV+F89qfD45+G872vSNo9zNe8+iScb/6FpFfDueQjwvT+kmaFc9oPSZobn39PIWty21eHsi+RNEMBpwPVwP2SFobz+pMkPR/25wllKHRnRKwkmG6BFnfbO42ZbQI2STo8TPpqwukngIsUxDxA0r6SylPVFd7FV1rgvfRygiBUAE8SuF2P52tz8e4ELwBfCn8z5cCpYdrfwvQySRXASWG/tgDvSDojbFOSJoT7o8xsrpldDWygtbNHpw/iiiINCmI5jybwxjkRmCTpM+Hpb5jZJIKL7iWS4mEly4G5ZjbBAu+V5cArZjaB4E97QYrmiszsYOAy4Mdh2reAjywIDv8jWi6AqUhu+2Yz+6SZ7U9wt32imT1IMJf+1fBushGYDpwe9ud3wE879w3lJDcQXLxfI5jD7wnOA24Jp5ASvZH+liDa4AIFS2bvIP3dfQXwmKTXgReB74bplwDV4c3IUoLwsF3CguBPM4FXCcJ9/tbMXgvTHwAWAY8TOG+M81Xg/PDG4g2CUMEA10taHPbp72FZpw/jvp7aIf6MIlyVcjqwKTw1APi5md0l6RqCuzYI5p+/YGavSGoESuLTPpLqgVIzM0lnAceY2X+E5WvM7AZJzwE/NLOXQovjJTPbR9LDwI1m9mxY1wJgipnNSyF3ctunEcTa7Q8MAqab2XVhe1PNbJ6k/QkuBm+H1RQCa8zs2O5+j47j9A78GUV6RKAY7miVGMxBHw182sy2hRfe+Hx4XdKzgQZr0cbp5pPrO5GnI+oSlEQpcCvBCqD3QsXUXpxtAW+Y2ad3sk3HcXo5PvWUnieAb4Rzy0iqkjSUYHXLR6GSGAt8KkPtv0QYOU3SeOCALpSNK4UPQvkT5+u3EkyDQLBcc4ikT4ftFEvar1tS5ymSbgmf2yRuUUeoi8tyXjuy3JINWRzHLYo0mNmTksYBL4dLCGsI1urPAS6UtIzgQvtKhkS4FbgnnLd+k2AeeXNnCprZJkl3EqxuWkvruemZwO2StgOfJlAiN0mqJPhN/CZsq09hZpFGpkuHmd0N3J1tORwH/BlFTiOpECg2szpJo4C/AmPMbEeWRXMcpw/hFkVu0x94Nlx+KeBbriQcx4katyjyEElzaXmLNs7XzGxxNuRxHKd344rCcRzHSYuvenIcx3HS4orCcRzHSYsrCsdxHCctrigcx3GctLiicBzHcdLyf0vZ8za+FnksAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x432 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plot_objective(result=search_result, dimension_names=dim_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also show another type of matrix-plot. Here the diagonal shows histograms of the sample distributions for each of the hyper-parameters during the Bayesian optimization. The plots below the diagonal show the location of samples in the search-space and the colour-coding shows the order in which the samples were taken. For larger numbers of samples you will likely see that the samples eventually become concentrated in a certain region of the search-space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGECAYAAADZfzztAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3xb1fn48c8j7xVn2HF2nGlIyICYAGGFEFbYZZRZoJSwR4ECXYxv21+h0AKlQAlll1E2IWGFEBNGprN3QraznGEnTjyl5/eHrh3b8pIlWbLzvF8vvax77jiPLFuP7rnnniOqijHGGAPgCncAxhhjIoclBWOMMVUsKRhjjKliScEYY0wVSwrGGGOqWFIwxhhTxZKCMcaYKpYUABEpaoE6bhKRX4S6nnrqvlZEuoWjbmNM6yJ285o3KahqchCOE6Wq7mDEFMy6RSQHuFdV57ZsVMaY1sbOFGoRkd+IyBwRWSQij1Qr/1hEckVkqYiMr1ZeJCJ/F5GFwHHO8l9EZKGIzBSRDGe7h0XkXud5jog8JiKzRWSViJzolCeKyLsiskxEPhKRWSKS3UCstet+0Il9iYhMEK+LgWzgTRFZICIJIjJCRL51Xs+XItI1NL9NY0xrY0mhGhE5HRgAjASGAyNE5CRn9S9VdQTeD9g7RKSTU54EzFLVYar6vbM8U1WHAdOBG+qpLlpVRwJ3AQ85ZbcAe1R1EPBHYEQjIdeu+1+qerSqHgEkAOeo6vvAXOBKVR0OVADPABc7r+dl4C9N+w0ZY9q66HAHEGFOdx7zneVkvEliOt5EcKFT3tMp3wW4gQ+qHaMMmOQ8zwVOq6euD6ttk+k8PwF4GkBVl4jIokbirV33KSJyH5AIdASWAp/W2icLOAKYIiIAUcDWRuoxxhwiLCnUJMBfVfWFGoUio4GxwHGqesBpo493VpfUassv14MXatzU/zsubcI2jamqW0TigeeAbFXdJCIPV4uxxssBlqrqcc2s0xjThlnzUU1fAr8UkWQAEekuIp2BVLzNOgdE5DDg2BDV/wNwqVP3IGCIH/tWJoCdTvwXV1u3D0hxnq8E0kXkOKeeGBEZHFDUxpg2w84UqlHVr0TkcGCG07RSBFwFfAHcJCLL8X6ozgxRCM8Br4nIMmAF3uafwqbsqKoFIvIisATYBsyptvpV4N8iUgwchzdh/FNEUvH+DTzl1GWMOcRZl9QIIiJRQIyqlohIP+BrIEtVy8IcmjHmEGFnCpElEZgmIjF42/5vsYRgjGlJdqbQCojILCCuVvHVqro4HPEYY9quVp8U0tLSNDMzM9xhHPJyc3N3qmp6uOMwxgQmLM1HIvIycA6ww7nRCqcL5Q1AvrPZ71T1s8aOlZmZydy5NnpDuInIhnDHYIwJXLi6pL4KnFlH+ZOqOtx5NJoQjDHGBFdYkoKqTgd2h6NuY4wx9Yu0m9ducwaie1lEOoQ7GGOMOdREUlJ4HuiHdyC6rcDf69tQRMaLyFwRmZufn1/fZsYYY/wUMUlBVberqltVPcCLeEcqrW/bCaqararZ6enW4cUYY4IlYpJCrTH9L8Q7XIMxxpgWFK4uqW8Do4E0EdmMdz6B0SIyHFBgPXBjOGIzxphDWViSgqpeXkfxSy0eiDHGmBoipvnIGGNM+FlSMMYYU8WSgjHGmCqWFIwxxlQJKCmIyGNNKTPGGNM6BHqmcFodZWcFeExjjDFh0qwuqSJyM3AL0FdEFlVblYJ38nljjDGtUHPvU3gL+Bz4K/BAtfJ9qmqjnxpjTCvVrKSgqoVAIXC5M9l8hnOsZBFJVtWNQYzRGGNMCwnojmYRuQ14GNgOeJxiBYYGFpYxxphwCHSYi7uALFXdFYxgjDHGhFegvY824W1GMsYY0wYEeqawFsgRkclAaWWhqv4jwOMaY4wJg0CTwkbnEes8jDHGtGIBJQVVfSRYgRhjjAm/QHsfTcPb26gGVR0TyHGNMcaER6DNR/dWex4PXARUBHhMY4wxYRJo81FuraIfRGR2IMc0xhgTPoE2H3WstugCRgCpAUVkjDEmbAJtPsrFe01B8DYbrQOuDzQoY4wx4RFo81GfYAVijDEm/AKdZCdGRO4Qkfedx20iEtOE/V4WkR0isqRaWUcRmSIiq52fHQKJzRhjjP8CHebiebzXEZ5zHiOcssa8CpxZq+wBYKqqDgCmUnNIbmOMMS0g0GsKR6vqsGrL34jIwsZ2UtXpIpJZq/h8YLTz/DUgB7g/wPiMMcb4IdAzBbeI9KtcEJG+gLuZx8pQ1a3O821452gwxhjTggI9U/gNME1E1uLtgdQbuC7QoFRVRcTnTulKIjIeGA/Qq1evQKszxhjjCLT30VQRGQBkOUUrVbW0oX0asF1EuqrqVhHpCuxooN4JwASA7OzsepOHMcYY/zSr+UhErhKRqwFUtVRVF6nqIuBSEbmimbFMBK5xnl8DfNLM4xhjjGmm5l5TuB34qI7yD4F7GttZRN4GZgBZIrJZRK4HHgVOE5HVwFhn2RhjTAtqbvNRjKoW1S5U1f1NuU9BVS+vZ9WpzYzHGGNMEDT3TCFBRJJqF4pICjbZjjHGtFrNTQovAe+LSO/KAue+g3ecdcYYY1qhZjUfqeoTIlIETBeRZKe4CHhUVZtyR7MxxpgI1Owuqar6b+DfTpMRqrovaFEZY4wJi0BvXrNkYIwxbUigw1wYY4xpQywpGGOMqRLofAqJIvJHEXnRWR4gIucEJzRjjDEtLdAzhVeAUuA4ZzkP+HOAxzTGGBMmgSaFfqr6N6AcQFUP4B0t1RhjTCsUaFIoE5EEQAGcuRWaO0qqMcaYMAu0S+pDwBdATxF5EzgeuDbQoIwxxoRHoPMpTBGRecCxeJuN7lTVnUGJzBhjTItrVlIQkaNqFVVOo9lLRHqp6rzAwjLGGBMOzT1T+HsD6xQY08zjGmOMCaPmDoh3SrADMcYYE34BXVMQkXjgFuAEvGcI3wH/VtWSIMRmjDGmhQXa++h1YB/wjLN8BfAGcEmAxzXGGBMGgSaFI1R1ULXlaSKyLMBjGmOMCZNAk8I8ETlWVWcCiMgxwNzAwzLhkPnAZJ+y9Y+eHYZIjDHhEmhSGAH8KCIbneVewEoRWQyoqg4N8PjGGGNaUKBJ4cygRGGMMSYiBHpH8wYR6QD0rH6sQG5eE5H1eC9eu4EKVc0OJEZjjDFNF2iX1D/hHevoJ5xB8QjOzWun2HAZxhjT8gJtProU7/DZZcEIxhhjTHgFOnT2EqB9MAKpRoGvRCRXRMbXtYGIjBeRuSIyNz8/P8jVG2PMoSvQM4W/AvNFZAnV5lFQ1fMCOOYJqponIp2BKSKyQlWnV99AVScAEwCys7O1roMYY4zxX6BJ4TXgMWAx4Ak8HFDVPOfnDhH5CBgJTG94L2OMMcEQaFI4oKr/DEokgIgkAS5V3ec8Px34v2Ad3xhjTMMCTQrfichfgYnUbD5qbpfUDOAjEamM7S1V/SLAGI0xxjRRoEnhSOfnsdXKmt0lVVXXAsMCjMkYY0wzBXrzms2rYIwxbUigZwqIyNnAYCC+skxV7TpAiNU1eF19gj2onT91G2Nal4DuUxCRfwM/B24HBO88Cr2DEJcxxpgwCPTmtVGq+gtgj6o+AhwHDAw8LGOMMeEQaFIodn4eEJFuQDnQNcBjGmOMCZNArylMEpH2wOPAPLw9j14MOCpjjDFhEWjvoz85Tz8QkUlAvKoWBh6WMcaYcGhWUhCRo4FNqrrNWf4FcBGwQUQeVtXdQYyxQYvzCn16w4RzCslw9gqqj02zaYxpquZeU3gBKAMQkZOAR4HXgUKcgeqMMca0Ps1tPoqqdjbwc2CCqn6AtxlpQXBCM8YY09Kae6YQJSKVCeVU4Jtq6wK+Ic4YY0x4NPcD/G3gWxHZibdb6ncAItIfbxOSMcaYVqhZSUFV/yIiU/Hek/CVqlZOdOPCe3ezMcaYVqjZTT2qOrOOslWBhRMZwtlbx8YVMsaEU6B3NBtjjGlDLCkYY4ypYknBGGNMFUsKxhhjqlhSMMYYU+WQv9Gsqb19QtEjqTX0NGoNMRpjgsfOFIwxxlSJuKQgImeKyEoRWSMiD4Q7HmOMOZREVFIQkSjgWeAsYBBwuYgMCm9Uxhhz6IiopACMBNao6lpVLQPeAc4Pc0zGGHPIiLSk0B3YVG15s1NmjDGmBcjBsezCT0QuBs5U1V85y1cDx6jqbbW2Gw+MdxaPAJa0aKDBlwbsDHcQAcpS1ZRwB2GMCUykdUnNA3pWW+7hlNWgqhNwZngTkbmqmt0y4YVGW3kN4Y7BGBO4SGs+mgMMEJE+IhILXAZMDHNMxhhzyIioMwVVrRCR24AvgSjgZVVdGuawjDHmkBFRSQFAVT8DPvNjlwmhiqUF2WswxkSEiLrQbIwxJrwi7ZqCMcaYMLKkYIwxpoolBWMaICKZItJq74MRkYdF5N5wx2FaD0sKxhhjqlhSMK2C8419uYi8KCJLReQrEUkQkRwRyXa2SROR9c7za0XkYxGZIiLrReQ2EblbROaLyEwR6dhAXSNEZKGILARurVYeJSKPi8gcEVkkIjc65aOdON4XkRUi8qaIiLPuURFZ5mz/hFOWLiIfOMeZIyLHNxDLwyLysnP8tSJyR7V1d4vIEudxV7Xy34vIKhH5HsiqVt5PRL4QkVwR+U5EDnPKL3GOsVBEpvv51pi2RlXtYY+IfwCZQAUw3Fl+F7gKyAGynbI0YL3z/FpgDZACpAOFwE3OuieBuxqoaxFwkvP8cWCJ83w88AfneRwwF+gDjHaO3wPvF60ZwAlAJ2AlB3v5tXd+vgWc4DzvBSxvIJaHgR+d+tKAXUAMMAJYDCQBycBS4Mhq5YlAO+d3cK9zrKnAAOf5McA3zvPFQPfqMdrj0H1E3H0KxjRgnaoucJ7n4k0UDZmmqvuAfSJSCHzqlC8Ghta1g4i0x/vBWPmN+Q28Q7kDnA4MdcboAkgFBgBlwGxV3ewcY4ET20ygBHhJRCYBk5z9xgKDnJMJgHYikqyqRfW8jsmqWgqUisgOIANv0vlIVfc7dX4InIg3KX2kqgec8onOz2RgFPBetXrjnJ8/AK+KyLvAh/XEYA4RlhRMa1Ja7bkbSMB79lDZDBrfwPaeassemve3L8DtqvpljUKR0XXEFq3eO/RHAqcCFwO3AWOceI9V1ZIm1utz7GbE7gIKVHV47RWqepOIHAOcDeSKyAhV3dWMOkwbYNcUTGu3Hm+TCXg/eAOiqgVAgYic4BRdWW31l8DNIhIDICIDRSSpvmM5385T1XuX/q+BYc6qr4Dbq23n80HdBN8BF4hIohPDhU7ZdKc8QURSgHOd17UXWCcilzh1iogMc573U9VZqvogkE/NQSnNIcbOFExr9wTwrjOc+uQgHfM64GURUbwf4JX+g7dZaJ5zITkfuKCB46QAn4hIPN6zjLud8juAZ0VkEd7/wenATf4EqKrzRORVYHZlbKo6H0BE/gcsBHbgHWSy0pXA8yLyB7zXJd5xtntcRAY4MU51yswhyoa5MMYYU8Waj4wxxlSx5iNzyBKRZ4Ha9wg8raqvhCGW64A7axX/oKq31rW9MaHS6puP2qUmaZdeHgDyy5Op0CgAYiSKvsld691v04FdFFX4dv5Ii00hPb4dALv27Gd34X7cbiU+LpqMtHYkxMfUe8wlO7ejokRFe3zWqYKnzBtblEsYlNaZlTt3UuZ2kx4TQ355OQBd2hUQ7fLdv6AwmeKS2BplnVOTSUv1XufML91Cmae4al00SkZ03Z1bStVd9TzK1YHo6F71vqYKrWDTAZ/J7wDomdidaPF+r8jNzd2pqun1HshPaWlpmpmZGazDmQAE+701ka3VnylkZHTi2YlJ7KlI5PlNo6vKL+t1Cjf2P6fe/SbnzeORxe/XKBOEN4+/nf4pXfhkykL+9sIUulVbn5wYx3vP3UC75No9H73umDqJiT8tI7nzflxRNZNt2b4YSncnAnDVEcP40+ixPDp9Oi/Oncuvu3fnyTzvB++5Ryzi+H6Lax7YE88//nUepWUHk0KUS3jvj78gs4v3xtxZu77io7yaUxrc0XEVXWolhgOeCnZUSx5dOr1BYsLYen5LXo8sfYxVRWtqlA1I7sfDgx+oWhaRDQ0exE+ZmZnMnWszfEaCYL+3JrK1iWsKJZrC1F0nAeBCGN15GNf2OaPBfc7ufhRXZZ5IrMubF1Oi4/nt4Avon9IFgEnf+I6BVnSglJyZq+o95iPHn8qJ3TMp3pOAp6LqBiGiy+Ip3ZOAAGP79OP+Ud5Y7zruOMYNHAjOzURdkpO5OOsP9Ew+g8q3JiGqMyd2e4KfHX8MMdHeM43UpHgeuvr0qoQAcHTHsYzqdBZRzjf3xKgUChJ+D9FVoxxQ4erObvUeVySBDu3uazQhANzS/3r6JvWuWu6b1Jtb+/+q0f2MMa1Pq28+Gpg1QFetXI1bPawr2kb72CTS4lKbvH9h2QG2lRTQOymd+KiDTUPX3Psaa9bn+2x/9/WnctFZRzZ4zE37CtlfVoor2kNKTDwZCams3LWT1Lg4uqW089n+62++ofuQIQzs1Ikol/dDu7gin1L3HlJj+yHiTQZ7iorZvmcffbp0JC6m7pO8/RV7KSjfSee4HsS4vGcWWrEGiEWie+HxFFFesY6Y6N64XL6xNGRr8TYAuiZ08VknIrmqmu3XARuQnZ2tdqYQGYL93prI1qzmIxHpAPRU1UVBjsf/WJxv1FHion9Kt0a29pUam0hqbKJP+ZjjsnySQnS0i5NG9m/0mD1TfJPS4Wn1N8lGu1wcnl5zfUJ0OgnRNcs6JCfQITmhwbqTotuRFF3zw16iD8bsciUTFzukwWPUp65kYIxpW5qcFEQkBzjP2ScX2CEiP6jq3Q3u2Epdcd7RrN+8i69/WIHHo7RLjufe8WNJ75QS7tDCyuPxMP/bFezM28OQ4wfSrU/wrj86N6CNB+jVq/6L3yZ0Mh8I1v1/prXy50whVVX3isivgNdV9SHnjsw2KSYmiofuPJubrzqJ/F1F9M9MJy621V+XD8i+Pfv57UVP8dPiTQCICFfffy6X3zMuKMdX1QnABPA2HwXloMYYv/hzoTlaRLoCl3JwtMc2r3OnFAYP7HrIJwSAN5+YXJUQwDvs+uuPTmTjqq1hjMoYE0z+fNL9H94BwX5Q1Tki0hdYHZqwTDity9vFlBkr8CiMPXYg/Xt6m4jm5Syrc/vcb+ouN8a0Pk1OCqr6HvBeteW1wEWhCKo121m8n/+tWcSmfQVkd+7BeX0GERsVFe6wmmzqrFU8+Oxk3B5v683rE2fzh/FnMO7EQXRIb8emVdt89umQ4V8vJmNM5Gpy85EzTPDUyknMRWSoM9qicWwuKmTcp6/y+LzpvLN6Eff+8BnXTX0Pt8f3DuVIU1JewRuz53PfJ1+wr6Pidr4ueFT551vfUlZewYU3neqzX5feaYw6qzkjPxtjIpE/1xReBH4LlAM43VEvC0VQrdW/l8xiR3HNybN+2LqBqZt/ClNETeP2ePjVmx/y5y9yKIp3U9pR2Ncb3M5tGwX7isnbUcixZw7j9y+PJ2tEHzp1SWXMJcfw2Md3E9vA0B/GmNbFn2sKiao6u9pUfuCd9co4luzybVqpLD+914AWjqbppq1ex5yNNcc30iihpKOStB3i46Lp3CEZgBPOPYoTzj0qHGEaY1qAP2cKO0WkH6AAzjy11u2kmqwOdffZH9ghrYUj8c+q7TvrLHc7M/hecdYIkhLj6tzGGNO2+HOmcCvePuSHiUgesA64KiRRtVI3Dj6GLzasorDs4CB0w9K6ckavgWGMqnGHd6k7mXVLTuH+W0/ktOMOa+GIjDHh4k/vo7XAWGc+WJeq7gtdWK1T39SOTDr3Wt5YMY8N+wo4unMPrhg4jBhXZPc+OnlAH0b16cWP6zZWlXVITOClX15C747twxiZMaalNZoURKTOYSwqry2o6j+CHFOr1jM5ld9lnxLuMPziEuGFKy5g4qLlzN2YR/fUdlw6YggZKcnhDs0Y08KacqZQOdhPFnA0MNFZPpeDk4abVi42KoqLjzyCi488ItyhGGPCqNGkoKqPAIjIdOCoymYjEXkYaNLoWeId+3kukKeq54hIH+AdoBPewfWuVtUyEYkDXgdGALuAn6vqen9flDHGmObxp/dRBlBWbbnMKWuKO4Hl1ZYfA55U1f7AHuB6p/x6YI9T/qSznTHGmBbiT1J4HZgtIg+LyCPALODVxnYSkR7A2cB/nGUBxgCVc2G+BlzgPD/fWcZZf6rUujHCGGNM6PjT++gvIvI5cCLeexWuU9X5Tdj1KeA+Dl6b6AQUqGrljW+bge7O8+7AJqe+ChEpdLav0ZG++rj76enp5OTkNPVlRKSioqJW/xqMMW2Dv+NBuwEP3qTQ6IA+InIOsENVc0VktP/h1a36uPtZWVk6enTQDh0WOTk5tPbXYIxpG/wZEO9O4E0gDegM/FdEbm9kt+OB80RkPd4Ly2OAp4H2IlKZkHoAlWMs5AE9nfqigVS8F5yNMca0AH+uKVwPHKOqD6nqg8CxwA0N7aCqv1XVHqqaiXfwvG9U9UpgGnCxs9k1wCfO84nOMs76b1TVZuDyg7vCzcLpy1ny40o8rWB0VmNMZPGn+UjwNh9VcjtlzXE/8I6I/BmYD7zklL8EvCEia4Dd2Cisflk9fx0P//xpdubtBqDHgC488t6v6TGga5gjM8a0Fv4khVeAWSLykbN8AQc/zBulqjlAjvN8LTCyjm1KgEv8iMk4VJVHr3u+KiEAbF69jX/c/B/+8fUfwxiZMaY18af30T9E5Fu81wmg6b2PTAvYsDyPzat9h+5eOmM1Bfl7aZ9us6MZYxrnb++jBXiHy44GEJFeqrqx4V1MS0hMia+zPDomyibBMcY0mT+9j24HtgNTgEl4h7iYFKK4jJ8690zj6NOH+pSfculxJKYkhCEiY0xr5E/vozuBLFUdrKpDVXWIqvp+CpmweeCVmxl7xfHExMUQnxTHueNP5fanrw13WMaYVsSf5qNNQGGoAjGBS26fxG9evJF7J4zHRgcxxjSHP0lhLZAjIpOB0spCm08h8lhCMMY0lz9JYaPziHUexhhj2hh/uqQ+0tB6EXlGVRsb9sIYY0wE87dLakOOb3wTE2xbCvfy33kLWb97D8O6d+WK4UNJiY8Ld1jNUn302169eoU5GmMOTcFMCqaFbSoo5KLX3mZPcTEAU1b/xKRlK3nv6suIj2l9b2310W+zs7NtzCtjwqD1fXKYKq/MnleVECqt2JHPpOUruXjoYL+Pl1+ygoW7/8fe8jy6JAxleMcrSIzuGKxwjTGtQDCTgnV5aWGrd9Y9qviaesobkl+ykk823o5by6qWNxbN4KLMl4hx1X23tDGm7fHn5jUARCSxnlVPBxiL8dPgLp39Km/Iot3/q0oIlQrLN7N237RmxWaMaZ38GeZilIgsA1Y4y8NE5LnK9ar6avDDMw355cij6NYupUZZdo/unJk1wO9j7S3f6le5MaZt8udM4UngDJyZ0FR1IXBSKIIyTdM5OZlPrruKO084juEZXcju3JWxvftSWl7R+M61dEmoe8SSrvWUG2PaJr+uKajqplp3y7rr29a0EIXJc5azYVcBAAvXbuWT+ct4+8bLSIpr+j2Gwztezsb9P1JQdnDQ2/4pp9E9cUTQQzbGRC6/xj4SkVGAikgM3gHylocmLNNU/5u9qCohVFqzYxcf5i7h6lFHNfk4CdHtuaj3f1i771v2lW+hS8JQuic1fX9jTNvgT1K4Ce/F5O5AHvAVcGsogjJNt2LbjjrLl2/N9/tY0a44BqaeHmhIxphWzJ9hLnYCV4YwFtMMWRnpfLlktU/5wC5pYYjGGNPa+dP76G8i0k5EYkRkqojki8hVoQzONO7nI4fSo0PNqTb7pHXgohFHhCkiY0xr5k/z0emqep+IXAisB34GTAf+G4rATNN0SErg3Zuv4J3Zi1i9fSeHd+3MpUcPabXjHxljwsufpFC57dnAe6paaOP2R4b2iQncNPqYcIdhjGkD/EkKk0RkBVAM3Cwi6UBJaMIyxhgTDv5caH5ARP4GFKqqW0T2A+eHLjTjr9079/HRGz+yemkevfp15mdXj6JLDxvQzhjTdP4OiHcYkCki1fd7PYjxGD+oKl9tn0HOjjm4KzxseLeAws8qAGHB7LV8+8Ui/vn2zWR06xDuUI0xrUSTk4KIvAH0AxZw8E5mxZJC2Ly+/lPe3zzlYMHZ4IqJw/OJd1TTwj0HmPj2LG6458wwRWiMaW38OVPIBgapqk1+EgEOVJTw6ZZvfcpdo0vxfBEHpd5OAJvX72zhyIwxrZk/A+ItAbqEKhDjn4LyfZR6ynzKJRZo56laPnxojxaMyhjT2vmTFNKAZSLypYhMrHw0tIOI9BSRaSKyTESWisidTnlHEZkiIqudnx2cchGRf4rIGhFZJCI2+E49MuI7kRbb3qdc9wjs9L6tmf0zOOfn1lXVGNN0/jQfPdyM41cA96jqPBFJAXJFZApwLTBVVR8VkQeAB4D7gbOAAc7jGOB55+chq8zjZmdJEenxycS4oqrKo8TFjf0v4bHlr1Ch3qGyoyWKcfGnoFfH0rtfZ04+cwjxCU0fKdUYY/zpkvqtiPQGBqjq184MbFGN7LMV2Oo83yciy/EOqHc+MNrZ7DUgB29SOB943bluMVNE2otIV+c4h5x31s7jqWXT2F16gLS4JO4+YgwXZw6vWn9sp6G8kP1Hvt85H1BOSDuKzvEdwca0M8Y0kz+9j24AxgMd8fZC6g78Gzi1iftnAkcCs4CMah/024AM53l3YFO13TY7ZYdcUpiTv4EH50+uWt5Zup/f535Kv5Q0jux08DpB5/iO/KxHk94CY4xplD/NR7cCI/F+qKOqq0WkSZMBi0gy8AFwl6rurT48hqqqiPjVo0lExuNNUKSnp5OTk+PP7n4pKaugYF8xHlVSEuNISQz+mEJFRUU+r2HLgULuiOrvs+3yWfMoTFwT9BiMMQb8SwqlqlpW+YHu3MDW6Ie5MyHPB8CbqvqhU7y9sllIRLoClZMC5AE9q+3ewymrQVUnABMAsrKydPTo0X68jKablruah2BG11oAACAASURBVJ+fhNtz8GVedeYI7rz05KDWk5OTQ+3X8Nu5E/lgg++H/8+7HMUVR432KTfGmGDwp/fRtyLyOyBBRE4D3gM+bWgH8WaQl4DlqvqPaqsmAtc4z68BPqlW/gunF9KxeIfUCFvT0bMffF8jIQC8PWU++XuKQl73OT3rHvr6nJ6DQ163MebQ5U9SeADIBxYDNwKfAX9oZJ/jgauBMSKywHmMAx4FThOR1cBYZxnnmGuBNcCLwC1+xBdU5RVuNmzb41PudntYt3V3yOs/PqMv9w8ZS1K0t/dQSkwcvxt6OsekZ4a8bmPMocuf3kcevB/UL/qxz/dAfeNr+1wddXodRcQUnzHRUfTp2tEnAURHuejbrWUGmbt+4HFc1ncEefsL6JHUnsRo615qjAmtRpOCiCymgWsHqjo0qBFFkNsuOZH7nv0Ut/vgHcJXnZlNWvvkgI67dFkeW7YWMHhQd7p19b0Brbqk6FgGpjbpen7AFi/P46PP5lNQeIBjR/TlgnHDiY3xd8xEY0xr1pT/+HOcn5Xf4N9wfl5FEy40t2YnDe/H63+8kk+/X8KBkjJOGTGAE4b1bfbxSkrK+cNDH5A7fwMAInDlZcfRr0+wIm6+mblr+e2fPqy6hjJ3wQZyF27gsQcvCnNkxpiW1GhSUNUNACJymqoeWW3V/SIyD++1hjZrYK907rnilKAc680PZ/L9tvVEdYDYPYIq/PftGdx/d/hH83j17R99LqrPmLuW5au3cviAri0SQ/Wuxr169WqROo0xNflzoVlE5PhqC6P83P+Q9tnG5TxW8T07zvCw9XwP209344nxfgjv318a5uhg0xbfi+oAmzaH/qJ6JVWdoKrZqpqdnp7eYvUaYw7y50P9euA5EVkvIuuB54BfhiSqNmZXyX7u/nEiFdEHv4mXdIOCI73LUdHhz62DsnzPBkTg8Kxuje67Na/uhGKMaX386X2UCwwTkVRnubD6ehG5RlVfC3J8bcL0rWsp87h9yg/0VvqsTKBdSkIYoqpp/NUnsnTFFoqqnbVccl42PRuYtW1vYTH/7w8fMG/W2pYI0RjTAvzuWlI7GVRzJ97B7UwtSfV0JU2KjuXJxy9nw/qlIau7YFcRK+atJ6NHR/ocXv+3/gF9M/jvc9fz5bSl7Ck8wHHZfTlySMPt+s8+8bklBGPamGD2N6zvfoRD3snd+tE1sR1bD+ytUX7XqJPpk5nOhvWhqffjl3J46f9NpKLMe5Yy8tRB/O7564irZzjtjh2SuPxnI5t0bI9H+f6b5UGL1RgTGYKZFNp099RAxEVF88aYy3lk7ld8v20dHeMTuS5rJNcMzPbZtqSknBk/rKK4uIxjjxtAx07Nuydi4+ptTHjkY6rPnjp76jI+eGEaV9x1RrNfS3WuKBeUexNOdHD/llicV0jmA5NrlK1/9OxgVuFz/FDUYUxrY2cKLaRvu068NuZy3B4PUa66LyyvW7uD++5+iz279wMQExPFfb87l1NOrX+8I1U36tmDuDoicvC4s75eWiMhVJo5ZXFQkoLLJYw9ayiTP8oFILPmQIbGmFYqmN1efgjisdqs+hICwDNPfVmVEADKy9089ffPKS72nYsZoHT/O+zdPpK9249k745RlBUfnB01KSW+zn2SU4N3UfvGX5/OsDMHkpS5n1TvPBvGmFauyUnBmQXtDhH5hzOP8j9F5J+V61X1ttCEeGioqHCzaMFGn/L9RaWsWLbFp7y8dAbFhb9BPd5Rx9Wdx4E9t1NRvgSAk849knYdknz2O/vqEwIP1uOBoiI2btrMj/s3MWbLosCPaYyJCP6cKXwGZOIdJTW32sMEQVSUi/btE+tc1ynN97pC2YF369jSQ/mBDwBITk3kr+/cyvATBhIV7aJbZhp3PX45x581LPBgVeGJJ+h/WH9++Md93P7DF4Ef0xgTEfy5phCvqneHLJJDnIhw8c+P4T8vTKtRfsxx/enVO62OPSrqPI5qedXzvoO689e3QzDobFQUPPwwz5TGcemzf6PrvoLg12GMCQt/ksIbzjzNk4CqO5xUteXGQWjjLrtyFIlJcUz+dD4lB8o44aTDuOraupt7YhPOpbz44zrKz6lj69DIOHccb/34I/dMn9RidZrmsZ5Wpqn8SQplwOPA7znY/VSB5g8banycd8EIzrtgRKPbxcSfTlzyrykteg4oBUkiPuVuouOODX2QjotGHsGmvNUUR8ewu6I8v8UqNsaEjD9J4R6gv6ruDFUwpmlmTVvOR698x+585ZhT/szF1/ehXYdBiCulReOI2b6NvomxrJj0BevOPNX3KrkxptXxJymsAQ6EKhDTND9OWcqfb3uj6h6ETT/tYN4P23nmo6Nb/kaRigqYNYvDEsI/dpMxJjj8SQr7gQUiMo2a1xTuCHpUpl4fvPStz01pa5dvIfe7VRx98mEtG0zv3i1bnzEm5PxJCh87DxNG+Vvr7umTv7W+cQqNMabp/Bk620ZAjQBDj+nH1I/n1SgTEYYd2y9MEUWeunrahLqOpvbksV5AJtI1OSmIyDrqGPROVa33UQu65tdnsDR3Pds2HewJfOVtp9I9s657GYwxxj/+NB9VH9IzHrgEG++mxaV3bc8Ln9/DjClL2Z2/l6NOGEjv/hnhDssY00b403y0q1bRUyKSCzwY3JBMY2Jjozn57CAMV2GMMbX403x0VLVFF94zh6COoW+MMSa8/PlQ/zsHrylUAOvxNiEZY4xpI/xJCmcBF+EdKbVyv8uA/wtyTBGjqGA/Py1cT7d+XUjv0Snc4Zg6BLunUTCP1xKxBdJzqSV6aZnWx9/7FAqAeUBJaMKJHB8+PZmXf/cWpcVluKJcnPXLMdzx/A24GpgkxxhjWjt/kkIPVT0zZJFEkFW5P/H8r1+tWva4PUx+8WsGHt2fcb86NXyBhdC+8r3sdxeREdcVEWHbnn1UuN30SGvf6L5uz/5GtzHGtA7+JIUfRWSIqi4OWTQR4rsPZtVTPqPNJYVyTxn/3fAyc3bPwIOHjjFp5M/PYva8YgAG98rgsWvG0bOe5LCt8F9s2/tsS4ZsjAkhf9pCTgByRWSliCwSkcUi0ibnYYxPjKuzPK6e8tZs8taPmbX7Bzx4ANhdvhMdMBNXtBuApRu3c+8rdbc9FxyYwpbCv+FRO1Mwpq3wJymcBQwATgfOBc5xfrY5p151IrHxMT7l4341NgzRhNbs3T/6lEXHuWnffW/V8orNO1i33XcupT0HbCgsY9oaqT3iZmsjIvuAleGOI0BpQGufpyJLVQOa0EFExgPjK49HZL6vkfxehSq23qqaHoLjmgjUFpLCXFXNbnzLyGWvofWI5NcZybGZ1sP6VxpjjKliScEYY0yVtpAUJoQ7gCCw19B6RPLrjOTYTCvR6q8pGGOMCZ62cKZgjDEmSCwpGGOMqWJJwRhzyBGRTBFZEsb6rxWRf4Wr/oZYUjDGmDZGRJo9AZolBWNM2Djf2JeLyIsislREvhKRBBHJEZFsZ5s0EVnvPL9WRD4WkSkisl5EbhORu0VkvojMFJF6540XkREislBEFgK3ViuPEpHHRWSOM67bjU75aCeO90VkhYi8KSLirHtURJY52z/hlKWLyAfOceaIyPFN/B2cKyKznNfwtYhkiIhLRFaLSLqzjUtE1jh11FmPiDwsIm+IyA/AGyIyWERmi8gCJ84BTYnHkoIxJtwGAM+q6mC8c7Zc1Mj2RwA/A44G/gIcUNUjgRnALxrY7xXgdlWtPcH59UChqh7tHPMGEenjrDsSuAsYBPQFjheRTsCFwGBVHQr82dn2aeBJ5zgXAf9p5HVU+h441nkN7wD3qaoH+C9wpbPNWGChquY3Us8gYKyqXg7cBDytqsPxTp+8uSnB2BzLxphwW6eqC5znuXhnd2zINFXdB+wTkULgU6d8MTC0rh1EpD3QXlWnO0Vv4B3kE7yDfA4VkYud5VS8iaoMmK2qm51jLHBim4l3orGXRGQSMMnZbywwyDmZAGgnIsmqWtTI6+kB/E9EugKxwDqn/GXgE+Ap4Jd4k1q99TjPJ6pqsfN8BvB7EekBfKiqqxuJA7AzBWNM+JVWe+7G+2W1goOfT/ENbO+ptuyheV90Be8ZxHDn0UdVv6ovNlWtAEYC7+MdLfoLZ70L7zf+yuN0b0JCAHgG+JeqDgFuxHm9qroJ2C4iY5z6Pm9CPVXj2KvqW8B5QDHwmXOcRllSMMZEovXACOf5xQ1s1ySqWgAUiMgJTtGV1VZ/CdwsIjEAIjJQRJLqO5bzrTxVVT8Dfg1UNkd9BdxebbvhTQwvFchznl9Ta91/8DYjvaeqbn/qEZG+wFpV/SfeM446z6Jqi7ik4Fw8WuxcHJkb7niMMWHxBN4P6vl4hwQPhuuAZ51mIKlW/h9gGTDP6ab6Ag2fcaQAk5xJxr4H7nbK7wCynYu6y/C26TfFw8B7IpKL79DnE4FkDjYd+VPPpcAS5/UeAbzelGAibpgLp5dBtqpG6pj1xhjTIpweWE+q6oktVaddaDbGmAgkIg8AN1OzqSv09UbgmcI6YA+gwAuqaiM/GmOaTESeBWrfI/C0qr5S1/YhjuU64M5axT+o6q11bR8JIjEpdFfVPBHpDEzB2ytgeq1tqqZtjIuPG9G9ZxcAoiWWmk2FgSkpq6izPDYmCpf41lPqduNRj095jCuKaJeLsnI3Hk8d66NdREVFBR5wNe4KDxXlvvGLy0VsXPBPEFetWrUz0Ckbq7+vCQkJI3r27BmU2Dwepby0vK4a0Tqvqkmdf0ZRLiHKJbhcEXcpDgCPx+MTm1uVMncdfwcixEfV/Xfg0Qo8uKuW167eEPB7WyktLU0zMzODcSgToNzc3Drf14hLCtWJyMNAkao+Ud82/QZm6o0fH05GfH+u6ftMUOv/9VMf8f2idTXKOqQkMOmJG4iN8f2H+veSWTyam+NTPvncaxncMYP/vTeLf7/ou/6Be0ZwxhljgxU2ABtXb+OmUx+l9vt79T3juOKuM4JaF4CI5AZzKsjs7GydOzc4/QzKSsq5euRD7N2zv0b5sacdQWFaCvMXbKhRntg5gR2xvh+kf7lxHLHF2xg9enRQ4gq2nJwcn9gKS0s49v3nKK6omRQv6T+Ex48fV+dxthav5I11d1Ut3z/4i6C9t8F8X01g6vufjaivPCKSJCIplc/x3lTS6KBV7WI6M67b3Y1t5rf7rjqVPl0P3jXfLimOP40fV2dCALju8BGc3vPgneQxLhf3H3UygztmAHDh+SM48fiBVeujo13c8MuTiQvBN/deA7pww4MXEB178Azk6DGDuOjGU4JeV6SLjY/hvmd+QXJqYlVZ74FduPlPF3HPnWfQo3uHqvLU1AT+330XcvlpR1WdDYrAhScP4bSRWS0ee6BS4+L5+/HjSIqOrSob1qkrD4wYXe8+XROyODH9F7gI7tmraR0i6kzB6Vf7kbMYDbylqn9paJ+BWQN0xYoVuCQ0f8CqyoLVeRwoKWPEYT2Jj41pdJ+Ve/LZsK+AI9O7kZ7g29153bp88rYWMOiwrnTsmFznN7xgKdi5jxXz1pPRsyN9Du8ekjogss8UKpUUl7FoxmoSk+IZPLIvlXeEejzKosWbKCurYPiwXsTGepP0lp2FrN6UT99uneiZ4U0coXyvAtVQbPvKSpm9fRMd4hM4Kr1pfwdF5bvZWrKSge1G2ZlCG1Tf/2zIeh+JSD9gs6qWishovDdOvO7cRFInVV3LwRtBmlYPrpAlBPC2vR45sIdf+2R1SCerQ/1NsH36pNOnT1CaaBvVPi2FY08f0iJ1Rbr4hFhGjhnsU+5yCcOH9fIp75aWSre01JYILeRSYuM4tWd/v/ZJjunIgJjjQhSRiVShbD76AHCLSH+8c8f2BN4KYX3GmAgkIuNFZK6IzM3Pzw93OKYRoUwKHmeMkAuBZ1T1N0DXENZnjIlAqjpBVbNVNTs9vWXOkE3zhTIplIvI5XjH8qgcRbDxBnljjDFhE8qkcB1wHPAXVV3njE/+RgjrM8YYE6CQXGgWkSjg96padXu2qq4DHgtFfcYYY4IjJGcKzhCvvUUkttGNjTHGRIxQDoi3FvhBRCZSc+KHf4SwTmOMMQEIZVL4yXm48I4/bowxJsKFLCmo6iMAIpKoqgdCVY8xxpjgCVnvIxE5zpkVaIWzPExEngtVfcYYYwIXyi6pTwFnALsAVHUhcFII6zPGGBOgkI6SqqqbahW569zQGGNMRAjlheZNIjIKUBGJwTv70PIQ1meMMSZAoTxTuAm4FegO5AHDnWVjjDERKpRnClr9jmZjjDGRL5RnCjNF5D0ROUukjgmNGyAiUSIyX0QmNb61McaYYAnlmcJAYCzwS+AZEXkXeFVVVzVh38rrD+2aUlHegZ38a/XHzN69kvYxSVzc8yQu7z2m2YE3V7nHzd/nfM/bKxZRXF7O6ZkDeGjUGNITk1i/bzePzJ3C99vW0ik+iV9mjWT8oGNr7F9R4eaV/3zLZ5/Op6SknBNOyuKW20+jQ8fkkMeuqrz5/ize/zSXwr3FHDOiL3fcMIZuXdrX2O7Tl3J4/19fsXPLHoYcP5Ab/3wpfQaFbka3QO3dW8xzz0xh+rTlxMRGccZZw/jVjadQQgV/mvc1kzcsAw903BKL6/syBvTO4MZrTmLkkX3CHXrIrF2ymQl/fI/FP64irXsHLr3jTM6+tmbHwI/WzeW51RMpj9pfz1FMWxXKm9cUmAJMEZFTgP8Ct4jIQuABVZ1R134i0gM4G/gL0KSJl+9d8ALbSnYDsLtsHxN+mkysK4aLep4YhFfSdI/P/o4Ji+ZULU9au4LNRYW8e+7lXP3N2+TtLwRgR3ERjy74hvioaH6RdXA2vJcm5PDeOzOrlqdNXca2bYU88/y1IY/9fx/PZcIb31Ut/zjnJzZs2sUbz19PdJT3hHLquzN57oF3qrZZ+N1KfnfxU7w8+08hj6+5/vTQh8zPXQ9AWVkFH743m/LyChYcWci3W9dWbbelawWJRwmr5+zgt3/6iJeevobMnp3CFHXo7N9bzG8vfoq9u4oA2LFpN//6zVskpSYw+sKjAVhZsJWnfnoHV7Ti1ym+aRNCefNaJxG5U0TmAvcCtwNpwD00PAPbU8B9gKcp9XhUqxJCdZ/m1ZlzQsajytvLF/qUL9ixlddX5lYlhOreXDPv4P4eZfKn8322Wb40jzWrtwU32Dp88sUCn7K8bQXMXbC+avmzV6f7bFOQv4/vJ/nGHQnyNu+uSgjVTfpuYY2EUOlApuKJUsor3Hz29eIWiLDlfT9xXlVCqG7yKwff23+vmILLFTlzt5uWJd4v9CE4sMgqvPMnvKKqm2utu19VfYbRFpFzgHGqeoszr/O9qnpOHduNB8YDpKenj/jzS75j7EW7ouib1LITvS3ZuZ26fpudE5PIL/H9R4xxuchq35mioiKSk5NZvWobdb0dPXt1JCEhtAPO/rQunwq3bx7u1iWVlOR4ADau3EppSbnPNp27d+CCS84LeHL36u9rRkbGiHfeeaeRPRpWWlrOhvW7fMo1Gso71L1PTIGAB9qnJpCR7tt6WfleRaKmxFaQv4/8LXt8yuMSYuk1sAsAG4p2UqolVetuPO/qgN/bStnZ2Tp37txgHMoESETqfF9DeU0hS+vJOHUlBMfxwHkiMg6IB9qJyH9V9apa+0/AO+8zWVlZ+kHKYso8FTUOdH73UYzOGh3gS/DPf7/4kK83/lSjLC0hkUlnjWP0p89R5ql5794vBo7gxuzR5OTkMHr0aKZ+8S4zflxdY5sOHZN4671LiImJCmnsc5Z8yadfLapRFhcbzYevnl+VFF778RM+fPLzGtu4XMLLc4LTfFT9fc3OztbRo0cHdDyPR7n6smfZvq3mWdrwo3qTO2YfG4pqfjjG5gudp3r/Jf720EUcO6KvzzEr36tI1JTYtqzdwQ23PYTHU/Nf86r7zqnad/LGBTyxxubDOlSFsvdRmog8LiKficg3lY+GdlDV36pqD1XNBC4DvqmdEOpy/+GXkRgVV7U8tH1fftV3XKDx++3/ThjLYR3TqpY7xifwz1PPpUtSCo8few5J0Qe/7Y/s3JO7h55cY//bf30mffp2rlpOTU3k9w9eEPKEAHDjNScx5PCDF4wTE2L5/d3jqhICwM/vOovsMYOrluMSYrj9iSvJ6JVGJHK5hN8/eAEdOyZVlfXs1Ym7f3M2T406n4yEg9+qo/dBh9lRRLmEyy88us6E0BZ069uZWx67nLiEgzPjjjx9CBffdnrV8tm9hjMkYUidZ63mEKCqIXkAXwHX4+1FdDLwMvCYH/uPBiY1tt3AgQNVVXV/ebHO2rlcV+3dpOHk8Xg0d1ueTt+0TovLy2us21dWojl5a3TJrq01yqdNm1Zj/6VLNuvcOWu1tLTm/i1hxZptOnPuWt1/oLTebdYt26yzpyzWvXuKqsqAuRrEv58RI0YE7TWVlVVo7ty1unjRRvV4PAfL3RX6w9Z1Omv7Bt28dY/+OOcn3Z6/t8FjVX+vIo0/se3dU6SzpyzW9cvz6t1mTeE2fWHZ1KC+t8F8X01g6ntfQ9l81ElVXxKRO1X1W+BbEZnT6F4OVc0Bcpq6fWJ0PCM7HeZ/lEEmIhyV0a3OdckxcZzcrV+j+w8aHL4unln9MhrdJvPw7mQeHrndUGuLiYniqBG+XUxjXFGM6pJZtdy9VvfbtiylfRJHjz2iwW36tcugX7sMbmyhmExkCGVSqLwiuVVEzga2AB1DWJ8xxpD5wGSfsvWPnh2GSFqnUCaFP4tIKt4uqM/gvRHt1yGszxhjTIBCefNa5RAVhcApoarHGGNM8AQ9KYjIM1Bnd30AVPWOYNdpjDEmOEJxpmB3phhjTCsV9KSgqq81ZTsReUZVbw92/cYYY5ovlBeaG3N8GOs2xrSQ6sOX9OrVK8zRBEftHk5tqXdTSOdoNsYYVZ2gqtmqmp2enh7ucEwjLCkYY4ypEs6kYEO1G2NMhAl5UhCRxHpWPR3quo0xxvgnlJPsjBKRZcAKZ3mYiDxXuV5VXw1V3cYYY5onlGcKTwJnALsAVHUhcFKDexhjjAmrkDYfqeqmWkXuOjc0xhgTEUJ5n8ImERkFqIjEAHfinVvBGGNMhArlmcJNwK1AdyAPGO4s10tE4kVktogsFJGlIvJICOOrUuGJzBMY1Ybjcle4ne28U5FWzrFcWe4vjx6co9njUZ8pG+ujIZiiy99jqiruOuaY9q6r8Cmrb9vmcLs9jcbrz+8zkpS5S8MdgmlhoRwldSdwpZ+7lQJjVLXIObv4XkQ+V9WZwY8QJufN48U1U9lSvIfD23XnzsPGcVRH38lYWpKqsqLgFVYXvEWpp4DOCSM5Ku0+UmIzq7Z5d0IOH73yHQOPWM4vbl1Gp8672FiQwZPfjWLxmj4kzd9L/4R2XHPbWE46Y0ij9X2+7RO+2fEV+yuKGJg8iIJlA/nixzzcbuW04QN44KJTSE2K99m3osLNRx8/Rvtebwf710Bx+WI+XHYcg9Ke4bDO9c8Z765w8/pTX/HZO7M4UFTCyFMO55YHzye9a3vKS7+nZO9fcJcvwRWVSXzKPezaczLPPvE5c2f8RFJyHOddcjRX3zAal8v/HtL52wp49v9NYvZ3q0hMjGXcpSO55tZTiYo+OH3q3uIS/jo5h8+XrMIlwrnDDuP+caNJjI1p4MjhN2PH5yzZ/TTtY/LDHYppYaHsffQ3EWknIjEiMlVE8kWkwfmWnVniipzFGOcRkq9XM3eu5pHF77Ol2Dt5+/K9edyV+yrbiwtCUV2TrSl8hyW7n6XUswdQdhTP4tstt+DWMgAmvTmDV574nE6d13Png9/TqfMuAHq1385j4ybSucsu8kemsG53AY/e/y5L529osL6pO75g4pYPKKrYh6KsLFrKxo6TKS0vp9zt5rPcFdz/+md17vvu2/+mz4gX6NSlMKi/g0qZyXlsLLy2wW3e/NfXvDshh6K9xXg8ysypy3jwhleoKF/H/l3X4i5fAoDHvZ4DBXfw8jOPMvuHNXg8yr69Jbz50ne8/cp3fsemqvzx1jeYmbMCj9tD0b4S3n1pOm88X3Ma8gfe/4JPFiynrMJNSXkF781dwkMff+13fS1p8/61rC18yBLCISqUzUenq+pe4BxgPdAf+E1jO4lIlIgsAHYAU1R1ViiC+3jTbJ+yEnc5n21ZEIrqmmzt3g99yord29m2/3sAPnvH++sYc/ZPuKJq5svYKDfjDlsKLmF/rzg8HuXzDxoetPa7/Gk+ZXHJZbTvfvCDfsaKDeTtqvnBr6qUuD7xiSHY0uL28s2a1+tdX/n7qG79qm1sWfsy3hPP6pShQ33f98kfzfM7ruULN7F+9Xaf8s/fPzjj7Pa9RXy7ap3PNl8uWcXe4hK/62wp07e/TqwrNE2qi/MKyXxgco2HiSwSivZgABFZoqpHiMh/gPdV9QsRWaiqw5q4f3vgI+B2VV1Sa13VAFvp6ekj3n33Xb/j23RgF/srfP8xO8WlkB7Xzu/jBaKoqIjk5GQACsvW4NFyn22SYroR60pl/aptlJe5yehWRFJKmc92BcUJbCtqh6vUQ3Sxh+R28XTtUf8sqHnFm3DX0eZeUhRHRenB1sV+XToSF1OztXHXnpUkJHljOPuMO3JVtf52niao/r527tx+xCtv/B8A5Z4MUuM717nPT8u21NlW37u/h6go37O+A/tj2bat5vvrcrnoN7Dxuanh4Ht1YH8peRt21fUa6H94VwDKKtys3uG7DUBWlzSiXcH9Tlb97ygQu0o342Jf1fLPzrwl4Pe2UlzXAdr1mqdqlAV7MLmWmI6zLQyIJyJ1vq+h7H00SURWAMXAzSKSDjT565GqFojINOBMYEmtdROACQBZWVk6evRov4P7YOMsnln2iU/5K8fczOD2Pf0+XiBycnKofA3zd85hTeE7Nda7JJaTe39G3IYMMAAAHelJREFUXFQHXpwxiQ9f+Y5jT9nAnQ/94HOsuz79GbM3ZZI+cy8JO8q5768XM3r08Hrrfnvja3yXX7PJw10hzJs2hIoy759Hr/T2TLzqEkRqtru/+OLnjBgRvOsJ1d/X/9/emcdHUWWL/3t6yUbYE0DCLgFFZJGMCOiIC+owOuoI6u/pPBdccGMYdcZtnFGfjvNGHfd9Ax0cRVye+nCXgOyymQDCAyTse8i+dbrP74+qJJ10Z0+lO3C/n09/uurWXU71ra5T99xb5wwZFqM9jnuMYr+XYT1X0T62Y9gyyz75N+mf/VgtLbFjPDO/m0BZ/uUh+V996Sy+/mJAtbRzLxjB+BvGN0jGir4qLfFx5YR/kJ9bXO346eedyHU3VdX17DMz2bI/u1qe4b2P4Zb/OLNB7TWG4OuoOaw59D2bcqY3XyBDm8Qx85Gq3g2MBdJU1QcUAhfWVUZEku0RAiISD0zAfiO6pbmo9y84P+UkxHbBFOPycOug81pdIdRkaJeb6BY/unLf60pkdLf/ItbdGYArbpvASaemsnReH774YBB+vyW/z+9ixorRLN/Wlw6biml3sJzzLz2ZMybWPTC7KGUyg9sPqdyPlXjyMqsUQo9O7fnHVRNDFALAxRfex/qlaQT8zrixKvJ7ySu7vVaFAHDjfRcweHhVn3Xo3I57n7qChPbjiGt/JxBjH3ERk3AFp597D+07xFfmP2F4b66/7exGyxYb5+Wexy6jY+cqLy6Dh/Zi6l3VnxgfmzyRlE5VI5N+SZ3522/PaXR7rcmIrqdRLhdRHjD+Mo9GHDMfgeXqAuhH0IhEVWs1EIvIMGAm4MZSWLNV9aG62hg8eLBu3LixyTLuKspmR9EhBnc4hs4xzR96N4VwT3i5pZso8WfTNW4YHld8SJltm/aSvT+fwcM8eL3bOVTcmy0H3SR7Y8nfnU/v/sl0O6ZTg2XYVbyDPF8uxyam4pUYMrL2UO4PMLx/Tzzuum8OG35axfFDRrWYiQFg0NC+unJlRp0KIZjN63ZRkFfMkFH9iImpGgAH/Afx+9bh9hyLy9MLgNISH+szdtChUwLHDurRKLlq9lVZWTnrV2+jXft4Uof0DFvGHwiwevtu3C4Xw3sd06SVTk2RrbnsL97D+pxFnNFzsjEf1dOGMR81rMG3gWOBNVS9yaxArUpBVTOAkU7JFI6UhC6kJNRuc48UHWNTqet22De1B31TK25o/ekRBz0627v9GnejA0iJ701KfNUT9/D+4W9w4Tju+JMa3V59dIhLbrBCABh4QkrYdJc7CZf79GppsXFeRp48IGz+xhIT42HE6GPrzON2uUjr16tF2mtNusUfQ7f4SZEWw9DKODmnkAYMUSeHIgaDwWBoUZw0Gq4FGv/IajAYDIaI4eRIIQlYLyLLCVowrqq/cbBNg8FgMDQDJ5XCAw7WbTAYDAYHcNL30XwR6Qukquo3dgQ2d33lDAaDwRA5nPR9dD0wB3jZTkoBPnaqPYPBYDA0Hycnmm8BxgF5AKq6CQjvq8BgMBgMUYGTcwqlqlpW8SasiHhwyOOpwWAwHEm0xgt4teHkSGG+iNwLxIvIBOB94FMH2zMYDAZDM3FSKdwNHAAygRuBucCfHWzPYDAYDM3EydVHAeBV+2MwGAyGNkCLKwURyaSOuQNVHdbSbRoMBoOhZXBipHC+/X2L/f22/X0lZqLZYDAYopoWVwqqug1ARCaoarDH07tEZBXWXIPBYDhKCI6o5+6QHGFpjk4aE/bUyYlmEZFxQTtjHW7PEEE8zi5vbh4HD0ZagqMaVX1FVdNUNc2d0HB36IbI4ORNegrwgohkiUgW8AJwbV0FRKS3iMwTkfUisk5Efu+gfK1Kmd9PoIlexEv8VsxmVaXUFxpP2a9+inw1g9RXoepDtWGB2Et8Pnz+Esr9Acr9gQbL2A8iG7KuFlQV/623Nbue8kAAXyD0N/QFyvFrw3+nKrkC+ANlBFQp8zesb5pDxfVX5isPG9O6NnIK9zsolSEacXL10UpguIh0tPdzg4+LyFWqOrNGsXLgDlVdJSLtgZUi8rWqrq+tnYCWsO7AvQzuei8eV2Qip9XF9twc7p//Ld9vzyIxJpYrhw7njlPG4W5A0PYvd/3EE2u/I6sgmyR3Ip6NHgqyyjm+TzfumHQ6g/t35d4Vr7GldBOK4ipI4o4hVzC+/0AANJCN5j0IJV8DHjT+QqT9PYgrIaStNTv3MOPH5+jVcwXtE0o4sL8TK78YxrCUMdxx1Zl0aBcXVsa5qzfwwb8+pCNEXaSi2Usy+N/3PmHme+8ybWAaV117GaMGNC7YTVF5GY/8+BWfbM8koAHujj2Bw6VFBCjjxc2z+SF7LbHuGCZ0H8PV/S/E66r7L6WqrDr0FpmH51AayGP3oWS+Wn0ixyedyF/POZO+nRseLa8h7Dicy4Nzv2PhlizcCO7sAN1LYrl0wkiu++2YsGFWAb7NfIQTO80iyRv6EGI4snHcnKOquTUVgk3IKEBV96jqKns7H/gJy2dSXS2wu+Aj1h+8vwWkbVn8gQBXffIhC7ZnoUB+WSkvrlrOsz8srbds5uHdTF/2AVkFVtD3g/4C9g7IwZ/o56ft+7nt+Y+5+4cX+dn3f4hLcbmADgd5KOM1duXmAaA506HkcyxdWwLF76H5odFNs4uK+a/vnuH4gQtpn1ACQHK3HM68fBELflzDgy99Ub1AIAAFBazO3MgDMz9mxLLvm/ErtTC2bPOXZ/DYu3NJW7EIgNT0b/jDi+9xYNdeKCiw8jWAv6yey/tZqykNlOPTALm+Ym5f/hEPrXuZZdmZBFCK/aV8sjudmVmf1Ftf5uE5rDj0BqUBq496dj3AJeMWs3jbz0yZ/RH+BsrVEAKqXP/OR3y/xbr+ylFKuwj7Y0t5/aOlvPP5yrDl1u2cx2lJM41COEqJpI2/zkC1ItIPKzTnsoZUtr/wa3z+nOZL1YIs3bWDrNzDIenvrs+st+ycrWvw1zQ3uaC8p/VHLaWErb6tIeUSOhcye+1qtHw7lIVRPsWfooGiaklz125kUJ/NIVljYvz0H7qLxWt+5sDhgqoDqvD44wwbMYTlT9/DbYu+CCkbMWzZTh0zkh+eqZLttkVfsODxO+napxc88YSVrx4KfKXM3bEuJH1l9kZ+LtwZkv7V3iXUF2hwQ+5nIWnt40sY0GMv2w7nsHT7jnrlaijLs3ay9VDo9Vdqm/X/57vw1+HBw//E40wYaUMbIJKTg7X+e0QkEfgAmK6qeWGOV65mSE5OomTTXQAs3LYCiaL5zvyyUm7vHmqucIuL9PT0yv2CgoJq+wCDinxMcw8MKSspgqurC1yKN6972HYTyWX+9+vBPy28YJ4lBHsxjyss4kQ9j5is0CfD8T3iOfnMWNasXIbXE+T5fPx4Dgw6jk779uIN+OHOO8O31QiC+7V79+4hv0mDGT+evQMH0/XAPks2G5/LTWHPFDp1T4bv6x/d+DXAza7qsZy7E8tUjiMmN3QeQID58+fXWaen9HS6MjYk/ZLEdhT3iyF74wbSs0KVfUOoeR0VlJYxLTXcQFtx9wePxx32Ny4s+Q0L9gXPUTV/TsbQdojkHTTss4iIeLEUwixV/TBcHlV9BXgFIHVQH41LfY6OscM5uect4bJHjCKfjzEzXiavtPok8OTjT+Dm8eMr99PT0xkftA/wze6NPLhkdkidcRlxeA55EIHjLs8hT7KrHS8piOHJkX/ihO7d0IP/BH9W9QpiTsHV5dZqSVmHDnPP1/czfmT1J0dV+OilM0hOGMDUa84KkWXu6g1kvPxH7lgQ+vTbFIL7NS0tTWv+Jo1h1sLVZLz6TjXZHj/9An7z7zcYdExSg+t5c/5bLD+4rXJ/mnsgn8UfpluH3Rz2VX9eGdt1BFOG1C3zkv3ryDhcvV99fhevfHEuHunAwt9eTEKMt8HyBVPzOir2+Tj9yVfJLal+/cXkKu32waXnjAy57gAWbtzC2I7PNEkGQ9snkuajRTUTxJr1eh34SVX/2dCK2nmP5YTkR1tSthYhwevluXPPp2t8fGXa6J69uHfc6fWWPbvnYK4fNBavWF3kUiFmawyeQx7iYjz86dIzeHTUTcQGqibXfcVeLuj8a4b26I6IIJ2eAnfQSMUzGOnwt5C2+nXtzMWDrmPj9r6VVpUyn5slnw+jvbsPD908MayMvxoxmMv2/Uyxx0u25ecqarhszHAm7dlCscfLe8PHUuLxclX29kYpBIC/jbqAQUFr62Ncbp48+RLuPn4KXWKqllcOat+XqQMn11tfWtI19G43unK/uMzL3BVpxHs68fSFE5usEMIR7/Xy1KRf07Vd1cICT5ESfwBOObEvN04eF7bcqYOnkL5vGOXmVdOjEsdGCiLSCfhPoF9wO6o6zf6+NUyxccDvgEwRWWOn3auqc2trxyWxjEn5tNZVFJHmtD79WHz1jazeu4dOcXEM7trwm9IfTzyLq1NHsyXvIAM7JFFWGGDXwVyO651M+wRrNdD7v/wbS/b9xJ68fM7qPYxOQQpIvEMg6WvwZYB4Ee/QWtu69KQRTCx9lVW71tEhoQBXQS/OuCCeIQN64HKF/21l9256xbg5tGAhW8eO3t7gE2sFPHv30DfOw/70BfTvnkJ5zn6Sr70Gdu+Gnj0bXE+fxM58NmEqmYd3Ux4IkJu5meM79QDgjZMfZENeFgnuWPonNmxVk9eVwMRe/yCnbDtF5dnkFSQzdrwyvGcPYj0t/3ccO6Av6dOvI2PnHuI8XsryyujSsR39eta9WOzM4XPYcXADG/e8D/ylxeUyRC9Omo/mAkuxvKQ2aEmFqi6kngnoUFxRqxAqiHG7GZ3SuKWQFSTHJZIcZ48G4qBn1w7VjosIY3sMgR7hy4u4IWZk+IM1SIyN5ZcDTmq4cOXlsGxZtZFQ1GDL1i0+3o7s1BuWLYMDTRvQnNjZUiTpVE3Iu8XNCR2PbVJ9nWL60CmmDz1DVwe3ODFuN2l97euvnrV8wfROOo7eSfdjlMLRhZNKIU5Vb3ewfkOk6ds30hLUTjjZ4uOhT5/Wl8VgaEM4OafwtohcLyLHiEiXio+D7RkMBoOhmTg5UigDHgPuo2r5qQIDai1hMBgMhojipFK4AxioqsYbmcFgMLQRnDQfbQaK6s1lMBgMhqjByZFCIbBGROYBlW/PVCxJNRgMBkP04aRS+Nj+GAwGg6GN4KTr7JpusQ0Gg8EQ5Tj5RvNWwji9U1Wz+shgMBiiFCfNR2lB23HAZKIwEIvBYDAYqnBs9ZGqHgr67FLVp4BfO9WewWAwGJqPk+ajYCc6LqyRQ/QEOzAYDAZDCE7epJ+gak6hHMjCMiEZDAaDIUpxUin8CriE6q6zLwdCgwQbDAaDISpw+j2FHGAVUOJgOwaDwWBoIZxUCr1U9bzGFhKRN4Dzgf2qWntUmFYiP7cQ8bpJtIPaNIbi8jI8Ljdel7vevEU+Hwne5kfdCmiAwuJ8Yt0JxMSG1qeqqBYi0q7RcSh8AR8KxLhaLjpYXfj9fsqKy4hPjMJ4DRHAV1ZOwB8gNj7GsTZUlaLiMhLiYxARNuVscKwtQ3TipFJYLCInqmpm/VmrMQN4Dnir5UVqOCu//pGHH/6A/R3iIdbLMR3ieeie3zIk9Zh6y+4szOahjE9YdnArcW4vl/QZxR+GnBOiHAKqPLl4MfEHDjDl2Wf5RUoKD511FoOSGhcysoJvsz7im92z0UQfxZtcdM8YyfQ7p5PQ3rqpFhbP5VDuw5SXb8Xj7k2XjveQmHBxvfUWlhfxZtYslmevBODkLqO4pt9/0M7TrklyNoTsPYeZ3P068rMLOG50Krc9N4VBo5oW0KatU1JYyvN3vMW82Uvxl/s5ZeJIpj19NZ27d6y/cCOYt2QjL876nl17cxh3dQaThi4myVPYom0Yoh8nHeKdCqwUkY0ikiEimSKSUV8hVV0AZNeXz0n2bN3HXbe8xv7kDmA/be/JK2baX98jv7BuS1hAA9y07G2WHvwZRSn2l/GvrUt4YeN3IXlfXbGCF5YtIxCwAtP9sGsXV3/wAaXl5Y2WOTNnCV/nzUITfQDEpwY4PHElT979EgClZWvZd+gGysu3AlDu38H+7FsoKV1Rb90vbXmDJYeW41c/fvWz5NByXtryZqNlbAwHdx8mP7sAgA3LNnHPeY9QlF/saJvRyrPTZ/DV29/jK/UR8AdY/OlKHv7dcy3axoYte/nrk5+xa28O3U/aw5Th3xqFcJTipFL4FZAKnANcgGUSusDB9lqM72YtpDQl9D274rJy0pduqrPsqkPb2FZ4KCT9w+0rQ9JmZ4YOovYVFjI/K6vhwtos3PV5SJo7Hjb6VlCYV0x+0WzAXyOHkl/07zrrzfXlsTonVJevzskg15fXaDmbSt6hfBZ9vLzV2osWSopKSX9/aUj62kUb2blpT4u1Mzd9Hf6AtVjwnPNW45UGRdA1HIGIaogniogjIv2Az2qbUxCRG4Ab7N2hwNrWkcwxkoC2HndisKq2b04FNfp1MLCx2VK1PNHcV07J1ldVk5taOAL9Gs191FBa4xzC9mubVAo18q5Q1bT68kUz5hzaDtF8ntEsW2tyJPwOkTwHJ81HBoPBYGhjRJ1SEJF/A0uAwSKyU0SmRFomg8FgOFqIOl9Eqvr/GlnkFUcEaV3MObQdovk8o1m21uRI+B0idg5ROadgMBgMhsgQdeYjg8FgMEQOoxQMBhsR6S0i80RkvYisE5Hf2+ldRORrEdlkf3e200VEnhGRzfYLmifV3UKLyOgWkdUi8pm9319EltkyvCciMXZ6rL2/2T7ez2nZIkltfdcWqdnHrY1RCgZDFeXAHao6BDgFuEVEhgB3A9+qairwrb0PVS9opmKtw3+xFWT8PfBT0P5/A0+q6kDgMFCxMGMKcNhOf9LOdyRTW9+1RWr2catyRCsFEXGJyCMi8qyIXBVpeZqKiLQTkRUicn6kZWkKInKRiLxqP7meE2l5akNV96jqKns7H+uPmQJcCMy0s80ELrK3LwTeUoulQCcRqd85VhMRkV5Y0Qtfs/cFOBOYU4tsFTLPAc6SxnpAbEPU0Xdtipp9HAmiVimIyBsisl9E1tZIP8/2p7RZRO6urbzNhUAvwAfsdErW2mihcwC4C5jtjJR10xLnoKofq+r1wFTgMiflbSlsc8tIYBnQXVUrfErsBbrb2ynAjqBiO3H2RvQU8CegwgdFVyBHVSucZQW3XymbfTzXzn/EU6Pv2ho1+7jVibolqUHMoIa3VBFxA88DE7D+AD+IyCeAG3i0RvlrsV6pX6yqL4vIHKyhf2syg+afw3BgPdB4390twwyaeQ6qut/e/rNdLqoRkUTgA2C6quYFP2CrqopIqy/Zs0eJ+1V1pYiMb+322wo1+y7S8jSGaOnjqFUKqrogzOTYycBmVf0ZQETeBS5U1UexHO5VQ0R2AmX2bk1vcI7TQucwHmgHDAGKRWSuqrbaU0QLnYMAfwc+rxjiRysi4sW6qcxS1Q/t5H0icoyq7rHNQxVKbhfQO6h4LzvNCcYBvxGRiVgPCB2Ap7FMVh57NBDcfoVsO0XEA3QEQj01HkHU0ndtiZA+FpF/qeqVrSlE1JqPaqGxw/UPgXNF5FlggZOCNYJGnYOq3qeq04F3gFdbUyHUQWP74TbgbGCSiEx1UrDmYCuv14GfVPWfQYc+ASrmpK4C/ico/T/tVUinALlBZqYWRVXvUdVeqtoPK6ztd6p6BTAPmFSLbBUyT7LzH7EvJdXRd22GWvq4VRUCRPFIoSVQ1SKqVmO0aVR1RqRlaCqq+gzwTKTlaADjgN8BmSKyxk67F2uUM1sslyvbgEvtY3OBicBmoAi4pnXFBaz5pndF5GFgNdaNEfv7bRHZjBWf5PIIyNaahO07VZ0bQZnaJG1NKbTmcN0pzDlEKaq6EKhthc5ZYfIrcIujQoVBVdOBdHv7ZyxzXs08JcDkVhUsgtTTd22O4D5ubdqa+egHINV+YScG6+nnkwjL1FjMORgMhqglapWChPGWak+m3Qp8ibUOebaqrouknHVhzsFgMLQ1jEM8g8FgMFQStSMFg8FgMLQ+RikYDAaDoRKjFAwGg8FQiVEKBkMdiEi/mn6f2hIi8oCI3BlpOQxtB6MUDAaDwVCJUQqGNoH9xP6T7YJ7nYh8JSLxIpIuIml2niQRybK3rxaRj8UKipMlIreKyO1iBS9ZKiJd6mhrlIj8KCI/EvRymljBTx4TkR/ECqpzo50+3pZjjohsEJFZttsFROTvYgV+yRCRx+20ZBH5wK7nBxEZV4csD9ieatNF5GcRmRZ07HYRWWt/pgel3yci/yciC7GcQlakHysiX4jIShH5XkSOs9Mn23X8KCLR4g7GEClU1XzMJ+o/QD+sQCoj7P3ZwJVYb32m2WlJQJa9fTWW+4n2QDKW6+ip9rEnsbxo1tZWBvBLe/sxYK29fQPwZ3s7FlgB9AfG2/X3wnrQWgKciuWqeiNVS7872d/vAKfa232w/PXUJssDwGK7vSQsp3ZeYBSQieUsMRFYh+UuuiI9Actp3mbgTruub4FUe3s0lm8d7PwpwTKaz9H7aWtuLhxBRApUNdHhNqYCRar6Vr2ZW77tq4GvVHV3a7fdwmxV1Qq/NiuxFEVdzFMr4Eq+iOQCn9rpmcCwcAVEpBPWjbHiifltrAhrAOcAw0SkwgFdR6yoa2XAclXdadexxpZtKVACvC5WaMWK8IpnA0OkyiV3BxFJVNWCWs7jf1W1FCgVkf1Y8RxOBT5S1UK7zQ+B07CU0kdq+f1CLJfmFS6lxwLvB7Uba38vAmaIyGwsJ5KGoxijFFoQEXGralgX3ar6UqTaxnpqXgu0daVQGrTtB+KxRg8VZtCaMSeC8weC9gM07doX4DZV/bJaouXevKZsHlUtF5GTsfwmTcJ6C/xMW95T1PJP1BBC6m6C7C6sgDwjah5Q1akiMhor4tdKERmlqke0m21D7Zg5hRqIyB+DbMYPBqV/bNti14nIDUHpBSLyhG1/HmPvP2LbZ5eKSHc7X+UqENs+/N8isty2/Z5mpyeIyGzbBv2RWAHX0+qQtWbbf7FlXysir4jFJCANmCUia2w7/CgRmW+fz5fiYAjJViALy2QCVS6km4yq5gA5InKqnXRF0OEvgZvE8tuPiAwSkXa11WU/nXdUy1PnH7ACJgF8heVOvCJfyI26AXwPXGRfM+2Ai+20BXZ6vIi0By6wzysP2Coik+02RUSG29vHquoyVf0LcIDqzg4NRxlGKQQhVvzgVCyvkyOAUSLyS/vwtao6CusGO01EKkIbtgOWqepwtTw1tgOWqupwrD/o9bU051HVk4HpwF/ttJuxgq0PAe6n6mZXGzXbfk5Vf6GqQ7Geos9X1TlYtu8r7KfEcuBZYJJ9Pm8AjzTsF4pKHse6Ua/Gsrm3BNcAz9tmoGDPm69hRcFbJdYy1Zep+6m9PfCZiGQAC4Hb7fRpQJr94LEeK0xpo1ArWNEMYDlW2MnXVHW1nf4e8CPwOZbzwgquAKbYDxHrsMLVAjwmIpn2OS22yxqOUozvI6rmFOzVIZOAHPtQIvCoqr4uIg9gPY2BZS8+V1WXikg5EFthuhGRUiBOVVVELgMmqOp1dvkCVX1cRNKB+1R1kT2SWKSqA0XkY+BpVZ1n17UKuEFVV9Qid822L8GK75oAdAGeVdW/2+3dqaorRGQo1h//Z7saN7BHVc9p7u9oMBjaPmZOoTqCpQRerpZo2YzPBsaoapF9k62wX5fUsOX7tErT1mX/LW1AnvooCVIIccALWCtxdthKKFxcZwHWqeqYJrZpMBiOYIz5qDpfAtfatmBEJEVEumGtMjlsK4TjgFMcan8RdlQvERkCnNiIshUK4KAtf7B9PR/LlAHWEslkERljt+MVkROaJXUbRUSet+dZgj+RiJ6GiFwTRpbnIyGL4ejGjBSCUNWvROR4YIm9bK8Aay38F8BUEfkJ66a61CERXgBm2nbmDVh239yGFFTVHBF5FWuV0V6q25JnAC+JSDEwBkthPCMiHbGugafsto4qVLXVo6bVhqq+CbwZaTkMBjOnEEWIiBvwqmqJiBwLfAMMVtWyCItmMBiOEsxIIbpIAObZSx4FuNkoBIPB0JqYkUIbQESWUfX2aQW/U9XMSMhjMBiOXIxSMBgMBkMlZvWRwWAwGCoxSsFgMBgMlRilYDAYDIZKjFIwGAwGQyVGKRgMBoOhkv8PVtuNa3SwEYUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x432 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plot_evaluations(result=search_result, dimension_names=dim_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate Best Model on Test-Set\n",
    "\n",
    "We can now use the best model on the test-set. It is very easy to reload the model using Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = load_model(path_best_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then evaluate its performance on the test-set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 1s 64us/sample - loss: 0.0490 - accuracy: 0.9882\n"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(x=data.x_test,\n",
    "                        y=data.y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can print all the performance metrics for the test-set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.048988553384863916\n",
      "accuracy 0.9882\n"
     ]
    }
   ],
   "source": [
    "for name, value in zip(model.metrics_names, result):\n",
    "    print(name, value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can just print the classification accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 98.82%\n"
     ]
    }
   ],
   "source": [
    "print(\"{0}: {1:.2%}\".format(model.metrics_names[1], result[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict on New Data\n",
    "\n",
    "We can also predict the classification for new images. We will just use some images from the test-set but you could load your own images into numpy arrays and use those instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "images = data.x_test[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the true class-number for those images. This is only used when plotting the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "cls_true = data.y_test_cls[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the predicted classes as One-Hot encoded arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = model.predict(x=images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the predicted classes as integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAD1CAYAAADZANcvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9eXBc133n+zm9791Ab0BjawAEAYoSRUukrdCWZSlREjmOl7Fle2rGyWx5zsvL1ExSM/UyW2ry3h9Tk6mZvJo3lRdX7KkZpzKelLNZrrEsybIsmbY2LuIKEiSAxr51N3rfu8/7A7g3AEVSIACiG+D5VLGIXu69p/vX93t+55zf+f2ElBKFQqFQ3B1DsxugUCgU+wEllgqFQrEFlFgqFArFFlBiqVAoFFtAiaVCoVBsAdNODg4EAjIaje5SU1qfWCxGPB4XzW7HXvGg2Rfg7NmzcSllsNnt2CuUjbfOjsQyGo1y5syZnZxiX3HixIlmN2FPedDsCyCEmGp2G/YSZeOto4bhCoVCsQWUWCoUCsUWUGKpUCgUW0CJpUKhUGyBHS3wKBT3yurqKuPj49RqNWq1Go1Gg3Q6TblcZnFxkVQqpb83EAjQ29uL3W6nvb0dh8NBT08PNputiZ9A8aCixFKxpySTSd566y2KxSLlcplqtcr09DTpdJrz588Ti8X0946MjPDUU0/R3t7OoUOHCAQCBINBJZaKpqDEUrEnzM/PMzExQSwW49y5c5RKJd27TCQSFAoFcrncpmPS6TRjY2P4fD5KpRIdHR088sgjWK1WLBYLRqOxSZ9GsdvU63WKxSLZbJazZ89SKBTo7+/H5/MRDAbx+XzNbqISS8XecPnyZb71rW8xPT3NmTNnqFQqaOkBpZRIKanX65uOWVpaIpFI4HA4CIfDRKNRnnjiCaxWK+3t7UosDxCVSoV4PM7Nmzf53d/9Xebm5vjyl7/M0aNHOXXqlBJLxcFneXmZZDLJxMQEs7OzxONxSqUS1WoVACEEFosFk8mEw+HAYrHoHme5XCaXy1EqlchkMqRSKZaXl/F4PDidTjUcP0DU63Wy2SzZbJZ8Pk+hUKDRaGAwGDAYWmMdWoml4r7RaDR44403+OEPf8jo6Chnz56lWq3qQglgNBoJBAK4XC6GhoYIh8Ok02nS6TTz8/Ncu3ZN9zoMBgPvvvsuiUQCn8+H1+tt4qdT7CbFYpGZmRnm5uaoVqsYjUbcbjdtbW1YrdZmNw9QYqm4T9RqNarVKvF4nOnpaVZWVsjn8wBYLBbMZjM+nw+bzUZHRwcul4toNEooFCKTyZBOp6nVakxOTuoCW61Wyefz5HI5arVakz+hYjep1+u6RymlRAiBw+HA4/FgsVia3TxAiaXiPtBoNPQwoIsXL/LOO+9QKpWQUuJ0OgkEAkSjUb7yla8QCoV00fR4PNjtdiqVCpVKhVdffZWVlRVSqRSLi4s0Gg0KhQL5fP5985uK/U2pVGJ2dpaFhQUAXC4XR44c4eTJk9jt9ia3bo09F8tarYaUUo+xE0IgxN8k8jEYDBiNxk3Pt8qchWJrSCkplUrkcjkymQyrq6vA2pDbZrMRDAbp7Ozk4YcfJhKJ4Ha7MZvN+txlpVKhXC7rYUJms3nTb2Tj34r9jba4V6lUSKVSZDIZTCYTZrMZr9fbEgs7GnsqluVymbGxMVKpFFeuXGF5eRmHw4HNZtNFMhAIcPjwYWw2G06nU//SWsUVV3wwUkry+TypVIpyuQyseQoej4ejR4/y5S9/mc7OTvr7+3G5XJhMpk0T+WNjY1y7do23336bmZkZisUitVoNq9XK8PAwQ0NDuN3uZn5ExS5RKBRIJpNcv36d119/nUwmw8DAAH6/v6WEEvZYLGu1GktLSywuLnL27FmmpqZwu924XC6MRiNms5nu7m59tbNarWK1WnE4HJhMu9PUWz1Zxf2hWq1SqVRoNBq6R9nW1kZfXx8/8zM/Q3t7Oz6f77bhP8lkkmvXrjEzM0M2m9XDjEwmE4FAgFAo1DKT/oqdUalUSKfTxONxJiYmqFarjIyM0N3d3XLRDnsqlvl8ntOnTzM2Nsb169dJJBJYrVZ9mGUwGHC73Vy8eBGr1YrT6cRqtRKJRHA4HDu6ttVqxW63EwqFGBkZweFw4Pf7VazefcBgMNDT00NbWxtf/OIXefjhh3E6nXg8Hrq7uwkEAtjt9jtOrywuLnLx4kWmp6epVqtqfvIAUyqViMfjrK6uUi6XsVgsDA0NMTg42HLRDnsqlqVSifPnz3PhwgXi8bi+Ono7DAYDDocDs9lMT0/PjsXS4/Hg9XoZHh7G4/Hc1bNR7AyDwUA4HCYcDtPe3s4TTzyBxWLRd944nc47evdSSuLxONevX2d1dVWf41YcTCqVCqurq2QyGSqVCjabjb6+Pg4dOoTT6Wx28zaxp2Jpt9s5efIkfr+fxcVFcrkcDocDq9VKuVymUChQLpfJZDLUajUymQxSSv3L1Cb7tQWA26ENs7XtcJVKhWq1SiqVwmazYTQaGRsbIxKJ0NPTo+ZC7zNWqxWPx4PRaMRkMumLd7fSaDSYmZkhHo8zOTlJKpUin88jpcRqteL1enUBVsPwg0M8HtdzAthsNrxeL8FgkGAw2HI23lOxdLlcPPvss6yurjI9PU0mkyEcDtPW1sbq6ipLS0uk02mmpqZIp9OMjo6SzWZZWVmhUqnomWe0OLzbfqD1xQJthVXbEaBRKpUIhULkcjlOnjzZcr3XQcPhcGxpVNBoNLh69SpXrlzRF/8ajQYANpuNrq4u+vr66Ovro7u7u+VuJMX2WFhY0Bd2bDYbfr+frq4uIpHIgz1naTQaaW9vx2q1YjQaKRQK+Hw+3G43brcbj8dDPp8nEAiQzWYJBAIUCgWWlpaoVCqEQiHcbjeJRIJEIvG+8wshMJvNGI1GfD4fFouFCxcuMDExsSlUyWazYbVa1UJPC6ClaMvn84yPj3P16lVWVlZ0W2le5fDwMNFoFKfTiclkUrbb52ibFnK5HMlkklKppK9bmEymlrTxnoqlxWJhYGAAKSWNRkOP1DcYDPpj7bVqtUo2m6VUKjE1NUWhUKC3txe/38/8/DwLCwvvm8syGAyYzWZMJhPhcBir1cp//I//kcXFRcrlMsViEavVqmcxUfGbzadarXLz5k0WFxd55ZVXeOONN/S94zabDZfLRX9/P5/5zGeIRCIEg0E1dXIAyOfzZDIZFhYWmJycxGAw6LbV5rdbjT0PSt/qgkqj0dADlEulkh6k7PV6N+0t3ogQQp8X2xj1r4Wd2Gw23G63WtxpIRqNBvl8nnQ6TTab1RMoAHp2ofb2doLBIG1tbbsWQqZoLuVyWR9RaNEOmqPTqk5My/7yDAYDdrsdm82Gw+Gg0WhgNpsxGAx0dXXR0dFx2+OEEFSrVa5evarHdObzeXw+H5FIhJGREX7mZ36mpTboP8jU63U912U6nd7UEYZCIZ544gmOHDnC0aNH9akVxf5neXmZ0dFR5ufnaTQaeqigy+VSYrkdtC/tVg/wbjeMNpRPp9P6iru2+6O9vZ22tja8Xm9LG+VBoF6vk8vlSKVSxONxEomEHuFgNpv1RBuRSIRQKKTH3CoOBpVKhUwmQ6lUAtbuce2+bNURX0uL5b2iDelSqRTf+973eOedd5icnARgaGiIX/qlX+Lw4cP4fD6sVqsSyyaytLTEn/3ZnzEzM8M777zD0tISKysrAPT19TE4OMjJkyf53Oc+pyfaUBwc0uk0MzMzJJNJpJQEg0F+/ud/nt7eXjweT7Obd1sOnFhqCRymp6e5ceOGXqrA6/XS399PR0eHKknQAhQKBa5fv87ExATj4+OkUilqtRpCCLxeL93d3fT29tLf36/Hxyr2P9rIr1gskk6nKRaLCCGw2+10d3fT1dXVslMtB0osi8UiZ86cYXZ2lrm5OV0o7XY7bW1thMNhvF5vy4UkPEjUajU9eUIsFmNqaop8Pq9PlZhMJo4ePconP/lJent7cTgcdwxkV+wvpJSsrKyQTqe5ePEiP/3pTykWi0QiEfr7+zly5IgexdKKHCixLJfLTE5OMjk5yerqKqVSCbvdru8L93g8OBwOdeM1kXq9TqFQ0DcbrKysUC6XqdfresKNSCTCsWPH8Hg870vPpti/SCnJZDKsrKwwOzvLjRs39OiUQCBAZ2cngUAAs9nc7KbelgMhluVymXg8ztzcHG+//TYTExMkEgkMBgMnT57Uix6Fw2Hsdru6+ZrI3NwcL730EpOTk3o9Hi0zUW9vr566Tdu8oGx1cNA8y4mJCZaXlykWi3i9XgKBAD6fT99Q0qo2PxBiWSqV9PCTd955h/HxcWq1GiaTiccff5zPfvazdHd3Ew6Hm93UB565uTm+853vsLCwwMrKir4aqiVMGRkZoa+vj7a2tia3VLHbNBoNVlZWiMViekcphNBzV7b6WsKBEMtyucz8/Ly+Uwegra0Nl8tFR0cHoVAIl8vV5FY+2ORyOVZXV5mfn9fnrTSP0ufz4XQ6GRkZ4bHHHqOzs7PZzVXcJ7TtjcViEQC3201fXx/hcLilhRIOiFjm83muX79OLBajUCgA0NPTQygU4tChQwwMDLS8IQ468Xicq1evMjo6yvT0tJ7U12q16l7/xz/+cT7xiU+oMKEDipSSZDLJ7OysngjH7/dz/Phxurq6WnauUmPfi6UWLqQlENW8FW3uS9vW2KrzIAedRqNBo9HQa4cvLCxsSugrhNATA2slRtSWxoOHVgc+m83q96nT6cTn8+l5T1s97nlf/yrr9Tq1Wo1UKsXVq1eZm5ujUqngcDh48skn+fCHP8zQ0JDyKptIuVymVCpx9epVXnjhBZaWligWi7pYaklPenp69M0CqmM7WDQaDb143fT0NNeuXcNkMtHZ2cng4CAnTpzAbre3vGfZ2lL+AZTLZT1dWzqdplAoYLFYNoUjtEoZzQeVfD6v2ygej+sJnY1Go+5RhkIhOjs7VVjXAaXRaOgepZYsRQiBy+XC6XTqSb1bnX3tWc7MzPDGG28wOjrKjRs3KJfLDAwMEA6HGRkZ4fDhwy27G+BBQErJlStXOHfuHG+++Sbj4+NUKhVqtRput5ujR4/S2dnJpz/9aYaHhwkGg81usuI+UKlUeO+995iamiIWi5HNZunu7mZoaIhwOLxvOsh9KZaNRoN6vU46nWZ6eprFxUW9t/L5fIRCIX0OTNE8tJIg09PTrKysUCgU9OG3xWIhFArR1dVFZ2envg1VcfDQEjxrdbeq1SoGgwGXy9Wyu3Vux74Uy8XFRebm5jh9+jQvvPACmUwGWFtZ+7mf+zkGBwdVTGULIKXk2rVrfP/732d1dXVTlcb29nY+97nPMTAwQE9Pj0pscoCp1+ukUqlNmaW0lGwWi0V5lveTbDbLwsIC09PTjI2NIaXUs2pHo1EGBwdVbZ0WIZFIMDk5+b5ytna7naGhIQ4dOoTb7d71RTgtYcPdKkMaDIZ9c6PuZ6SUlMvlTYl+taoG+2GuUmNfiWW5XKZWq/Hee+/x4osvcvPmTer1Ok6nk6GhIaLRKAMDA3oCBkXr0mg0KBaL5PN5zGbzHbPfw9qNpXkgmveplSG5ta54rVajUqmQTqe5cuWKHvy8Ea0CaH9/P8eOHdMrTyruD7VajVgsxujoKKlUCoBwOMzDDz9MV1fXvumw9tUvpFqtUiqVmJiY4I033tiU2Le7u5uenh46OzvVQsE+oNFoUC6X9dCiWz3PjWilQgwGgy6WG2s1bSyLXK1WyefzLCws8NZbb92xCqhWJnlkZASLxaLE8j5Sq9WYn59namqKbDarry1Eo1H8fr8Sy91ESkmtVuOdd95hbGyMc+fOkU6nkVLi9Xrp6uriIx/5CL29vWr4vU9IJBL8xV/8BX6//wMD0V0uF729vXr2KIPBoC8WaQlkNbSqgel0mmvXrul7zzeiCa7ZbKarq4tgMMjw8PC+GhLuB6rVKslkkvn5eT1k6G6dYquzb8SyUqnwxhtv8OKLLzI3N0cymcTj8eD3+4lGozz99NOEw2Hcbnezm6vYAktLS3zzm9/c0nuDwSBPPPEELpdL35GVTqcplUqcP3+emzdv3vY4rfDZnajVarS1tXH48GEGBweVWO4ylUqF2dlZZmZmSCQSm+Ys9yMtL5ZaVmUtEYMWfgBrK6qPPvoow8PDeuIMtaLaWng8Hjo6OshkMvp8lcZWb5xiscjs7Cw2mw2n04kQgmKxqHuQG7dOamhDdqPRSFtbGxaLBYfDsSlUZWBggGAwiMfj2TdDwf1EpVJhYWGBhYUFKpWKvsXRbDbrFTvdbve++e5bXiwbjQapVIrV1VVmZ2eJxWL6Cmd/fz/PP/88XV1ddHd3Y7PZ9s0X/yAghKCzs5Njx44xOTmpT53cK5lMhgsXLui2FUJsqjGvPaflANj4G7Db7YyMjBAIBOjp6dmU+i0SiTA0NER7e7uas7wPFAoFrl27xsTEhB7ep5U2Hhwc5PDhw/sqb0PL/kK0ecpiscjk5CQLCwv6Bnyfz6fXaeno6NB/7PvlS3+Q6Ojo4KGHHsJkMpHL5e4YzqOtbGs217JHbXzNZDLhdrv1kJONowi73Y7T6cRkMm3yHm02GyMjI/h8Pjo7OzdN0wQCAX1Eon47u4u2+JZMJlldXaVSqQBrKdk0j3K/ZcFvWbGs1Wokk0mWl5f5r//1v3L58mW9UuOxY8f46Ec/yqOPPsqJEyfUamaLYjAY+MQnPsGHP/xhzp8/z+uvv64nP7kVKSXLy8tkMhnGxsYYGxt733scDgcnT57UvZONotjf38/w8DBut5tQKKQLqRZ2pIUHbRRYo9Goi65KtrJ7NBqNTeFbExMT5PN5jEYjg4ODHDt2jO7u7n03ZdayClOv18lkMiSTSRYXF5mdndXnKt1uN5FIhEAggMPhUELZwrhcLlwuF5FIhGg0Sr1ev+1cZaPRwOFwkMlk9JCiW/F4PESjUV0sN+a9jEajRKPR94mlonloolmpVDCZTDidTvx+P+FweF8m425Zlclms7z55ptMT08Ti8VIJpNUq1WEEASDQY4cOUIkElE3xT5hYGAAv9+vD8FvNxSvVCp6QbONw3ANo9GI1+vFbDa/z0u02+16JUj1m2gu2vyx2WzGbrfj8/kIBoNYrVaefvppTp06RXt7e7Obec+0pFhqCX0XFhb0kraapyGEwGq14vV6VfGxfYTT6VQxsA8QmmC63W68Xi9ut1sfYUQikX0ZptVyYpnL5VhYWGB8fJzXX3+d2dnZ94WcKBSK1kUTyq6uLr761a9SLBb10cDAwAAWi2Vfev8tJ5ZaWduFhQUmJiaYnZ297f5ehULRuggh8Hg8nDp1qtlN2TVaTiyLxSIzMzPMzc1RKBSoVqv6/JbL5cJms+F2u9UKuEKh2FNaTm2KxSJzc3N6rRYtPmtjYSstD54SS4VCsVe0vNoYDAbcbjc2m41Tp05x6NAhjh8/js/nUzVbFArFntHyYmk0GvH7/QQCAf7W3/pbPPvss7hcLiWUCoViT2k5sXQ6nUSjUaxWK8888wyFQoFgMIjX66Wnp0ffiK+EUqFQ7CUtJ5bhcJhf/MVfpNFo8KUvfUkvmyqEwG6376uaHQqF4uDQcmJpNBr1Wt8qiFmhULQKYjsps/SDhVgBpnavOS1Pn5TygalZ8QDaF5SNHwS2ZeMdiaVCoVA8KOy/PUcKhULRBJRYKhQKxRZQYqlQKBRb4K5iKYTwCyHeW/+3KISY2/DYstuNEUL8wYbzjwkhPjDdkBAiJoS4JIS4KIR4WQjRsYPr/1shxD/7gPf8nQ1tfE8I0RBCHN/uNZtNE2z820KIq+v2elUI0beFY/baxs8KIc6uX/OsEOKZ7V6vFWiCjT8uhDgnhKgJIb6wxWP22sZ+IcRrQoicEOK/bOW8dxVLKWVCSnlcSnkc+CPgD7THUsqKEGJXQ4+klL+14Xr/L/CXWzz0aSnlMeAM8C83viDW2DUPWkr5pxva+BVgUkr53m6df6/ZaxsD54ET6/b6c+D3t3jcntkYiAO/LKV8BPhV4E928dx7ThNsPA38PeB/3ONxe2njEvBvgLuK6kbu+eJCiP8mhPgjIcTbwO/fquJCiMtCiOj6339XCPHOeg/2NSHEvRQ6+dvAt+6xeW8Ah4QQUSHEdSHEN4HLQI8Q4p8LId5d77l+b0N7/9W6F3saGL7H6/1t4H/e4zEtz/20sZTyNSmllgb9LaD7Hpt3320spTwvpZxff3gFsAshrHc7Zr9xn20ck1JeBO5euP3O7IWN81LK06yJ5pbYrlJ3A6eklL99pzcIIY4AXwI+ut6j1YG/s/7a14UQJ+5ybB/QD/zwHtv1KeDS+t9DwB9KKY+y9uUNAR8GjgOPrw8VHge+vP7cJ4GTG9rw60KIX/+A632Jexf0/cJ9tfE6/xB48R7btdc2/jxwTkr5/qJA+5+9sPF22Gsbb4ntut/fllK+v+rUZn4WeBx4V6xtT7QDywBSyn/0Acd+GfjzLVxD4zUhRB24CPxrwAdMSSnfWn/959f/nV9/7GLtS3cDf6V5OkKIF7QTSin/6G4XFEJ8BChIKS9vsY37jftqYyHE3wVOAE9tsT3NsPFR4N+vn/cgcr/v43tlz218L2xXLPMb/q6x2UPVSu4J4L9LKf/FNs7/ZeD/uIf3Py2ljGsPhBC+W9oogH8npfzaxoOEEP90G23b2MaD6lXCfbSxEOLngH8FPHUPHtue2lgI0Q38FfArUsrx7ZxjH3C/7+N7pRn38ZbZjQnTGPAYgBDiMdaGzwCvAl8QQoTWX2sXW1v5HAHagDdvef7aDtr4EvAPhBCu9XN1rbfrDeCzQgi7EMIN/PJWTrY+0fxFDuB85R2IsUs2FkJ8CPga8Gkp5fItr7WEjddv0v8F/I6U8ic7aNN+IsYu3sd3olVsvB12Qyz/AmgXQlwBfhMYA5BSXmXNlX5ZCHEReAXohA+c6/gy8D/lhn2YQogAa73KtpBSvszaytybQohLrK3CuqWU54A/Ay6wNnf27oZr3m2u4+PAjJRyYrtt2mfspo3/A2vDp2+vLxi8sP7+VrLxbwKHgN8VfxNiE9pu2/YJu2ZjIcRJIcQs8DzwtfVztpqNEULEgP8E/D0hxKwQ4qG7XX9f7A0XQnwKGJBS/udmt0Vxf1A2PvjsdxvvC7FUKBSKZqO2OyoUCsUWUGKpUCgUW0CJpUKhUGyBHe0JDQQCMhqN7lJTWp9YLEY8Hn9gCgA9aPYFOHv2bPxBypSubLx1diSW0WiUM2fO7OQU+4oTJ+7Hzq7W5UGzL4AQ4oEqsaBsvHXUMFyhUCi2gBJLhUKh2AJKLBUKhWILKLFUKBSKLaDEUqFQKLbAbqeTvy80Gg3q9TrT09Mkk0kSiQTJZBKr1YrT6cTn8zEyMoLVasVqtWIwqD5AoVDsLi0vllJKqtUqpVKJn/zkJ5w7d47z589z/vx5/H4/3d3dPPzww/zGb/wGgUCAQCCgxFKhUOw6LS+WgC6Wy8vLTE1Nsby8TC6Xw2g0YrVaWV5eJpFIYDKZ8Pl8mM3mZjdZcQfq9TqlUoliscjc3BwAAwMDuN3ubZ2vWCxSLpcxmUyYzWaMRiMm0774WSs2UKvVyOVyFItFZmdnqVarhMNhHA4HHo8Hp9PZ7Ca2vlg2Gg2y2Syrq6u89957vPbaa5TLZYQQ5PN5YrEYVquV9957j56eHsLhMHa7vdnNVtyBQqHA7Ows4+PjfP3rX6fRaPB7v/d7fOhDH7rnc0kpmZubY35+nra2Nvx+P3a7HZ/Px3oJBMU+IZvNcunSJaampvjDP/xDkskkn//85zly5AgnTpxgeHgYIURT7drSYtloNKhWq8TjcZaXl0kmk+Tza1nmDQYD9XqdWq1GuVzWPQyVcq61qVarZDIZUqkUCwsLAJTL268Fls/nSSQS1Go16vU6Pp8Pr9erxHIf0Wg0KBQKzM3NMTMzw+zsLPF4nFgsht1u5/Dhw/p9rcTyNtTrdfL5PMlkkr/8y79kdHSUGzduNLtZih2SyWS4fPkysViMfD6P2WzedgcnpWR6epqzZ8/qHeexY8f44he/iNV6oCrXHliq1SrFYpHx8XH+9E//lLm5ORKJBOVymR/84Ae8/fbbdHd3c/z4cYxGY1PXI1pOLBuNBrVajWq1qg+/Y7EY4+PjZDKZ2x4jpaRer1OtVqlUKlQqFQwGA0II/X9Fa1Aul0kmk6TTaWq12o7ml6WUlEolstks2WyWXC5HZ2cn9fpWi4Iqmo02h53JZJienmZxcZFqtYqUkkQiQS6XI5/Pt8SIseXEcmVlhYsXL5JMJrl27RqJRIKzZ8+ysLCgD8FvReuZMpkMVqsVn89HMBjE4XDQ0dFBW1vbHn8Kxa1IKZFSkk6nuXLlCktLS5TL5R3NLwsh8Pv9RKNRxsbGGBsbIx6PUyqVMJvNaqFvH5BOp7lx4waTk5PE43EymQy1Wg2TycTw8DChUIhIJILZbG56lEvLiWU+n2d8fJzFxUXOnDlDIpFgdnaWbDZ7x2MqlQqJRIJGo8HExAQej4dKpYLX68Xj8SixbAGklPrc1NLSEvF4nEajsePzOhwO2traEEKQyWTI5/P6/KXJZFKjihanVCoRj8dJJBL6ugOsrUkEg0F6enrweDwYjcYmt7QFxXJxcZEXX3yRRCLB0tISpVKJUql012Py+Tyjo6PYbDZisRg2m42Ojg48Hg/PPvssUkrcbjder3ePPoXiVtLpNMvLy0xMTBCLxWg0GoyMjBCJRLYdNgRrHWWhUCCTybC8vMzq6iqVSoVqtYrFYlFi2aJonefMzAyvvfYasVhs00Kf0WhkYGCARx55hGCwNdKLtpxYxuNxTp8+zerqqj7n+EHk83kmJib09xoMBvx+P263m0gkQldXF0IIJZZNJJfLMTMzw9zcHAsLCzgcDgYGBujr69tRDJ0Wg5vNZkkkEmQyGSqVipq3bHG0tYmVlRXeffddEokE1WpVf91kMtHX18eRI0daZmTYMmI5OTnJlStXeOedd/QJXuCuQzWj0aj/M5lMSCkpFov6cK/RaPDmm2+SSqV46qmniEQiytPYYwqFAsVikatXr2cbhO4AACAASURBVPLqq6/q4SDhcJhjx44RjUa37VlqiwCTk5Mkk8ldbrniflIoFMhms6ysrLCyskI6nd7UwRmNRkKhEL29vTsaeewmLSOWly9f5hvf+AZzc3P6sPuDVsCMRiMulwuj0YjNZtNjLiuVCplMhkwmw4svvsj3v/99DAYDv/ALv9AScx8PEplMhng8zttvv82f/MmfIITA5XLR09PDk08+STQa3fYij5SS+fl5Ll++zOLi4q7MgSr2hmw2y/z8vB5buXEIbjAYsFgs9Pb2MjIy0jIOTtPFUvM8tHCSjWECd/qSfD4fPp8Pj8eD3+/HYrFgs9moVqvMzs5SKBRYXFwkl8vpvdXq6ipTU1O43W4CgUDLGOAgI6VkeXmZmzdvsri4SLlcJhgM8vDDDzM0NITb7cZsNm/LFlqomCbGuVzuPnwCxf0im82ysLBAKpXa1MkZDAba29vx+/1YrdaWuk+bLpZLS0ssLCwwOTnJwsKCHkt5p61NBoOBQ4cO8eijjxKJRBgeHsZiseBwOPThXjwe58UXX2R8fFw/bnp6mh/+8IcMDg7ysY99TIWV7AFSSq5cucJLL73E2NgYpVKJrq4u/v7f//t0dnYSDAa3FTyuxVdqWyevXbtGo9FoiVg8xdaYn5/nzJkzxGKxTXazWq0cOnRoxwt/94OmiaUW3rG0tMSNGzdYXFykWCxSrVbf19NoISBOpxOz2Ux3dzcDAwOEw2EikQgmkwm73U6xWCSdTmM2m/X3ajdRKpViYmICs9nM0tISTqezZUISDhpSSgqFAuVyWd+qWigU9JR6gUCAtra2bSe8aDQaJJNJVldXyeVy1Go1fXRht9sxmUxNj8lT3J1isajbb+P9brVa6ezspKenp+VyPDRFLKWUZDIZcrkc3//+9/nOd77D6uoqy8vL7/MQDAYDXq8Xu93O0aNH6erq4sknn+TjH/84VqsVl8ule6HVapVQKMTy8jI//elPmZub02/a0dFR5ufneeihh3QP52Mf+xhut1sJ5i5Tr9cZGxtjYWGBM2fOcO7cOaxWK6FQiK6uLgYGBnaUHapSqfDTn/6U69evMzExAYDf76evr49oNIrL5cJutyvBbFGklKysrDA6OsrCwoJ+v2tZw5577jmGh4eJRCJNbulmmiaW2pbG1dVV5ufnKZVK1Go1vZcRQmCxWLBarQSDQdxuNz09PfT09NDV1UUwGNTTcmmYzWZ8Ph+1Wk1PqFCtVjcl2mhra2NqagqDwaDvIFFbIncXrTNcWVkhlUqRz+f1nVVutxu73Y7FYtnWubXkKisrK8zNzZHP5/VFo2AwiM/nU55lC6Pt4S8UCqTTaYrFoi6WBoNBv99DoVDL7e9v2jBci7MqFotkMpn3rWTa7Xb6+/sJBoN86UtfYnBwkGAwiMfjwev13nby12g06h7Lk08+SSAQ4Ec/+hGXLl2iXq9Tr9eZnJzk29/+NkePHuVDH/oQUkra29u3ffMq3k+tVmNsbIzz58/rK52RSISnn36ahx56aNvD73q9TiqVIplMcuHCBd566y2WlpYwGo0cPnyYz372s/T392Oz2dRooQWRUrK4uEgymdS3p27csWO1WnE4HHR1ddHT04PNZmtyizfTNLHU9grXajVqtZr+vMFgoNFoYDab8fv9dHR0cPToUR566CGcTuddexvNG5VS0tnZSalU4r333kMIoSfbyGQyZLNZPB4P+XyeSqWiFgZ2mUajQSqVYnFxkXw+T71e18OFQqHQtoVMi5/NZrPE43F9h5fBYMDn89Hf3084HFZC2aJIKcnn86RSKVKpFOl0Wn/NaDRiNpv1ee1WSPZ7K01fDb8Vu92O3++np6eHL3zhC/T09NDf368v2GwFbauU2+3m8uXLzMzMkEqlSCQS97n1DzZSSn374fz8PLFYTL8hurq6eOqpp/Rh8nYoFAqcO3dO3wmUy+VwuVz4fD56e3sZHh7G4XCoIXiLIqXU81SmUqlNr/l8Pj7ykY8wMDDQkkIJLSiWZrOZ9vZ2urq6OHnyJD09PXrM1VYxGAwEAgFsNhvhcBi/368n29DQvEnNw1XsHK1eUrlcJpVK6RmAANra2jh8+PCOSj5Uq1WmpqaYnJwklUpRLpdpb2/H6/USCATo6OhQQtnCSCn1banFYnHTa06nk8HBQfr7+1turlKjJcRy4wp4X18fv/qrv0p3dzfd3d243e5t32CaEG78p5HNZnnzzTdZWFjgmWeeabn5kf2KNhedz+fJZrO0tbURjUbp7Ozc8SKaNo2SSqWoVqsYDAbcbjfBYBCXy7VLn0Bxv9BC+Obm5vR4aoPBoDtIH/rQh+ju7sbhcDS5pbenJcRyo4j19fXxla98Zdc2z99OKLW8im+//TaLi4s8/vjjhEKhXbneg4w2L6ytduZyOQ4dOsTw8PB9EUsAl8ulF7ZStDaaWC4sLOg7rkwmEzabjUAgwKOPPkpHR0fLxVdqNEUsG42GXmhqdXX1vpx/eXlZXwSIx+MUCgX9dS3A/ciRI/p8qGLnCCEwmUxYLBY8Hg/t7e3U63WSySTZbJZSqYTFYrnnkYIWLpTL5ZidnWVqakpPBO12uwmHw3g8HhX+1aI0Gg3S6TS5XI6pqSnGx8f1+17bB66thLfynHNTxLJWqzE6OsqlS5eYn5/f9CPfjR98vV4nFosxOTnJ+Pg4s7Oz+h5xLYC9ra2Nn/3Zn6W/v5/29vYdX1PxN9EIdrtdz3CtJbtIJBJ6Jiin03lPdq7X6+RyORKJBFeuXOHq1at6ct9gMMjg4KCyYQtTq9WYm5tjeXmZixcvcubMGf1+1JLgOJ1OPd9Dq9K0oPTV1VWWlpYoFAo77km04V+9XqdcLpPP55mdnWViYoLV1dVNiTm0dG4WiwWLxbLtRA6K96PlH914A2i1cWZnZ7lw4QI+n49IJHLX8B6tllK9XtdrKqVSKWKxGIVCYVMqr421lhSti7bwV61WN4UKWiwW2tvb98XW46Z5lhMTE5w7d47l5eUdn69er+vDvIWFBRKJBK+88grvvffe+/Icmkwm3G63HopkNBqVWO4iWqxce3s7oVCIRCLB3NwcP/jBD5ienqa/v59nnnnmriue8XhczySUTCYpFAp6Yt94PK6/T7Ob2oHV2jQaDYrFIrlcblOCX1iLknj44YcZGBjYUaTEXtC01lUqFb2n2Y1zaZXgYrEYKysrevEj7fyaKLrdbrq6ugiHw9hsNlWn5T6gBYmHw2Hm5+f1XKOLi4tYLBYmJyfvumNqdXVVT7KwurpKqVQikUiQz+epVqub7GW32/F4PCqaoYXRgtG1LPYa2tqBtk211UcHrS3lW2RlZYXvfe97zM/P86Mf/YhEIkE8HtezpsNaYSuXy8WxY8f4/Oc/TyQS0ZMutLr7v98wm82cOnWKQ4cO4ff78Xq9LC0tMT09zcLCAhcvXvzA441Goz69oiVzrtVq77vZDh8+zDPPPKMnVFG0HpVKhatXr+rpEwF9UScajfLMM8/Q0dHRsvGVGvtaLLX6K6urq3rG5TuVGDAajdjtdnw+HwMDAwSDQT2dl2J30RK4ms1murq6mJ+fp1qtsrS0pO/vvttGALfbrYufyWTS0/nVarVNx2meSXt7u7Jji6JtaU4mk6ysrOibFLTUiy6Xi1AoRFtbm/Is7ydXrlzhu9/9LsvLy1y6dEnPYnI7tOJl0WiUw4cP4/F4VALg+4Qmlh6Ph1/8xV/kiSeeYGFhgampKT2P4d0Kig0MDBCNRnVPcWpqihdeeIGlpSWuXr26KSu60WhUVRxbFC0udnl5mQsXLnDmzBndkdG2NXd0dOjhe60+wmsJsbw149CtQeTajXWrN7KwsMC7776rJ224XfJgDS1FmM/no729vWUDXw8K2hyitrNGS62Xz+f1vKV34qGHHmJkZER/PDo6yoULFxBCMDY2tum9aiW8dWk0GpRKJXK5HEtLS8zNzemvaSFmLpdLzyLW6rSEWG4UwXQ6zfXr1/WytYVCgZ/85CcsLS3pYqgxMzPDpUuX9HyVWvjQ7YZ4x44d41d+5Vfo6upSHmUTcLlcdHV1UavV6OzsvOsw/NbdW2azmba2NrLZ7CbbaTux5ufncblcLR2j9yBSLpeZnZ1lZmZGH35rhMNhjh49Snd3977p7JoqlrdLYqHVVdGykqRSKb7zne9w7do1MpnMpp04tx57t6HYwMAAn/zkJ1ve1T+o2Gy2ba9YG43GTfOYG8nn8yQSCQwGgxLLFqNareplRTZWbwT0tYP9VDywKWJpNBrp7+/n+PHj+lyjxuLiIt/97nd1t7xUKjE7O0uxWNwUzAroeSq1v29FCEF3dzd+v1/VDD8AbMwUpaFlSFdbVlsPrbBcqVTSp13MZjMmk4m+vj4ef/xx+vr6lGd5N4xGI8PDwwghiMfj3LhxQ39tdnaWb33rW8DfCOCtK6C343bPm0wmBgYGOHr0KH19fUos9zF3GrZ7PB46Ojr2uDWKraAFo28M4bNardjtdgYHB/noRz+6r2olNUUstdXS7u5ufD4fdrtdDw/ZmK7tdpP3Gx9rBtCyLGtVHi0WC9FoVN8d0N/fT2dn5959QIVCoQ/DV1ZW9PhYrWy1zWbDarXuq5CvpnmWhw4dIhQK8eabb+JyuSiVSnomme2cz+VyYbPZiEQitLe385WvfIWjR4/qRbJsNpvyLPcxd5pmUTZtXYrFIjdu3GB8fFy/t10uF21tbXg8Hux2+75aQ2iKWAohsNlsNBoNIpEIhw8fJpPJ6JlpPiho+VasVqtelH1oaAi/309XVxd+vx+n07nvjKJ4PxsXA7URhxLK1kXbfZXP5/X1Bs1u2g6t/WbDpomlx+PB6XTyC7/wCwwODjI1NcW1a9eYmprixz/+8ftWz+5GIBDgueeeo6uri2eeeQa/34/b7cZisegG2U9GUWxGK1excYFPC0ZXnWDroQlluVzWc8pqIX8mk0n/t9+S2DRtwsBgMOhzl729vUgp9VjJcDhMoVCgXC7TaDSoVCp6xUetVvjGRAzhcJjOzk4ikQidnZ27lmVd0RrUajWy2Sz5fB4ppZ45yuVy7Ytg5gcZLWWfJoobO7n9JJTQAkHp4XAYn89HNBrl5MmTLCws8Nhjj7GyssLZs2dJJBJcv36dfD5PNBolHA4zMjLC8PAwsNaLtbW1cfToUdxut9qZcwBJpVK89dZbLCwsUK/X8fl8PPXUU0SjUQ4dOtTs5ilugzbV1tvbS61W03PXer1eOjo69mXNpKaLpdVqxWq16gHFTqeTSqXC4uIiy8vLWCwWlpaWEEIQCoXo6uri0KFDPPLII/oclsPhIBgMYrPZ1LDsAFKpVEgmk6TTaT30pLu7m8HBQRWI3oJo014mk0nfXmyz2fQtjg6HY1/uomu6WN6Kx+NhZGSEaDRKf3+/nlWoUqnoYUZ+v3/TUNtoNOJwOPQs6IqDhd1up6enh1AoxPHjxwkEAnzkIx8hEomoQnMtisFgoK2tjWeffZalpSVgLYZ6ZGSEzs5OfD5fk1t477ScslitVsLhMLBW6VGh0EqlCiE4ceIEkUiEI0eO4Pf7m900xV1wOBwMDw8TCoW4du0aNpuNzs5O/H7/vqzG2XJiqVDcSmdnJ88//zwAhw4dwuVy7cub7UFDi382mUw888wzpNNp3G43DoeDSCTS7ObdM0osFS1PMBjkU5/6VLObobhHDAaDXt721KlTzW7OjtkfmzIVCoWiySixVCgUii2gxFKhUCi2gBJLhUKh2AJKLBUKhWILiHvJ7vO+g4VYAaZ2rzktT5+UMtjsRuwVD6B9Qdn4QWBbNt6RWCoUCsWDghqGKxQKxRZQYqlQKBRbQImlQqFQbIG7iqUQwi+EeG/936IQYm7DY8vdjt0OQgirEOLPhBA3hRBvCyGiWzimvt6ey0KIbwshtr1pWAjx34QQX/iA9wghxH9eb+NFIcRj271eK7DXNt5w3c8LIaQQ4sQW3runNt7w3pNCiNpW39+qNOE+/rgQ4ty9fHdCiJgQ4tL6PfWyEGLbJTuFEP9WCPHPPuA9fiHEa0KInBDiv2zlvHcVSyllQkp5XEp5HPgj4A+0x1LKihBit/eW/0NgVUp5CPgD4N9v4ZjienseBirAr2988T608TlgaP3f/wb8f7t8/j2lCTZGCOEG/gnw9hYP2WsbI4Qwsvb7e3m3z73XNMHG08DfA/7HPR73tJTyGHAG+JcbX1h3UnZzJFwC/g1wV1HdyD1ffL1n/iMhxNvA79+q4uu9f3T9778rhHhnvQf72voP8G58Bvjv63//OfCz4t5yz/8YOCSE+IQQ4sdCiBeAq0IIoxDiPwgh3l3vub663j4hhPgvQojrQogfAFtJjvgZ4JtyjbcAnxDiQNXZvc82Bvi/WROi0jaatxc2BvjHwF8Ay9toY8tzP20spYxJKS8CjW027w3WbBxdt9s3gctAjxDin2+w8e9taO+/EkKMCSFOA8MfdAEpZV5KeZp7+A1uV6m7gVNSyt++0xuEEEeALwEfXe/R6sDfWX/t63cYfnUBMwBSyhqQBraUtHC9d3wOuLT+1GPAP5FSHmbNY01LKU8CJ4FfE0L0A59j7Yt9CPgV4NSG8/1fQohP362N68yuP3fQuC82FmvTFj1Syv91rw3aKxsLIbrWj9vXo4YtcL/u453yKf7GxkPAH0opj7JmxyHgw8Bx4PH1If/jwJfXn/ska/bX2v/rQohNI5Htsl33+9tSyvoHvOdngceBd9edQzvrvbSU8h9t87q3wy6EeG/97x8D32DthnhHSjm5/vzPA8c2zJ94WfvSPw58a/2zzAshfqidVEr5u7vYxv3Irtt4fRj1n1gbot0Le23j/wf4P6WUjXsb2Ow7Wuk+BnhNCFEHLgL/GvABU+sjOFiz8c8D59cfu1izsRv4KyllAWB9tMF6G/9otxq3XbHMb/i7xmYP1bb+vwD+u5TyX9zDeeeAHmB23YvwAokPOKa43uPprBt1YxsF8I+llC/d8r5P3kPbbm2jRvf6cweN+2FjN/Aw8KN1G3UALwghPi2lPHOX4/baxieA/7l+jQDwSSFETUr519s4Vytzv+7j7fK0lDKuPRBC+Hi/jf+dlPJrGw8SQvzTPWjbroQOxVgbDmlDrP71518FviCECK2/1i6E+KA6ES8Av7r+9xeAH0oppRCiSwjx6g7a+BLwvwshzOttOSyEcLI2N/Kl9fmuTuDpLZzrBeBX1ufCnmBt6Lewg7btB2Lsgo2llGkpZUBKGZVSRoG3gE9LKc+0ko2llP0b2vjnwG8cQKG8lRi7dx/fESHEtR208SXgHwghXOvn6lpv1xvAZ4UQdrG2ePjLO7jGHdkNsfwLoF0IcQX4TWAMQEp5lTVX+mUhxEXgFaAT7jrX8Q3AL4S4Cfw28Dvrz3ey1vNtl68DV4FzQojLwNdY86r/Crix/to3gTe1A+4yZ/k9YAK4Cfwx8Bs7aNd+YTdtfCdaycYPIrtmY7EWcjULPA98bf2cCCECrHmH20JK+TJrK+xvCiEusdaRuaWU54A/Ay4ALwLvbmjLHecshRAx1qeFhBCzQoiH7nb9fbE3XAjxm8C0lPKFD3yzYl+ibHzwEUJ8ChiQUv7nZrdlO+wLsVQoFIpmo7Y7KhQKxRZQYqlQKBRbQImlQqFQbIEd7QkNBAIyGo3uUlNan1gsRjweP9BRyht50OwLcPbs2fiDlCld2Xjr7Egso9EoZ87cLZb4YHHixP3Y2dW6PGj2BRBCPFAlFpSNt44ahisUCsUWUGKpUCgUW0CJpUKhUGwBJZYKhUKxBZRYKhQKxRZQYqlQKBRbYNdrlygUCsVOqNVqNBoNyuUy9fr7cxMbjUasViulUon5+XlqtRpOpxOTyYTNZsNsNmO327FarbvaLiWWCoWiZWg0GhSLRarVKslkkmKx+L73OBwO2tvbWVlZ4ZVXXqFYLBKJRHC73YRCIdxuN+FwmGBwd/cWtIxYNhoN6vU6uVyOubm1xOMOhwOz2YzP58NisWA2mzEYdjZzIKXUr1UoFAD0XumAlxBQKFqOSqVCPp+nWq2Sy+WoVCrE43HK5TLxePy2Yul0OgkEAiSTSW7evEmpVCKTyWCz2Whvb8fpdPLYY48dXLHUvrQLFy7wzW9+E4CBgQHa2to4deoUHR0dtLW14XBsu2Q0APV6nVKpRDab5ebNmwghOHLkCF6vF6PRqARTodhDEokE169fJx6Pc+nSJVKpFNevXyeTybC0tKQ7NBvRPMdKpcLc3BzVahUt1aTBYMBkMvE7v/M7PPLII7va1qaLpZQSKSXpdJq5uTmmpqaYnp5GCIHL5UJKSa1W2xURazQa5PN55ufnyWQyTE1NYTQa6e3txel0IoTAaNxKJVfFbqB5+blcjmq1qs9Vud1u3G53s5un2EXq9fqmIXa5XKZcLrOwsMDk5CTxeJxYLEY6nWZ2dpZsNksikbitWGq/l3q9TiqVolarbRJMo9FIPp9/33E7peliWalUqFQqvPrqq3zjG98gmUwyOzuLy+XSvT2Hw0EgEMBk2n5zq9UqlUqFM2fO8Md//MekUilWVlbw+/34/X4MBgPt7e079lwVW0ObwM9ms7z88svMzs6ysrJCPp/nl37pl3juuecwGAw7nnZRNJ9Go8Hq6ir5fJ5z584Ri8UYHR3l+vXrFItFstkslUqFQqFArVajWCxSr9ep1W5fZaRUKrG4uAj8zWLQXiQxb6pYSikpFovk83lmZ2e5fPkylUqFarWKzWbDaDRiNBqxWCxYLJYdXatarZLP51leXubKlStkMhkqlYpunI09k+L+oY0UarUamUyG1dVVYrEYExMTLC4ukslkOHHiBI1GQ02JHBCklJTLZQqFAnNzc9y4cYNLly5x4cIFGo3Glu89IQQGgwEhxCYv8tbR4P0aITZNLIvFIpVKhddee413332X8+fPk8lkCAQCPPTQQ/T29vL5z3+ejo4OIpHIjq83Pj6uX2dxcRGXy8Vzzz1HT08PQ0NDhEKhXQ81ULyfZDLJ2NgYS0tLvP7666ysrDA6Osrq6iqlUol6vc7i4iLFYhGLxYLdbm92kxU7pNFo6MPqc+fOcfr0aVKpFNVq9Z68QpfLhcfjwel00tbWdsfO1Gg00tHRsZsfAWiSWEopqVarlEolxsfHeeutt5idnaVSqWCxWIhGowwNDfH444/v2opWIpFgdHSUqakpcrkcPp+PkZER+vr6CAQCOJ3OXbmO4u4UCgWmp6eJxWL86Ec/YmVlhUQiQblcBta8Am1VVA3B9xe3ip4mZhs9y4WFBW7evHnb4zeK30bba8/b7XZ8Ph9er5dIJHLH34fBYMDj8ezos9yOpollMpkkmUwyNTXF+Pg42WwWKSWhUIhPfOITdHV17cr8YSaToVAocPXqVd544w2q1SqHDx9mYGCAY8eO0dXVpYRyD6hUKhSLRSYmJvje977H4uIii4uL5PP5TYHHUkquXbvGX//1XzM4OMjJkyexWCzK629htKmzubk5JiYmsNvtBINBHA4HXV1dwJqAaVNqNpsNk8mE0WjUh+FOp5POzk5sNhs+nw+r1Yrf78fhcOD1enE6nTidTlwuF3a7Ha/Xe9dpmiNHjuz652yKWDYaDVKpFEtLS8zNzTEzM6O/1tbWxsmTJwkEAthsth1fK5/Pk0gkmJyc5MyZM3R2dvLYY48xNDTE8PAwoVBIDfX2gEqlQi6XY2Zmhh//+Mesrq6SyWRoNBrAZq9icnKS1157jWKxyNGjRwGwWCxqDrNFqVarFItFpqen+elPf0pbWxtDQ0O0t7cTCoX0GGaj0YjJZMJqterrENVqVfcEBwcH8Xg89PT04HK5GBwcpK2tje7ubgKBAGazGZPJpJ9jr9lTsZRSUqlUKJVKzM7O6qECAL29vQwNDfHYY4/hdruxWq3bHoZJKcnlcpTLZd5++20uX77M5cuXkVLicrno7++nu7sbu92O2WxWN+EesLKywpUrV7h58yaFQoFyuXzHuSot9q5er1OtVgkEAoyMjOB2u+nr68Nut6shegtQr9ep1+tcuHCB0dFRbty4wcWLF4lGo/T19ekdodFoJBAIYLVaefrppwmHw9jtdpxOpx4R4fF4GBgYwG6309bWhtVqJRgM6kNvzeZGo7Fptt9zsSwUCuRyOcbHx7l69SrxeByAoaEhnn/+eQYGBnQ3fLtooQqpVIqXXnqJ7373u+RyOaSUeL1ejh49qvdeO11lV2yN2dlZXn/9dW7cuEE6nb6rWM7Pz7OwsMDo6Cg//OEPiUajfOYzn6G7u5u2tjbdy1SdXHOp1WpUKhVOnz7Nt7/9bVZWVpifn+fxxx/nYx/7mL54oy24hEIhnn/+eX3NwOv1ks/nWV1dxWKx4Pf7N4mhZt9WsfOeimWlUmFiYoKVlRVu3rxJLBajUqng9XoJBoP09PQQCAR2vOwvpSSTyZBIJEin0+TzeWw2G16vl+7ubnp6egiHwzuK21RsDS3YPJFIEIvFWFpa0m8gzVvQvPtsNqsv9GghRqVSiVQqxfj4uB7+5fV6dS9DsfdothkfH2dlZYXJyUkSiQRms5n+/n6i0SihUAifz7fJRgaDAbvdjhBCn7e02Wy4XC59iK2FBrUie6oW2WyWl156iRs3bnD69GlmZmYIhUL09fVx5MgRTp48id1u3/FNUK/XmZ+fZ3JyksXFRdLpNCMjIxw/fpwPf/jDnDp1St8Prri/5PN5stksY2NjnD59mkKhQLVaxWq10tHRgd1ux+PxYDQauX79OktLS/qxWjzm3Nwcr776KgMDAzz77LN4vV59gUCx92h5FV588UXOnDnDhQsXmJ2d5dFHH+XJJ5/kyJEjPPLII7cdubndblwuly6I2vwltI4HeSf2fBheKpX0Oatyuaz3NjabDavVuu05RK23S6fTZLNZZmZmmJmZ0VfZ29vb/3r10QAAF/lJREFUGRoaoqurS+/VFPcfbc9/oVCgWCzSaDRwOp14vV6Gh4dxu936VtNKpYLVaiWfz5PL5fSV0nq9TjabJZVKMT09jcViYXBwEJ/P19KeyEGj0WhQq9VIJpOkUinm5uaYn5/XtyS63W56e3sJh8N6qrRbbXO76ZP9Yr89Vwzty9L+aRO6mqe3nclbLW5zdXWVl19+WfdE1ut8A3D8+HG++tWv4nQ61TzlHqGFiMViMZaXlykWizidTvr6+hgeHua3fuu36Ozs1Le2nTlzhlgsxrvvvsu5c+fIZrOsrq7qO61isRhf//rX6erq4td+7dd49NFHsdvtyp57gLY9NZ1O88orrzAzM8Pp06e5du0aRqMRt9vNQw89xC//8i/r3uNB68ia6l5p25ZqtRqFQoFkMqnPXWgJPu/2ZWs9nbZlMR6P6x7l4uIi8Xicer2uD/U6OjrUKuoeo807ars1LBYLgUCAUChET08PkUiEavX/b+9cg9s6zwP9vDgASBAAcSNAgOBVIinVJhlFvsi2EkeW5M44ozh2Uo93tpfJ7LbTNt1um047nV6ms5v+yLTb3e1k2t10xq2b/ulm4k437nQSJ05qR7Hd2rrYsilLFiXxDgIEIQC8gQTBrz+A85mUdaFkkSCo88xwBAjnO+c95z3nPd/lvRQpFotMT08DMDIygsvl0vOX8OHDmk6nMQyDQqGwZTHBFh/mmZydnWViYoLR0VFyuRyFQoGmpibtJhQMBnWo8k6j6rHhExMTOrvIyMgIDoeDhoYG/H4/999/Px6P57rtzaQbuVyOy5cvk81mefvtt8nlcrpH0tXVRSwWo6OjYwvPzMLE7DWajufNzc08/vjjdHR06GAA8+V4zz330NXVRTqd5vz58wCkUiltEEVE+9iZ/nbWy29rmJ2d5cyZM4yNjfHd736XkZERlpeXaWpq4plnnuHw4cN0dXXh8Xh2pKGELTaWax1TzZt8fn6e+fl5hoeHEREcDgdut5twOEw0GsXn8113f8lkkqGhITKZDOfOnSOfz3Pp0iU9h2K322lsbKSlpYXGxsYdNSSoFUxfSdPnrr6+nmg0Sjgc1g+VOSVjupOYrmNmsmezrblybhrXnTbM284Ui0XS6TTJZFKP3sLhMF6vl66uLt2xcTgc1RZ109hSY+lyuXjggQeIxWKMj4+TzWYpFAosLS2RzWZ1Ml7TpeDSpUs3nI9aXFwkn8/ruRQz3ZuI0NjYiMvl4qGHHuLTn/40e/bs2cIztYDyyGF0dJS33nqL0dFRoBx+evbsWQqFAvv27btmu2w2y8jICNlsVhtKAIfDQTwep729HZ/PR11d3Y7txWw3CoWCzjW7vLyMYRiEw2HC4TCRSIRAILCjDSVssbE0/bA8Hg+hUAi3202pVNJB9maP0Bx2Xbhw4aY9h7VDNBPTh8/n87Fr1y7279+P3++3eiFVIJPJaD+81dVVXWTK7XZfM1+hGbgwMzNDoVBYNydps9nw+/2EQiFcLpfl0bCFFItFZmZmSKfTFItFnZw7EAjg9XrvijywW3q32e12mpqacLlcPP3009x3332k02my2SzZbJZ0Ok06nWZoaIhSqURdXR11dXV0dnZec+4yGAwSj8dJJBK8/PLLOjuyzWYjEonov0AgcEfizC1uDREhFovR399PoVDQPcpEIoHH4yGVSgHoiJ5EIkE2m+Xdd99lcXERpRT19fV6cWdpaYn333+fbDbLQw89RCgUwuPxWLrdAhYWFjh//jzj4+N6cS2RSLCwsMDJkyfxer0EAgEikYhOhrHTOidbaizNGNFQKMRTTz3F8vIyk5OTTE9PMzw8zPnz5/nggw+4fPkygF7FHhgYuGZ+ut27d3PgwAFOnjzJa6+9ts5YhkIhWltbtbG0qA6xWIy+vj7GxsYQEZ3l2uPxaLeusbExcrkcb7/9NqOjowwODlIoFPR0TKlU0lmLzp07x9TUFGNjY7S2tmIYhmUst4CFhQUuXrzI6OiozmSeSCRIp9OcPHkSEWHXrl3ce++9BAIBHWiwk6jKOMZcyBERAoEAdrsdh8OB1+ultbWVcDjM6uqqdlbv7e295kKPGSdsKsUwDDweD16vl76+Pvbs2UNzc/NWn55FBRHRuQdNXZkZp0ZGRvje976Hx+PRJU9HRka4cuUKxWKRWCxGNBplz549zMzMcPLkSV1HulAoMDw8rBeCrJfh5mNOoSwvL+upEdPtb3x8HLvdrqPmIpEIfX19urNjs9nWjRQcDgf19fXU19fr3mexWGRhYUFnIHI4HOue7e1A1SZ9zCF2Q0MDSil6enp08TLTzcS8kNdb9ZyfnyeTyegLarfbaW5uJhKJcPjwYR588MEbrqZbbD5mHed4PI7L5dI9y6mpKc6ePbuuRIC5mNPR0UFPTw8HDhzg2Wef5f3332dmZoZUKqXDV0+fPk0ulyMcDrNr165qnuJdQalU0p4rUH42zTnn9957j7Nnz+rntLOzk0OHDhEKhdi9ezd2u51MJqMzSHm9Xr04ZDI/P8/ExAQOh4Pu7m48Hg/BYNAylmu5VvjTRlfVTN/MZDJJqVTC6XTS3t5OPB7XxcesRYDqYvpFNjc309fXp+ekV1ZW1jmdm6MMj8dDT08P9957L729vQQCAe3wvLKyohcYstksqVSKXC7H4uKidimy2Bzq6upoaWmhVCqRyWRYXl7Wv5mp2qCsx3w+z+joqPZ2sdvt5HI5VlZWSCaT2o96bUemUCiQTqex2+1ks1ncbjfRaBSPx0M0GiUYDG75OV9NTd9dIyMjfOc732F4eJilpSUCgQBPPPEEPT09dHd3W76V2wAzUcKBAwdoaGjgxIkTPP/888zOzlIqlXSv0m6309/fT09PD4cPH+bQoUO4XC48Hg+Li4vs37+fpqYmxsbGWFhY0CvsBw8eJJVK0djYaA3HN5FgMMiRI0cYHR3lBz/4gY62uhqlFKlUildeeWVdZihz1Gj2Pq/OS2kO6e12O4FAAJfLRW9vL01NTTz77LN85jOf2ZLzvBE1aSyXl5dZWloik8kwOTmpFwrsdjterxefz2dl1t5GiAher5eWlhba2tpobW0lm82SyWQolUqIiO65dHR06J6EGcDQ0NBALBajWCzi8/lYWlpaFyKbTCax2WyWsdxE6urqaGtrw2az0dnZicvl0j1H+LAGvBmxdb0ytjfDMAydlaqhoYFCoUAymWRmZgaXy1VVF6WaNJZjY2NcunSJn/zkJ7z66qssLS3pHHkejwePx2MNybYZppuPOdSenp7mxIkTLC4u6oW8Y8eOMTAwQDAYXPeyC4fDHDt2jPHxcSYmJrh8+TIXLlwgmUzy+uuvMzc3x6OPPkpra6v1gtwkIpEITz/9NHNzcxw4cIDp6WleeOEFBgcHdYz+2mxRtxuzb86NLiwsMDg4SF1dHU1NTeRyOQYGBnjwwQerpuOatCgLCwtMT0+TyWS4cuUKSincbve6NG9WzPD2wuFw4HA4aGpq0rVW0uk08/PzNDQ0UF9fTzweJxaLfWRU4HQ6CYfDFItFIpEI+XyeoaEhlpeXdX4As57PdloQ2Ek4nU4ikQg+n49SqaRd88xkNaurqzrxjTnkXl1dZXl5WSe82SjmtmZtn1QqxcTEBJ2dnZt0dhujJo3l1NQUZ86cYXR0lNXVVe0q1NXVxe7du3WdFovth9frZe/evXR1ddHT00OpVNLzV2YexKtfdGZMuOlz6/P5dKLgqakpisUi+/btY25ujrq6OsvvcpOw2WzU1dURj8cJh8N8+ctfJpfLaeM4NTXFyMiIHo5nMhlee+01ZmZmSCaTOkLvVlBK6Sz7m1Gx8VaoSWM5NzdHMplkdnYWQK+2RqNRHX5lsT1xOp16ZTMWi224nc1mw+l00tzcrEun2mw2XVvJzBFgPtDWcHxzsNlsOpru6hXq8fFxnZ/UrFZw4cIFSqUSMzMz67YVEb3Yc3VvdO0Q3kwNZ6aDU0pZw/CNopRienpa+97Z7XbC4TAHDx6kvb39hindLGqb+vp69u7dS1NTE5/85CeBcuapbDbLhQsXOH78OG1tbezfv9+as64Cfr9/nb+03++nv78fv99PIpEgn89rI2nWwgqHw3R3d5PL5RgcHCSfz3Px4sXb6oVuNjV5R+VyOZ1PzzAM/H4/fX19tLS0WEOwHYzD4aCtrQ2/3093dzf5fJ58Ps/4+DiTk5O8++67KKX4xCc+YRnLKmAurpo4nU66urp0RQQo90zNKZd77rmH3t5ePvWpTzE5OYlSikQiod3Dths1c0cppRgbGyOTyay7mGay4HA4TCgU2vFpou52zFDZPXv2YBgGiUSCy5cvMzk5yRtvvIFSikceeUTX9rEW+qqH0+nUjuymsTSjtBKJhHZgX11d1XXl17ojmZheLtWeXqkZY1kqlbh48aJOtJHP56mvr6exsRG3200sFiMSiVRbTIstwOl06pHE6dOnefPNN5mYmGB4eBjDMHjqqaf0A2oZy+phZgyz2+3aWJrhzGNjY4yPjzM+Ps7U1BSzs7MMDg5+pJ68zWbTZayrPWqsmTtJKaUzo5uTxQ0NDUSjUV2cHT4sY1AoFFhcXLxt51iL7Yvp5B4MBuns7KS/v59IJIJSimw2y+DgIENDQ7qapEV1MAuZ+f1+YrEYra2tupQIoH0zp6enmZmZWRfRZRgGoVCIlpYWent7GRgYuKUFwc2gpnqW586d45VXXtFd+GAwSH9/vw7WV0rpzDQLCwusrKzg9/ut+asdhs1mIxwOEwgEOHjwIA0NDRw/flw7rb/44ovs2bOHnp4enE6nFc1VJRwOB5FIBMMwGBgYwDAMTp8+rZNxQHn9wfRoMOPLDcPA4XCwe/duotEojz/+OEeOHFmXpaga1JQVWVlZYXl5WV9Um82G3W5ndXWVK1eusLCwQDKZXLeNYRjr3mYWOwMzvthMAB0KhXTuy2QySTAYZH5+Hq/Xq2v2WGwtpo7q6+tpbW1lcXGRRCKhk2qsdVg3Q17tdjs+nw+3201vby+tra00Nzdvi6Q4NWUsr8Z0Q8jn87z55pssLi7yox/9iGw2S3NzMx6Phy984QvXTBxsUfvYbDa6u7uJRCIkEglef/11FhcXOXXqFEtLS4yPj2MYBk6n0wpSqAIigtPpxOfz8cQTT/Dwww/jdDppaGgglUqRSqUoFArMzc3hdDoJhUI0NjZy4MABotEon/vc53S6tmov7kCNG8tiscj8/DxXrlxhdHSUQqHA9PQ0c3Nz+P1+y+fyLsDlciEihMNh2traSKVSpNNp5ubmyGQyNDY2Wgmgq4zp3ud0OonH43R0dGAYhp4us9lsuFwu4vE4gUBAJ1NpaWnZVou2NW0sE4kEr776qs5M43K52LVrF9FolKNHj9LV1UVXV1e1xbTYROrr63E6nRw6dIh4PM6Pf/xjvv71r5PJZHj55Zfp7OzUCYgtqoMZ9WPW3jpy5Ahnz57VgSXj4+O0tLRw9OhRvRjkcrkIhULVFn0dNWUszXT0ZqLXUqlENpvV/lo+n4/+/n6amppobW2lvb3dekh2OKaTcyQSoa6ujtHRURwOB6VSiYmJCQzD0KvilhtR9TAMA8MwiMfjxONxXU/e7/djs9no6Oigv78fn8+3bev31IyxtNvtPPzww/h8Pk6dOsWpU6dobGwkGo0SDofp7+/H6/XqkMf29nZdy8Ni5+NyuTAMg5aWFvbu3cvc3BxDQ0M6a7eZHNiau9wetLa24vV6KRQKzM/P43a7CYfD2zpjWM0Yy7VJR2dnZ5mYmKC5uZmenh7a2to4evQoHo8Ht9uN3W7HbrdXfULYYuswde7z+YjFYiQSCc6fP0+xWCSTyZDP5/VQ0KL6+P1+/H5/tcW4JWrGWIqIrjn+2GOP0d3djdvt1mU3zeqB5pvJMpR3J+3t7TzzzDMMDQ2RzWYxDEMv9my3OTCL2qKmjGUwGCQYDNLW1lZtcSy2KfF4nGPHjvHOO+/w0ksv6YqE2WyWYrFYbfEsapiaMZYWFhvBzHsZi8X4/Oc/z9LSkg6HrGb9FovaxzKWFjsKM/nvrl27+MpXvoJSCsMwrlly2cLiVrCMpcWORESqHh5nsbPYnmv0FhYWFtsMy1haWFhYbAC53fq+ACIyDYzcOXG2PR1KqXC1hdgq7kL9gqXju4Hb0vHHMpYWFhYWdwvWMNzCwsJiA1jG0sLCwmIDWMbSwsLCYgPc0FiKSEhE3q78TYnIxJrvzs0SSkS+KCJKRO7fwLalijzvici3ReS2wzRE5G9F5Gc2uO0DIrKy0e23K1utYxHpEJEfisgZEXlFRFo30GZYRN6ttPm+iNx26nsR+W8i8ts32cYpIs9XjvmOiBy63eNtB6qg4y+JyPSaY/ziBtpstY4dIvLNyjHfF5Hfu9l+b2gslVIzSql9Sql9wDeA/21+V0oti8gd9/oVES/wG8C/bbDJYkWePmAZ+JWr9rcZMhrAnwDfv9P73mqqoOM/A/5OKTUAfBX42gbbPVZpcwL4/bU/SJk7OUr6JQClVD/wOPA/7/D+t5RqPMfAt9Yc47kNttlKHT8D1FV0fB/wyyLSeaMGt3zwSu/rGyLyb8CfXm3FKz28zsrnnxORNytvl7+qGJmb8ceUDVHhZhteg+NAt4gcEpHjIvIicFZEDBH5HyLyVuXN9csV+URE/kJEzovIy8BGc9j/OvAPQOo2ZNz2bLKO7wF+VPn8L8Dnb1G8H1PWcWdFb38HvAe0icjvrNHxf18j7x+IyAci8hNgzwaOoWVUSqWALHDTUU4tsQXP8cdhK3SsAHflReGi3NHK36jB7VrqVuARpdRvXW8DEfkp4FngYOWNVgJ+tvLbc3KNIbaI7AfalFL/fKsCVU76CeDdyn/tB35DKdUL/Gcgp5R6AHgA+CUR6QKepnxh7wF+AXhkzf6+KiJPXuM48Uq7/3urMtYYm6Jj4B3gC5XPTwNeEbmV3GnH+FDHPcD/UUrdS1mPPcCDwD7gPhF5VETuA/5D5f8+S1n/pvy/IiLrRiJrZHxSROyV++Q+YCemutosHQN8sWLQXhCRW712W6HjF4B5IAGMAn+mlMrcSKjb7X5/WylVusk2RyjfZG9JOYGBi0pPTCn1kTmMShf7fwFfukVZXCLyduXzceCvKRu9N5VSlyv//9PAgHw4v+ijfNEfBf6+ci6TImL2eFBK/dF1jvfnwO8qpVZlZydmuOM6rvDbwF+IyJco9yAmKD+AN+NfRKQEnAH+EPADI0qpf638/tOVv9OV7x7KOvYC/6iUWgCojDaoyPiN6xzrb4CfojwcHAFe36CMtcZm6fifKD9XS5VR3DeBwxuQZyt1/CBlnbYAAeC4iLyslLp0PeFu11jOr/m8wvoean3lXwG+qZS66cRpBS/QB7xSUUoUeFFEnlRKnbhBu8XKG09Tab9WRgF+XSn10lXbfXaDsq3lfuD/VY7RBHxWRFaUUv//Nva1ndkMHaOUmqTSsxQRD/BFpVR2A00fU0qlzS8i4uejOv6aUuqv1jYSkd/cqGxrZFwBvrJmH68DH9zqfmqAzdLxzJqvzwF/usGmW6Zj4D8C31NKFYGUiLxG+dm+rrG8ExOmw5SHvOYw2iyn+EPgZ0QkUvktKCId19uJUiqnlGpSSnUqpTqBfwWeVEqdEJG4iPzwY8j4EvCrIuKoyNIrIm7KPZtnK3OaMeCxm+1IKdW1RsYXgC/vQEN5NcPcAR1XtmmSDyfqf49yL8787dzHkPEl4D9VDDCVeyZCWcdPiYhLyouHn7vZjkSkoXJ/ICKPAytKqbMfQ7ZaYJg7p+PYmq9PAu+v+W1b6Jjy0PtwZT9u4CHghrLdiVWwfwB+QUQGKa9gfwCglDorIn8IfL/ycBSBXwNGROQ54Bs36TGuJUb5zXe7PAd0Aqek3CWcBp4C/pHyBTtL+eK9YTYQka8CJ5RSL35kb3cfd1LHh4CviYiifJP/GpSNKOWew22hlPp+ZX7tjUqvfw74OaXUKRH5FuV5yBTwltnGnMu6xlAtArwkIquUpwl+/nblqiHupI7/a2W+fwXIUJla22Y6/kvg+cr5CvC8UurMjY5fE7HhIvJfgFHLcO1cROQYsEsp9fVqy2KxOdS6jmvCWFpYWFhUm5p1tLWwsLDYSixjaWFhYbEBLGNpYWFhsQEsY2lhYWGxASxjaWFhYbEBLGNpYWFhsQH+HYwYyV7nD5tnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=images,\n",
    "            cls_true=cls_true,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples of Mis-Classified Images\n",
    "\n",
    "We can plot some examples of mis-classified images from the test-set.\n",
    "\n",
    "First we get the predicted classes for all the images in the test-set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = model.predict(x=data.x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we convert the predicted class-numbers from One-Hot encoded arrays to integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot some of the mis-classified images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAD1CAYAAADZANcvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9aXBc153Y+zu97zsaWwNoLCRIQqRIkbJEW7JWS7ZjaTxJ/MY10aRck8V+mVepvKmkXiYzHzJfkionb6nUy5SnyvUcz5QdO/ZUJvZkZMuWRIuWLVKkuO8AsQPdDfSK3rfzPgD3GuAigguABnl+VSwC6L63z+1zz//+z38VUkoUCoVC8fEYtnoACoVCsR1QwlKhUCjWgRKWCoVCsQ6UsFQoFIp1oISlQqFQrAPTvR4YCoVkNBp9gENpfU6ePLkopWzb6nFsFmqOH37UHK+fexaW0WiUEydO3Ovh2xIhxORWj2EzUXP88KPmeP2obbhCoVCsAyUsFQqFYh0oYalQKBTrQAlLhUKhWAf37OBRKO6Her1OLpdjaWmJ999/n1gsRq1Wo9FocODAAfbv34/D4cDr9W71UBUKQAlLxRZRr9dZXFxkbm6Ob33rW5w5c4alpSUqlQpf/epXaWtrIxQK4fF4EEJs9XAViodLWNbrdZrNJpVKhUajcdPrRqMRq9VKuVxmbm6Oer2O0+nEZDJhs9kwm83Y7XasVusWjP7RIJ1Oc/36dTKZDBcvXiSRSBCLxSiXy5jNZiwWC3a7XZ8PhaJVeGiEZbPZpFQqUavVSKVSlEqlm97jcDgIBAIsLCzws5/9jFKpRFdXF263m3A4jNvtpr29nba2RyYmedOZnp7me9/7HjMzMxw9epRcLke5XEZKSSgUwu12EwgE8Hq92O12pVUqWoZtKyyr1SqFQoFarUY+n6darbK4uEilUmFxcfGWwtLpdBIKhUilUoyOjlIul8nlcthsNgKBAE6nkyeeeEIJyw2kUqmQSCSIx+Pk83lKpRImkwmz2Uw0GqWrq4tIJKI0S0XLsW2FZTKZ5MqVKywuLnLu3DkymQxXrlwhl8sRj8cpFos3HaNpjtVqldnZWWq1GlrxY4PBgMlk4l//63/N3r17N/tyHhmy2Sznzp1jYWGBcrmMEAK/34/X6+VLX/oSn/70p+ns7CQQCGAwqGANRevQ8sKy0Wis2WJXKhUqlQrz8/OMj4+zuLjIxMQE2WyWmZkZlpaWSCaTtxSW+Xxe97hmMhnq9foagWk0GikUCpt9iY8E5XKZfD5POp2mUCjogtJms9HR0UE4HKa7u5vOzk48Hg9Go3Grh6y4D5rNJlJKff2Wy+U1foRyuUy5XL7t8UIIDAYDQgh8Ph9OpxODwbClD9CWFpbNZlNfXB999BETExNcunSJK1euUCqVWFpaolqtUiwWqdfrlEolGo0G9Xr9lucrl8vEYjHgN84g1VZjc7hw4QLvvPMOly5dIpFI0Gg0CIVChEIhfv/3f5/du3ezY8cO2tralKDc5jQaDcrlMpVKhWQyST6f58yZM6TTaQCklJw/f54LFy7c9hwGgwGXy4XdbufLX/4yL774Ik6nE7fbvVmXcRMtLSyllFQqFYrFIrOzs1y7do1z585x5swZms3mGq3w41j9lFqtRd64KIUQaqFuEJlMhtHRUWZnZ6lUKphMJrxeL21tbQwODjI8PIzf78disWz1UBX3SLPZpNFoUKlUyOVylEol4vE4uVyO8fFxFhYW9PeeO3eOEydO3HLNNRoNhBB4vV6cTifPP/885XJ5y6NUWlpYNptNfVv90Ucf8ctf/pJMJkOtVrsrrdDlcuHxeHA6nfj9/tt6WI1GIx0dHQ/yEhQrxONxTpw4QSqVol6v097ezu/+7u8SjUbZvXs3wWBQCcptTiwW4/r160xPT3Ps2DFyuRzz8/OUSiVSqRSVSgVYVoKSySSwvDaDwSCwvN6r1SrpdJparabvFLPZLNlsVglL4Cahpwmz1Zrl/Pw8o6Ojtzx+tfBbbdPQ/m632/H5fHi9Xrq6um5r9zAYDHg8nvu6FsXNSClZWlpibm6OYrGIlBKXy8W+ffv0rbfdbr/tsatRoUStiZSSbDbLxMQEV69e5Re/+AXZbJb5+Xmq1eot16UW1+z3+wF0rTSbzQLLES9SSsrlsu6z2Eq2VFhWq1VqtRqzs7Ncv34du91OW1sbDoeD7u5uYFmAGY1GLBYLNpsNk8mE0WjUt+FOp5POzk5sNhs+nw+r1UowGNRT5ZxOJ06nU7d/eL3ej11wu3fv3qzLfySYmpoiFosxPj5OsVjEbrfT39/P8PAw3d3dhEKhmzRKzYmXy+VIJBK6wLRYLHR0dGCz2bDb7ZhMLfGsf+SZmZkhkUhw/Phx3nrrLRYXF4nFYlQqFcxmMzabjd7eXtxu95odnt/vx+1267u5RqNBPB7nW9/6FvPz8zQaDaSUTE9Pc+rUKaSU9PT0bNkDc0vvNk3Vnpqa4le/+hV+v58dO3YQCAQIh8OYTCbdpmEymbBarVgsFiwWC7VaTdcEBwcH8Xg89PT04HK5GBwcxO/3E4lECIVCmM1mTCaTfg7F5iClZH5+ngsXLjAzM0O5XMbn8zEwMMDAwADhcBifz3eTzaparZLP54nH41y5coVmswksJxVYrVY8Hg8Wi0UJyxZASkkikeDy5ct8+OGHvP3221SrVarVKiaTCYfDoa/J9vZ2uru7CQQCRKNRent7cblcBAIBYHkbPjo6yo9//GPi8bhuaovFYly+fJmurq4tvdYtudsajQaNRoMzZ85w6dIlrl27xtmzZ4lGo/T19emLw2g0EgqFsFqtvPDCC7S3t2O323E6nVQqFZaWlvB4PAwMDOjqvNVq1bd1Pp8Pu92ua6cqbm/zaTQaVKtV6vU6UkosFgvt7e0Eg0FMJhPNZpOFhQX9oZlMJslms+RyOZLJJNPT0/r9YLPZuHr1Kg6HA7fbjdls1h+E4XCYzs5OXZhqC1Vt2zeOZrNJvV5nbm6Oixcvksvl9Hz+3t5eHA4H7e3tOBwOBgYG8Hg8+Hw+PZNOW68WiwUpJfV6HaPRuGbOhBCEw2F27Nih2za3ii0RlvV6nWq1yi9/+Ut+8IMfsLCwwNzcHAcPHuSZZ57RnyiawyUcDvOlL32JfD6v2x4LhQLpdBqLxUIwGFwjDLUvWy2Urader1Mul3X7k91up6enh46ODqxWK81mk6mpKRKJBG+99Rbnz58nnU6TyWQoFouk02l9G24ymQgGg/rWzmQy4XQ6sdlsPPHEExw+fJhgMEhfX5+uhSrtc+PQ1vH4+DjHjh2j0WgQiUQYGhri9ddfJxgMMjw8rNdb0HYQt1qfzWYTIYS+m1y9lru7u9m/fz/d3d1buqY39U7Snh5jY2MsLCwwPj5OMpnEbDbT399PNBq95dbMYDDoecLaIrHZbLhcLl2z0EKDFK2D5vWcmJgglUrRbDax2Wx0dnYSDAbJZDKkUinOnj3L9PQ0169fZ2FhgXw+Tz6fp1Kp3BQzWygUMJvNVCoVjEajXoBjdHQUs9mM3+8nFosRCoU4ePAgTqcTi8WidhUbgJb1Fo1Geeqpp/QdQFdXF11dXXg8Hl1QakLwdpRKJWZmZpicnNQD2K1WK2azGY/Hg9vtxmazbdal3ZJNFZaNRoNiscibb77JiRMnOHPmDDMzMzz++OM8++yz7N69m7179+JyuW4y+rvdblwul/6Fa+o7KA2yVZFScvXqVX7+85+TTCaRUuL3+zl48CA2m43x8XHi8Tjf/va3uXjxIpVKRQ8L0xbeahqNBrlcDuCmXcT4+DhHjhzB4/HQ1dXF8PAwgUCArq6uWzqRFPeP5gd45ZVXeOGFFwD0HaHZbNYzbtazPpPJJO+8886aB6vX69UdQF1dXTgcjo2+pI9lU4SlZttIpVJkMhlmZ2f1MBJYFoS9vb20t7frBRRu/IKFELf8m6L1kFJSKBQolUq6/VFLbavVamSzWfL5vO4pTyaTFAoF3QHndDpxOBy3TBy4kVKpRKVS0cNLCoUCCwsLBAIBEokEFotFdwgpNgar1XrPjlPNpp3NZpmenmZubo5qtYoQAo/Ho1cDs1gsW54wsuHCUqsvmc1m+dnPfsb09DS//OUvuXz5MkajEbfbzZ49e3jttdd07VFtqbc39Xpd94BfvnyZhYUFPS84Fovx05/+lGq1yokTJ0in08zNzdFoNHQt8MCBAxw4cAC73f6xxX+bzSYXL15kcnKSsbExLl26RKVSIRaLIYTg7bffJhqNEggEcLlcm/kVKNZJLpdjdnaWM2fO8D//5/8kkUiQzWYxm80cPHiQ4eFhhoaGcLvdWy4TNkVYanncs7OzTE1Nkc1mKZfLhEIhPUwoEAhgs9m2/OmhuH+klHrRjGKxSLVa1V8rlUp6yuP8/LxeXs9oNOLz+ejs7KSnp0ePcPi4uNhms6nvTkqlEouLi2SzWZaWlvRUO6vVytLSEuVyWdkuW5BarUYul9Pt19lslmazqd8P4XAYh8PREvO24cJyaWlJN+C/+eabTE5OUq1WCYVCfOlLX+LFF1+kv78fl8ulBOVDRLFYJJfLUalU1mThxONxfv7zn9NoNFhaWtIN+Tabjc9+9rM8++yzdHd3093drcfX3g4pJeFwmFKpxMTEBNeuXeOjjz7iL//yL8lkMrz33nu0t7czMDBAMplk586dhMPhzbh8xTrQHIDnz59nbGyMQqGg3w8ul4v+/n49FbYV2HBhWavVWFxcJB6PMz09zfT0NG1tbbjdbvr7+zl06JDu1VY8HGhRD9Vq9ab2HuVymdnZWf13k8mEx+PB4/HQ19fH7t278fl8+Hy+dX2WVoXGYrFgNptJJBKYTCYKhYLeBC0Wi+H3++nt7X1wF6m4L7SiG4VCgUQiQTqd1u8Vi8WiZ+BpsZitwIYLy3K5zOTkJFNTU/p2q62tjba2NsLhMH6/XwnKRxCj0ajbJP/O3/k79Pf388QTT9yz59rhcBAOhwkGg3p+vxbfOTc3h8PhYM+ePQ/6MhT3yNTUFBMTExw/fpy/+Zu/IZ1OU6lUcDqdvPTSS0QiEUZGRohEIretG7DZbIpmmUwmWVxcpFarIYTA5XLpeaFbHQ6g2BoMBgNWqxW3283+/fvZu3evnv52L2iZO263G7vdTqlU0qvWZDIZveWIojVIJpNcu3aNy5cvc+bMGd2ubbPZ2L17N4ODg3R3d697h7EZbLiwLBaLXLlyRc8NbjabzM/PUywWOXnyJG63G7/fTzgc1othbLXXS7FxmEwmLBYLbreb3bt309XVxY4dO+jr67svj7V2zqGhIb74xS8yNTXFT3/6UyqVCpcvXyaTyfCZz3zmAV6J4n5IJpNcvXqVWCxGs9nU01Pb2trYs2cPO3bsaLkKYJsiLMfGxpiamtKf9PPz8ywuLnLy5EmEEAwMDDAyMoLf71ctBR5ytO233+9n37599PT0MDQ0RE9Pz32dV8sTHxoa4rd+67c4d+4cR48eZWlpiWvXrul2MUVroGmW8/PzSCl123UwGGT37t3s2rWr5VJVNyV0aHVuMPzGATAzM4PJZGJubo7x8XHC4TCPPfaYbssyGAyUSiWklHqwus1mw2az6dpnrVajWCzqFYi03tNK4LYmwWCQJ598kvb2dg4ePKgHHT8otPQ4FV3Rmmj9l2KxGJOTk3ruv9vtZteuXQwMDOihQq22w9xwYal5vLRGYEIIPd/3/PnzXLx4Uf9iotEozz//PMFgkMHBQUwmE6lUilqtpveU1pxDGoVCgdnZWV2r0Eo+qYXSmvT29vLGG2/Q1dXFyMiI3ojqQWG1WgmFQrcs/abYWrRQoVQqxbVr17hw4YLuAff7/Tz77LP09fXh8/laTquETRCWVquVrq4uGo0GqVRqTYCyVqoNloVoLpdjamqKTCZDuVzGZDKRzWap1+vE43EcDodedUijXC6zuLiIyWQik8ngdDrp6OjA5XLR0dGh18pTtAYWi0XP+d2IHYC2a9EKxypaAy3lOR6P66X46vU6VqtVt1X29fXR3d3dsqmpGy4sA4EAL730ElNTU/zsZz9b07RoNVoR0SNHjuj1J7W/Syl17fPGupTa4jCZTPj9fux2Ozt37iQUCvE7v/M7PPfccxt9iYq7QGt929bWtiHaQ61WI5/P6+0rFFuPlFIv9H3s2DF+9atfcfHiRaSUeL1eduzYwYEDB3jxxRcJBoMtE1d5I5uiWfb09GAwGIhGo9jtdl1zhOUvUgtQrdfrt21jeyeMRiO1Wk1/UpXLZeLxOMlkErvdrkKUWgStZuGNRV4fFJVKhcXFRb0vvNFo1Kt1t+LW7lFAS39dWlpiYWGBWCxGPp8HlvtjhUIhgsGgXpu0VdnwuyccDvPbv/3b5PN5nnrqKRYWFvjhD3/IhQsX9CK/hUKBfD5/X328NdtosVjkwoULuu0qm82yb98+PvGJT7ScwVjx4JBS0mg0mJ6e5sc//jGTk5MUi0UsFgsjIyN6OwPF5lOpVDh9+jQzMzMcO3aMU6dOUa/Xsdls9PT08NxzzxGNRlt2+62x4cLSYrEQDofxer00Gg2CwSCRSITFxUUajQbNZhOr1ar39NY0zWq1qts51ov2Xk3lTyQSzM7OEo1GN+jqFLfDZDJhNptvsklqc6vVrXxQXk/tXsnn88zOzpJIJGg0Gnp1da0gg2Jz0RoLLiwsMDs7SzKZJJfLYbVa9UIpHR0dereDVmZT9iVatkZ3dzdtbW38s3/2z8hms7pw1MIItO14KpXi/fffJ5lMEo/H9coyd8PqKt2qY+PmYjQaiUaj2Gw2PvzwQ8xm85q5PX78OF1dXRw8eFCvgH2/HvF8Pk8ymWR0dJQTJ07oRYJ9Ph+HDx9mZGSEzs7OB3F5inVSq9VIp9N6y5AzZ84wPT0NwPDwMI899hiHDh3ik5/8JE6ns+XTnjfNiGMwGPQMjRu3QzMzM3R2dure8bm5Oa5du0aj0dCbsWto/Tm0YsCrtdHVW3itNJxWDk5Kqbbhm4TBYMDr9dJsNvUivrBsKtHaB8DygtHsl1pl7XulXC6TyWRIJpPEYjHK5TJ2ux273U4kEiEajeJ0Oh/I9SnWR7PZ1HtljY+Pc+3aNT3l1O/3Mzg4SDQapaura1vYk1tihD6fjx07duiCz+fzsXfvXnw+H/Pz8+RyOV1IRiIRenp6aGtrY2hoiGw2y4ULF8jlcoyNjd2TFqp4sAghCAQC2O12hoeHefzxx4nFYkxNTTE/P89PfvITfD4fFy5c0DN5QqEQQ0NDhMPhW1bFvxHtXsnn85RKJT744APeeecdrl27Rq1Ww+FwsGvXLnp7e+np6dE7gyo2j1wux/vvv8/ExATxeFw3rQG0t7frtuRWqFW5HlpCWLpcrjV5wRaLhf7+foQQ+g2u9fNob29nz5497Ny5k2eeeYa5uTm9P/X09LQSli2AVizF4XDQ3d3N4OAg9Xqdqakp0uk0x48fx2q1cuXKFfx+P+Vymb6+PoLBIKFQaF12TG03USgUyGazXLp0ibfeekuPy/V6vfT19a1pgqfYXAqFAhcuXGB8fJxMJkOtVgOW17LP5yMajRIMBrfNjq8lhOWNWCwWPZBdE5baE2l+fl4PYNd6Tl+4cGFNOJKG1gFScyApNhchBL29vTz55JPY7XYqlQq5XI54PA4sax71ep0PPviA0dFRSqUSY2NjesrqjZjNZl2Yzs3NkcvluH79OvPz81y+fJlSqYTP52PXrl2EQiGefPJJOjo61PZ7k8nn88RiMcbGxrh48SJTU1N6SnJXVxeBQICBgQE9eWS7rM2WFJZWq5VoNIrJZNKF5erQkJmZGWZmZojFYiwtLXHhwoWbKnIbDAZsNhter7elY7ceZrQiKdrOoVKpMDMzQyaToVKpkMlk9B48ZrOZWCxGf38/Dofjlltmp9PJyMgIFouF48ePMzs7y4kTJxgdHdW9q729vbz66quEw2H27NmD1+ttueo1DztLS0tcunSJK1eu8NFHH62pLNTf38/g4CDDw8NEIpFtIyihRYWl1shM68kSiURIp9N6frkWm7mwsKCXotcEpda7w+FwsHPnTvbt26e8oFuIw+HQNYlSqUQ0GsXn85HNZpmZmaFYLOqpbwsLCwgh1rQ5Xo3FYiGfz2MwGBgbGyOVSlEqlTCbzYTDYXp7e9m1axfDw8P4fD5CoRB2u31bOA8eJur1OktLSxQKBer1ur4r1DTLPXv20NbWdktBWa/XSSaTNBoNvS9Xq9CSd5F28xuNRvbt24fRaOTUqVO6sAT0dqqaxgnoXtXBwUE6Ojr4zGc+w0svvbSmSpFic/H7/fh8Prq7u3n66af1/P/5+Xn+9m//lvn5eT788EMWFxe5fv064+PjwO3bHK/2rEsp9Vi9xx9/nOeff57BwUGef/55rFarbvvcLg6Eh4VyuUwikSCZTK7p/240Gtm/fz9f+MIXCIVCtz32/PnzlEolDh06REdHx2YN+460pLDUcsBtNhuRSIRSqcT8/LxuvF8dsK5pIiaTCa/Xi9PpZOfOnUQiEdrb23E4HEqz2EI0z7bBYMBsNiOl1KtG7dy5E5/Ppy+uTCajZ2HdzlGnhRhp5pVgMIjP52NoaGhN73k151uHlhygZeUZjUYCgQBer5dgMKgXUZFSUqlUKBaLlMtlUqmU3v+9FWnJO0oIoVen+dznPsfhw4f1JkaJRIJEIkG5XCafz2OxWPS+K0899RQdHR289tprerk25dxpLex2O93d3XR0dDA4OKj3aMpkMvziF7/gypUrXL58mStXrtx07OoH4uHDh4lEIuzZs4e+vj46OjqIRCKYzWYlKLeYYrHI+Pg4c3Nz1Go17HY7zz77LNFolD179uhaZa1WY35+ngsXLjA9Pc1bb72F1WrlC1/4ApFIpOXmsbVGcwOa/dFisdDd3U1fXx9Go5FGo6F717TF5/f79UXT1dWlWp62KEIIvaq5zWbTt9M+n4/JyUnK5bL+70ZMJhM+nw+n00lfXx89PT16HKXW00mx9WiVnwqFgp7S6nA48Hg8mEwmms2m/vrs7CzT09PMzs6SSqVwu926s67V0h9bWlhqWT92u53f/u3f5qWXXuLixYtcunSJZDLJzMwMXV1dvPzyy7ozyG63t0yfYcWdMRgMhEIh/H4/r732Gi+99NJtt+GrKxZ5vV69wpTadrcWhUKB0dFRvUmhxWKhXC7rVYdmZmY4cuQIR44cIZPJMDc3h8/n4/HHH6ezs5O9e/fS3t5+Xz2ZNoKWv8OMRiNGo5Hu7m66u7tpNBrUajV8Ph8Gg4G+vj727t2rh4i02tNI8fFoNmdAFbp4SGg0GlQqFSqVip6GXC6XKRaLeqfNsbExTp8+TaFQ0PP4w+EwXV1dei+uVnPMtbywvJFIJILb7aZcLlMoFHA6nbS1td13brFCoXgw+Hw+9u3bx9zcHKdOnaJcLnP69GmuXr3K2bNncblczMzMMDs7S1tbG4cOHWJ4eJhXXnmFUCiE1+t9NHvwPGh8Pp9KXVMoWhiteInmCdeaEwJcvXp1zXs7Ojro6+tjYGCAXbt2tXQCwbYTlgqForUJhUJ8+tOf5vr161y+fJlYLKaH/Xk8HhwOB/39/USjUaLRqN7ls1XbSWgoYalQKB4owWCQw4cPEwgE+MlPfqLHUmrCMhAIcOjQIZ599lm6u7vZs2cPZrNZCUuFQvFooSWUdHR08PnPf17viVStVgkEAjidToaGhujv78fn892yon4rooSlQqF4oBiNRpxOJ/39/Xzta1/Ta4/Cb9JYVxfwbjVHzu1QwlKhUGwIWlzsw4KKtVEoFIp1oISlQqFQrANxr326hRALwOSDHU7L0yelbNvqQWwWao4fftQcr597FpYKhULxKKG24QqFQrEOlLBUKBSKdaCEpUKhUKyDjxWWQoigEOL0yr+YEGJ21e83d5S6T4QQXxFCLKz6jH+8jmMmhBDnhBBnhRBvCSHuuWmHEOLfCiH+5R3eYxZCfHvlMy8JIf7oXj+vFdjsOV75zP9FCHFRCHFBCPHddbx/U+d45X1/JIQYFUJcEUK8eq+f1wpswTr+w5X5PSuEeFsI0beOY1p+jj82YlRKmQT2awMA8lLK/7jqw0xSyvodR393fF9K+b/d5TEvSCkXhRD/Dvg3wD/XXhDL6QFCStm87dF3x5cAq5RyrxDCAVwUQvxXKeXEAzr/prLZcyyE2AH8EfApKWVaCLHekvabNsdCiD3Al4ERoAv4uRBip5Sy8SDOv9lswTo+BRySUhaFEP8r8HXgd9ZxXEvP8V1vw4UQ/0UI8Q0hxDHg6zdKcSHEeSFEdOXnN4QQx1eeYH8uhNjoBND3gCEhRHTlafEXwHmgRwjxr4QQH648uf501Xj/WAhxVQjxS2B4HZ8hAacQwgTYgSqQ24Br2TI2eI7/CfCfpZRpACll4i6Htxlz/FvA96SUFSnlODAKfOIux9nSbOQcSynflVJqpe4/ACJ3ObyWnON7tVlGgE9KKf/wdm8QQuxm+WnyKSnlfqAB/IOV174phDh0m0P/3soX8UMhRM9djusLwLmVn3cAfyalHGH5y9vB8pexHzgohPi0EOIgy0+X/cDngSdXjf9rQoiv3eIzfggUgHlgCviPUsrUXY5zO7BRc7wT2CmEeF8I8YEQ4rN3Oa7NmONuYHrV7zMrf3vY2Mh1rPGPgDfvclwtOcf3mrj5g3VsSV4CDgIfLmvQ2IEEgJTydrbIHwP/VUpZEUJ8Ffg28OI6xvOuEKIBnAX+BPABk1LKD1Zef2Xl36mV310sf+lu4L9rT0EhxI+0E0opv3Gbz/oEyzdMF+AHjgohfi6lvL6OcW4nNmqOTSx/98+zvFjfE0LslVJm7vBZmznHjwobNcfAskYKHAKeW+d4WnqO71VYFlb9XGethmpb+V8A35ZSrtsBsmJb0fgmy7aO9fCClHJR+0UI4bthjAL491LKP199kBDiX6x3bKv4XeAnUsoakBBCvM/yDfGwCcsNmWOWn+DHVr6/cSHEVZZv+A/vcNxmzvEssHpXE1n528PGRs0xQoiXgT8GnpNSVtZ5WEvP8YMIHZoAngAQQjwB9K/8/dY8eqYAACAASURBVG3g74sVA74QIiDu4BUTQnSu+vV14NKq1y7fxxh/Cvy+EMK1cq7ulXG9B3xRCGEXQriB19ZxrilWtF0hhBN4GrifsW0HJnhAcwz8NctaJUKIEMvb8usrv7fKHP8I+LIQwiqE6GdZmB+/j7FtByZ4cOv4APDnwOs32qS38xw/iPpJfwX8QyHEBeAYcBVASnlRCPEnwFtCCANQA/4AmBRCfBP4hpTyxA3n+udCiNdZfsqlgK+AvqjuueidlPKtFdvLr1e2EnngDSnlR0KI7wNnWN5a6NqNZue4hRr/n4FvrVyvAL4lpTx7r2PbJjzIOf4p8IoQ4iLL5ox/JaVMttIcSykvCCH+G3CR5XvxD7arJ/wueJBz/B9Y3iL/YGUupqSUr2/3Od4WueFCiC8AA1LK/7TVY1FsDGqOH362+xxvC2GpUCgUW41Kd1QoFIp1oISlQqFQrAMlLBUKhWId3LM3PBQKyWg0+gCH0vqcPHly8VGqoq3m+OFHzfH6uWdhGY1GOXHixoiBhxshxCNVfl/N8cOPmuP1s636VEopaTabNJvNNT2HDQZlTVAoFBvLthCWUkqy2SzFYpGzZ89y7tw5AoEAkUiEUCjE3r17sVg2pPSiQqFQANvIwVMoFEilUpw5c4a/+Zu/4ciRI5w9e5br169Tq9W2engKheIhZ1tolo1Gg7Nnz3L+/HlOnjzJ1NQUuVyOxcVFdu7cSTQaJRQK0dXVhdVq3erhKh4Q1WqVQqHA4uIip0+fBqCnpwen00lfXx8ej2eLR6h4lNg2wvLkyZO89dZbTE1NMTm5bJ8VQjA3N8fjjz9OT08PwWBQCcuHiHK5TDKZ5PLly3z3u99FSsmnPvUpOjo68Pl8SlgqNpVtISyFEIRCIXp7e8lkflP2UEpJPp9ndHSUWq3G448/rhbQQ0C1WqVSqTA+Ps6JEyeYmJhgdna5etalS5dIp9McPHhwi0epuBPNZpNKpUKtVmN+fp5yuYzL5cJms1EulymVShSLRdLpNLVajaWlJbT0a6PRSFdXF263m87OToLBIAaDYUududtCWBoMBgYGBnRN4/z58/qXmkqlOHr0KHNzc3zmM58hHF5vSxdFq5LP50mlUhw7doxvfOMbZDIZ5ufnEUKQSCQIh8O8+uq27iH2SNBoNMhms2SzWd555x3i8TgDAwOEQiESiQSJRILp6WnOnz9PJpNhYmJC9z9YrVZeffVVhoaGePnll3niiSewWCxb6sjdFsJSCEEwGKS3txev18tKeSZdYNZqNer1OqooyPZldcTDzMwMs7OzjI2NkU6nKRQKNBoNjEajChXbRpTLZUZHR0kmk1y7do2FhQXK5TJ+v59UKkUqlSKRSLCwsMDS0hL5fJ56fblvWr1eZ25uDoPBwKVLlzAYDLS1tdHR0YHZbMbhcGz69WwLYWk0GnnsscfYuXMnJ06cwGg00mg0lHB8iGg2m5w9e5Zr165x8uRJTp48SSqVIhaL0Wg0aDQaWCwWfD4fgUBAhYptAxYWFvjOd77DxMQEly5dIpPJYDQaMZlM1Ot1ms0mjUaDWq2m/6xRrVY5ceIEZ86c4fz583R0dPDJT36Sl19+mVAoxODgIEbjRvc/XMu2EJawvBU3Go2b/gUpNodms6lvv+PxOPPz8xQKBarVKgaDAbPZjN1up729nY6ODmw2251Pqth0pJSUy2UymQyzs7PE43Hi8TiZTIZc7uYmqDabDa/Xqx+r2TmbzSbVapVSqUQikaBardLX18fc3BxGo5Fms6mE5a1oNpvMzMywuLhIPB5XWuVDhqZVpFIpZmZmiMfjLCws0Gwut4g2m820tbXR3d3NG2+8weDgIH19d+peodhsGo0G9Xqd8+fP85d/+ZckEgnGx8fJ5XL69lpDM6k89thjfO5zn8NgMFCpVCgWi1y6dIlsNsvY2BjZbJZ0Ok0ul+PXv/41qVSKp59+mj179ugK1GaxLYQlLAelZzIZyuUyUkolLB8SpJTUajXK5TJLS0tks1kKhQKVym96XJlMJmw2Gx6Ph2g0Sn9/P06ncwtHrbgVmmZYKBSYmZkhmUxSKpX0LTeAxWLBZDJhNpuxWCx0dnYyPDyMwWDQ74FcLofD4SCdTtNsNikWixSLRRYXF7Hb7fT391MqlfT7QvNhbDTbQlhKKSkWi6RSKUqlkhKUDxHVapXTp08zPz/Pe++9x7Fjx9aEh8GyFuJwOHC73fh8Pnw+H2azeYtGrLgdmrY4PDzMV7/6Va5fv85f/MVfUKlUMBgMWCwWnnzySfr7+9mzZw+7d+/WHbfwG8306aefplwuMzc3Rzab5Xvf+x5Hjhwhk8lQqVQIBAK8++67dHV1cfDgwU1z9mwLYQnoKvqtUhuVh3T7onk9x8fHGR8fZ2JiAlh+QGoagxACm82GzWbDbrcre2WLIoTAaDQSDAY5dOgQTqcTu92uv2YwGIhEIjz22GM888wzfPKTn7ztuZrNJul0mqWlJX79618Dy971crlMLBZjbGyMRqPBvn37NuXaYJsISyklhUKBdDpNqVTS/y6EIBwO88orr6j0t22Gtr1KpVKcO3eOy5cvs7CwAPwmJEwTmKFQiJdffploNKrmeBtQq9XIZrPk83lgeesdCoWwWCwcPHiQ5557ju7u7o89h/aAlFLS0dFBf38/uVyOZDJJMpnk+PHjLC4usnv3bsLhMKFQaMOz97aNsCyVSmSzWd2WpZVn8/v9HD58mEgkgsvl2uKRKtZLo9GgWCySzWYZHR3lwoULpNPpm0wsUkp8Ph+HDx+mu7tb2Sq3AfV6naWlJQqFArC8Pff7/bhcLvbs2cOTTz55x3MIIXThFwwGiUQiTE9Pk0wmyWQynDt3jkqlwszMDABer1cJS1heMKlUiunpabLZLLA8AVpwajgcJhgMYjJti8tRgO4IyGazpFIpkskktVoNIYQuMN1uN8FgkJ6eHsLhMIFAQM3xNqBcLushQ6lUikwmg9vtRghxk1f849C29WazGavVqtuptdTIbDZLqVTSQ402mm1x50kpmZ+fZ3R0lFQqBSx7SJ1OJz6fj+7ubtrb27d4lIq7oV6v62EhWlylhiYw/X4/IyMjDA8P09PTo4TlNqFUKjE9Pc309DTxeJxkMqkHnN9NOUUtNMhisWCz2XRhWalUqFQqBINB8vk8pVJpTUD7RrGtvCKrnx4ej4fBwUEikYjyjG5DNK3BZDJhMpmwWCy6N1V7vVqt6sHMhUKBUqm0KRqE4v6w2Wy0t7fT3t6Oy+XSnTx3m5KsBaVnMhkSiQRLS0u3fN9mRcds28d0JBLh85//PENDQ6os2zbEYDBgtVqx2Ww4HA5cLhf5fH6NhrC0tMTY2Bg+n49YLAYsF1hQWVytjd/v58knn8TtdhOJRKjX65RKJarV6rofdppTd2lpidHRUU6fPr0p2uPH0dKaZbPZ1D1gmnZRrVYRQuB0OgmHw/j9fhU2tA3RvJ0OhwOv14vP57vpoddoNCiXy+TzeRKJBMlk8q5sXoqtwWg0Yrfb9fJq3d3dd63QNJtNlpaWSKfTFItFqtWqLiyFEHpQu8PhwOFwbMoDtKU1y2q1yqlTp5ienubUqVNcvnwZWNYu2traePzxxwmFQmobvg3RsjesVisHDhzAYrHw4YcfrtlqaQtkenqad999l2g0qldKV7QuJpMJl8tFJBLh9ddfZ2Zmhr/6q78iFoutO9umXq9z5coVJiYmiMfja16zWCx4PB7C4TCDg4P09vZuSuxtSwtLLV94YWGBXC5HuVzWA5NdLhderxen07lp6U6KB4emHdjtdkKhEO3t7TdlYkgpqdfrVCoVFhYWcDqdlMtl6vX6GvumovUwGAzYbDY6Ozv1WEkppW6/vBNSSr11zOrYavjNvaPdP3a7XWmW1WqVy5cvc+7cORYXF4FlW+Xg4CAjIyN0d3djs9mUDWsbY7PZeOaZZ9i5cyfXrl3j4sWLN71naWmJs2fPkslkGBsbAyAcDm9JTUPF+nE4HOzbt4+dO3fS19dHsVhkZGRkXcfW63VGR0c5deoUyWRyzWsmkwmr1YrT6cTlcuF0OjfFFNeywlLTKrSSXdrTxeVy0dHRQTAYxOFwqC34NsdkMtHR0YHdbsfr9WI2m/UcYU1zrFarJJNJnE4nqVSKQCCA3+/f4pEr7oTJZMLv9yOlxOVy0Wg01v2AazabZLNZFhYWbtIsNZuoFnu5WcpSSwrLarWqx95dvXqVa9eu6bXwOjs7OXDgAL29vcqx8xAghMDtdmMymdi/fz/ZbJZr164xOjqqpztqMZlTU1N897vfpbu7m3/6T//ppuYFK+4dIQR2u51ms7nuONlms0k8HmdiYuKmOpgdHR288sorDA0NberuoiWlTb1eJ51Ok0gkiMVixGIxSqUSQgh8Ph+9vb0Eg0Fls3oI0LziLpeL3t5edu3aRSgUWvMezSueSqX44IMPOHr0qG6WUWwPtFja9So4zWZTj6+8UbP0+XyMjIwwNDS0qRXzW0qzbDab1Ot1FhYWePfdd5mcnNSLK3R2duLxeNi1axe7du3C5/MpYfkQYTAY2LVrF06nE7/fTyQSYWpqirNnz+rb8mazqQcpf/DBB1QqFSKRCG1tbTidTtxu91ZfhuI+qVarzM3NEY/HWVpaotls6kHnNpsNp9NJV1cXIyMjenGOzaLlhGWlUiGZTPKLX/yC8fFxkskkUkra29vp6+tjeHiYXbt2KUH5kGE0Gtm1axcDAwN0dnYyODjIr371K65evUq5XKbRaOiViprNJsePHyeRSPD000+za9cuwuGwEpYPAdVqlcnJSWZnZ8nlctRqtTXCMhAI0N3dze7du3G73ZtqimspYallbIyOjhKLxUilUnoQeltbG319ffh8vq0epmKD0OxZoVCIRqPB7Ows3d3dLC0t6W0mms0mtVqNqakpvRD0zMwMBw4cwO/361W4FdsPre/O/Pw809PT+vZbU4za29vZu3cv0WgUk8m06QpTSwnLxcVFjh49yvXr1xkbG2NhYQEpJVarld7eXvbv309HR4fSKh9StNi5aDRKb28vS0tLfPDBB7pHVCv+Wq1WOX/+PAaDgVOnTuF0Ovm93/s99u7di91u35KFpLg/pJR62b4rV67o/XeklBiNRgwGA0NDQ3z2s59lYGAAi8XyaArLcrlMoVBgfn6ea9euMTs7S6VSQUqpVxzxer0Eg8F1B7Uqti9akY1QKMT+/fuZn5+nXq+Ty+WYmJjQq8w0Gg3959VbdcXGobV4qdVqpFIpisXibd+rVQzSQoi0UJ/VW+dGo0GlUtH79sRiMSYnJ5mfn6dcLgPL4YJa6mQkEiEYDG5JJExLCMtEIsHVq1c5fvw4f/3Xf83S0hLFYhGDwYDH48Hj8dDX18euXbsIBAJbPVzFJjEyMkJ/fz9zc3McOXKEyclJvvOd76zxjhaLRcrlMsVicU1jLMXGUKvVmJmZIZ1Oc+TIEb0NyK2wWCyEw2G8Xi+f+tSnaG9vx+/3rwn3KRaLJBIJRkdH+f73v08sFuPUqVNks1m9vml/fz+Dg4McPnyYZ555ZlNjK1ezpcKyXq/TaDRIp9NMTk4yNzdHPp+nXC5jtVqxWq0EAgG9yILmANLK1d8KLRVKq4WntmPbF4vFgsVioVwuEwwGyeVya+L0NMP/ao1SNbPbGBqNBoVCgUKhwMTEBAsLC0xOTjI1NXXbY6xWK6VSSdcKi8Ui6XQal8ulr+9MJsP8/Dxzc3PMzs6SSCR0GaAV/g2FQvT29tLW1obdbt+yNb2lwjKdTpPJZHjvvff4zne+o7fOtNvtDA8P4/F4GB4eJhAIYLFYuH79Okaj8WOfKkajkUgkgsfjWVNLT7F9sVqtdHd3U6lUNjVURPEb0uk0H3zwAXNzc/zwhz9kenpab019O4QQWCwWzGYzR48e1StMORwOBgYGGBwcZHFxUd92nz59Wt8pCCH0ikLPP/88r7/+Om1tbVuq/GyJsNQ0gVwuRzweZ25ujsnJSd3+tLrIgtbNT1PX74TmDa1UKjQaDaSUGAwG3eiv8sg3F82DrXE3c6B5vrXqQ1rx2Fv16VFsLNVqlUQiwezsLGNjY8zOzq4J6/k4TCYThUIBq9WK2+3Ws3mMRqMuLLUyjFqPLaPRiMfjwefz0dHRQW9v75Y/KDdVWGoeL60f8Jtvvsn777/P5OQkmUxGF26lUokrV65gsVi4evWqnji/HqOu0WjE6/Vit9sZGRkhGo3S1dVFX18fXq+X7u5utT3fRLS2Ahoej0efgzsxPz/PxYsXmZ6e5p133iGRSNyUubO6E6Ri48jn85w7d47JyUny+fy6BKWmWcJyq4lSqUShUMBgMOhdPSuVit5HZ3WtUofDwVe+8hUOHTrEyMgIDodjy9ObN01YaoKyVquRyWRYWFjg0qVLHDt2TFe9NTRPG6BXyL4VtxN4VqsVi8VCqVRiaWmJUqmk2zzb2tp0A7HWIVKxMUgpyefzxONx/btuNpt0dHTc8sbXtEZtEWazWSYmJrh+/TqnT58mk8nc1ApZm0dN+Kr53Biq1SoLCwssLi5SrVbXCEqtJ7j2s4bRaNSL/mrRCtqxN+Z734jJZOKxxx7j2Wefxel0tkTvpQ0fQbPZpFwuk8vlOHr0KPF4nPPnzxOPx7l8+TK5XG5d5eI9Hg9Wq5VarUatVtNV+tWTo235NFvKlStXiMfjnD17liNHjtDe3s6ePXsIBoPs378fj8dDe3u7akuxARSLRSqVCr/4xS94++238fl8hEIhurq6KJVKN22pms2m7mVNpVJ6HdPx8XGy2SzxeFw3rRiNRgKBAA6Hg0996lOMjIxw4MABvF7vlsTfPapojtSenh7a29sJBAJr4qDNZjM+nw8pJZOTk2SzWc6cOcP09PS6zq+Z1LZao9TYFGFZqVRIp9O8//77jI6OcvLkyY/VGG9Eq1ridrsplUoUi0XcbjfhcBiDwbCmlJfWs7hareod5jQ6OjqYmJigt7dXt4Vo8V+KB4eUUo9aOHv2LG+++SadnZ0MDQ2RTCZxu923bCFx9uxZpqammJyc1G3YuVxO35VomEwmvF4vgUCA559/ns997nM4nU5VCHqTMRgMesX74eFhotHomlRkq9VKe3s7jUaD06dPE4/HmZmZWZewNBgMa3wNrcCGC8ulpSXOnDnDzMyMXiZ+dSDr6qozLpeL7u5uLBaL3jRdCxvyeDzYbDZds9SO0YSllFJ/7fr166TTafL5PMVikfn5eaampqhUKkxMTJDNZvUt+eHDhwmHw0QiEfx+vx6uorg/tO20Vuk8nU4zNjZGMpkkk8nclJLYbDaZm5sjk8mQSqVYWlpa0w3QYrHodmifz8fOnTtpa2tj165duN3uu6poo7h/NG+10+lk3759PP/88/ruIZFIcPr0aT1OslqtMjExQSaTIZ1O33Qeo9GI3+9n9+7dGAwG8vk8TqcTr9fbUqayDReWuVyO48ePMzExwZkzZ27qD63VMwyHw/T09PDUU0/h8/mIRqO43W4GBwfxeDyYzWZMJpO+CG/0qq4WllevXiWVSjEzM8PCwoJedKFUKnH16lWEEHz00Ud4vV4SiQQDAwN8+tOf1oPglbC8f6SUehUpzd6VSCQQQvDhhx/ecgFognF1pRkhhF7eKxgM8uKLL9LX18eBAwdob2/XW4soNheDwYDb7cbn8/GJT3yCv/t3/65uBiuVSly6dIlkMsnc3Jy+Q6hWqzfZKrVQwPb2dj772c9isViYnp5GCEEgEGgpZ+yGC0ur1UokEqHRaOB0OrFarXqVY5/Pp3tH+/v7aWtrY+fOnTgcDsLhMFarFZfLpbeO0DQHTViu1iRW/y0UCmGz2fTGZs1mE4PBQKFQ0POMNTPA3NwczWaTgYEBvQitql7zYNAeaGazWRec2pb64xaA9mA0m8166bWenh7C4TC7du2is7NTt1mqohlbR7VapVwuE4/HuX79OslkklgsxrVr15iYmGBpaYlkMkmtVqNYLK5JHtCcrO3t7QwNDRGNRhkeHsZsNuvpjD6fb42ZbavZcGHp9Xp56qmnaG9v50c/+hGpVIre3l68Xi979+5laGiIkZERnnrqKf0powm91f+vBy0Xtb+/X9dOpJR84hOf4POf/zyJRILLly8zNTXF3/7t3+oG58uXL+sBr5qdRXF/aHNnNpux2Ww3hYZ8HDabDbfbrZfj6unp4eWXX6atrY19+/bhdrtVNMMWo2mQUkouXLiA1+vl1KlTHDlyhEKhQCKRWBNje2OGldZD59ChQ7zxxhu0t7fz2GOPYbFY9Pe0knMHNkFYGo1GXC4XwWCQvXv34vP5aG9vx+12s2PHDnp7e2lvb8flcj2wG18zDmu4XC5CoRAGg0EPIzpw4IBuFzOZTHR3d+Pz+TalpeajgLZ17ujoYHh4WA/j0tBaRTSbTb1SkKZtdHR00N7erkcrdHR00NnZic/nw263K21yCzCbzXg8HrxeLwaDYY09OhaLcfXqVaampkin03plqBvDi4xGo1531Ofz4ff7GR4eprOzE7/fj81ma4kQodux4SPTcju9Xi9/+qd/Sq1Ww2Qy6Z40rVn6RmoIWtqj1me4Xq/z+uuv69qnEAKn04nFYlGe8QeAEAKXy4XD4eBzn/uc7gWPxWL6AtLK8ZXLZYaGhvB6vboN8uDBgzzxxBP6TsNkMulBycqevDU4nU727t2L2+3mzJkzpNNpvdjN+++/z4kTJ6hWq7q2eaOg1KqHvfbaazz55JNEIhG6urrweDy0tbXp89zKbPjotCeK0Wiks7Nzoz/ulmiaprYlBFR3wA1G+84DgQB9fX168QQNt9utF/DVijprwrKvr4+enp4tHL3iRjQHW6FQwOfz6S0fblXcRgtG13YLJpNJb1vb29tLf38/XV1ddHZ26kJ0O9Daolyx7dHKcmk53hq1Wo1XX311zTZcq32oHGyth8fj4emnn2ZwcJBUKqWHeWkRJpOTk/p73W430WgUn8/H8PCwHuoVCATYuXMnHR0detWh7WRzVsJSsaHYbLbbag69vb2bPBrFvWKxWAiFQhiNRvr7+7HZbCSTSfL5vJ5xpaElfIRCIXbs2EEgEGDfvn0EAgFCoRAul2sLr+TeUcJSoVDcEYPBoNeXfeGFF9YUv0in02viJ61Wq55UomXIackl29knoISlQqFYF0ajEbvdztDQ0FYPZUtonSAmhUKhaGGUsFQoFIp1oISlQqFQrAMlLBUKhWIdKGGpUCgU60AJS4VCoVgHSlgqFArFOhD32hVPCLEATN7xjQ8XfVLKtq0exGah5vjhR83x+rlnYalQKBSPEmobrlAoFOtACUuFQqFYB0pYKhQKxTr4WGEphAgKIU6v/IsJIWZX/f7AS1YLIf5QCHFRCHFWCPG2EKJvHcdMCCHOrRzzlhCi4z4+/98KIf7lOt/bK4TIr/f9rcoWzPFXhBALqz7jH6/jmE2dYyGERQjxrZXPPCOEeP5eP68VUOv4tu/7IyHEqBDiihDi1Tu9/2OFpZQyKaXcL6XcD3wD+L+136WUVSHEg65adAo4JKXcB/wQ+Po6j3th5ZgTwL9Z/YJYZiM06P8LeHMDzrupbMEcA3x/1Wd8c53HbOYc/xMAKeVe4DPA/7lB99CmoNbxzQgh9gBfBkaAzwJ/JoQwftwxd/3hQoj/IoT4hhDiGPD1G6W4EOK8ECK68vMbQojjK0+wP7/TYKSU70opiyu/fgBE7nJ47wFDQojoytPiL4DzQI8Q4l8JIT5ceXL96arx/rEQ4qoQ4pfA8Ho+RAjxRWAcuHCX49sWbOQcPwA2Y473AO8ASCkTQAY49OAvZetQ65jfAr4npaxIKceBUeATH3fAvUrqCPBJKeUf3u4NQojdwO8An1p5ojWAf7Dy2jeFEHe6+f4Rd6+5fQE4t/LzDuDPpJQjLH95O1j+MvYDB4UQnxZCHGT56bIf+Dzw5Krxf00I8bVbXJcL+D+AP73xtYeMjZzjv7dys/9QCHG3zXY2fI6BM8DrQgiTEKIfOAg8jE2BHtl1DHQD06t+n1n52225V/X7B1LKxh3e8xLLN9mHYrnPhh1IAEgpP9ZOJYR4g+Un+XPrHM+7QogGcBb4E8AHTEopP1h5/ZWVf6dWfnex/KW7gf+uPQWFED/STiil/MZtPuvfsryNyYtt1D/kHtioOf4x8F+llBUhxFeBbwMvrmM8mznH/x+wm+Xt4CTwK5aFxMPGo7yO75p7FZaFVT/XWauhag1XBPBtKeUf3c2JhRAvA38MPCelrKzzsBeklIurzuG7YYwC+PdSyj+/4bP+xd2MbYWngL8vhPg6y5PZFEKUpZT/7z2cq5XZkDmWUiZX/fpN7s6etSlzLKWsA//7qnP8Crh6t+fZBjzK63iWtbuFyMrfbsuDMJhOAE8ACCGeAPpX/v42y0IlvPJaQNzBKyaEOAD8OfD6iq1o9WuX72OMPwV+f2ULjRCie2Vc7wFfFELYhRBu4LU7nUhK+ayUMiqljAL/D/DvHkJBeSMTPLg5Xt0P+XXg0qrXWmKOhRAOIYRz5efPAHUp5cX7GNt2YIJHaB0DPwK+LISwrphadgDHP+6AB+EF+yvgHwohLgDHWHkCSykvCiH+BHhLLHuxasAfAJNCiG8C35BSnrjhXP+BZdX6Bysq/5SU8nUhRIjlp8o9IaV8a8X28uuV8+aBN6SUHwkhvs+yjSoBfKgdo9k5HqQav415kHP8z4UQr7OsyaSArwC02ByHgZ8KIZosaxu/d6/j2kY8UutYSnlBCPHfgIss34t/cCeTxLbIDRdCfAEYkFL+p60ei2JjUHP88LPd53hbCEuFQqHYarZtoK1CoVBsJkpYKhQKxTpQwlKhUCjWwT17w0OhkIxGow9wKK3PyZMnFx+lKtpqjh9+1Byvn3sWltFolBMnbowYeLgRQjxSwUFF3QAAIABJREFU5ffVHD/8qDleP2obrlAoFOtACUuFQqFYB0pYKhQKxTpQwlKhUCjWgRKWCoVCsQ6UsFQoFIp1sBH9VRQKheJjOXfuHOfOncPpdOL3+9EKaQshcDqdmM1m+vr6cLvdWzzS36CEpUKh2FSklPyP//E/+PrXv04kEuHxxx/XhaXVaqW7uxufz8cXv/hFJSzvRK1WI51OI6XE7/djsTzwbp0KhWILqdfr5PN50uk0c3NzurA0m81UKhV8Ph+xWAy/34/T6cRms93hjBtPSwrLXC7H0aNHqdfrPPfcc3R03HMLYYVC0cIkk0lOnjyp/y6EwGg04vF46OnpoVAosHv3bnp7e7dwlMu0jLCUUlKtVsnn8ywsLJBIJGg0GtRqta0emqKFqNfrNJtNisUitVqNZrNJs9mkXq9Tq9UwmUz/f3tnHtzmeSb234sbIAEeuHmCpCiJOilHil1bWVvxpIntNK1r57B329nam2k23cx2trt/7LSz3bYzvXbame60m3UnnUk3U2/bTeOcluVIkS1ZthNRRyyJkige4AUQBAgQ90Xg7R/E91mnRckiAcnfb4ZDHN/xAi++53ue97mwWCyYTCYcDgc6nebDbESamprweDzq81KpRDabVedTSkkkEiEcDjeEoIQGEZaKUJycnOTIkSMkEgmCwSA2m40nn3yy3sPTaBBWVlaIRqNkMhlOnDjB3NwcmUyGfD5PKBRidnYWv9/P7t276e3t5dlnn6W1tbXew9a4DiEEDz30EC+99JL6WigU4vDhw6TTaXK5HMVikffff5+ZmRk8Hg9DQ0N1HPEqDSEsFc0glUoxMTFBKpUik8lgMBjQKrlrVKtVSqUSxWKRpaUllpeXmZqaYmpqimQySSaTYXZ2lvHxcXp7e7Farej1elZWVuo9dI1b4HQ62bJli3p9GwwGmpqaKJfLFAoFqtUq8XgcIQS5XK7Oo12lIYRlPp8nHo8zPj7O22+/jd1u57nnnqOrqwun01nv4WnUmUgkwltvvcXi4iInT54kHo8zPz9PKpViZWWFlZUV9YJKpVJcvHgRs9lMqVSq88g1bkUgELhG6+/o6GB8fJyFhQXOnDnTMALyahpCWJZKJdLpNPF4nGAwiN/vp6enh4GBAZqamuo9PI06IaVESkkqleLcuXPMzc3x9ttvs7S0xMrKCpXKtc34hBCUSiWWlpZIJBI3vK/ROLS2tl4jLIvFIl1dXVSrVYxGI4DqIW8UGkJYTk1N8fOf/5yJiQlaW1txu91YLBYMBkPDfWEa608mkyGdTrO4uMjk5CRTU1O8++67xONxUqkUlUrllsszVqsVv9+P2+3GYGiIn7fGHSKlRKfT4fV66erqwuFw1HtIQIMIS0VjyOVytLS0qLGVmrD8ZJLL5YjFYly5coVjx44xOzvL2bNnyeVyt9UWzWYzbrebtrY29Hr9Bo1Y414jhMDpdNLZ2YnNZqv3cIA6C8vl5WXV/K5Wq3R2drJ9+3Z8Ph/d3d20trbeVjuoVCrqRaQs6DscDkwmE8vLy2SzWVVTuV4b0ev12O12TCYTLpdLM/nrTDabpVAocPr0aU6ePEkoFOLSpUskEglKpdJHOvv0ej16vR6Xy8WuXbvo7+/XkhnuI/L5PNPT08zOzlIqlTAYDPj9fvr7+xsmoqFuwlJKyeLiIjMzM2pMZX9/P9/61rew2+3odLo1aZXlcpl4PE65XCabzSKEwGQyYTQaiUajzM/PEwqFmJ6evuFiM5lM9PT04HA42LVrlyYs64iUkmQySTwe5+jRo3zve98jn8+TzWaRUlKtVj9yf71ej9lsxu/38+ijj+L3+xsi60NjbWQyGS5evEg4HKZQKGCz2QgEAuzYsaNhnLx11SwzmQyLi4uk02kAdDqdqiFcT7lcplKpqMJPQQknKZfL5PN5hBCEQiGam5uZnZ1lcXGRhYUF5ubmbjim0WgkHo9jt9vJZrN4vV4CgQBdXV3r96E1gNVwoHw+T7lcJhaLkcvlCAaDRKNRpqam1PfW6qQxGAxYrVbsdjsej0czw+8zFOtOCU4XQuBwOHC5XA1z06urZhkKhTh37hwLCwvA6jrFrbTJXC5HLpfj4MGDvPbaa6qWeL0ZLoSgvb0di8VCOp0mm82SSCSIxWI3aCcGg0E12T0eD3a7nZdffpmvfe1r6/vhNSiXy0QiEZaXlzl+/DihUIizZ88yMzPD0tISqVTqjo5nsVhobW3F7/ezefNm7Ha7JizvIxwOB7t376atrY1oNIpOp6Ozs5PBwcGGmce6CEtFsOVyOZLJJEajkZ6eHtra2igUChgMBvR6PTqdDp1Oh5SSWCzG4uIikUiERCKhHmtlZUVdj1xZWUGn06mJ94ppBtw0bksRziaTiaamJgwGg5ZeuQFIKSkUCoRCIZaWlpieniYUCqlzm8/nkVKqVobJZKKtrY1qtUoikbgmzVHBbDbT2tqKzWbDaDQ2zAWmcXOU+SsUCqRSKSKRCNlslmKxqDp3lf+NwoaPREqppqiFw2FmZmbYt28fTz75JGazmXA4rF4cJpOJ5uZmqtUqR44cUVPc8vm8erx8Ps/i4qKaT2qxWHjkkUcIBAJqHvHp06eZnZ29Yc1SSkkul1MXlB0OR8Oo/A8q1WqVcrnM4uIiP/nJT5iZmeH06dNEo1FKpZIqCOFDzb+rq4unnnqKYrHIT3/6UxYXF9V5U3C73ezatYu+vj5NUN4HFItFCoUCly5d4vjx48zOzvLuu++Sy+Vobm6mra2t4a7FugjLVCrF8vIyy8vLahaGxWKhUqmwuLiI2WxWvyhFCKZSKVUoKt4xKSUmk4lisah6wq1WKy6XC4/HQ6VSoVqt4nA4MBqNVCqVG9bA9Ho9RqORlpYW3G43Vqt1Y7+QTwiKk0bRJKLRKKFQiPn5eWKxGMlk8pptgWs0y6amJvR6PQaD4ZbFMYQQVCoVCoUCOp0Oo9GohZ41CMp1rMydkogSi8UIBoMsLCyQTCaRUtLR0YHH41GtwkZhw4VlsVjk4MGDnD17ljNnznD58mV1kb9YLLK8vIzP5+MrX/kKfr9fNY+NRiMWi4W9e/eyffv2a453dbaGyWRiz549eL1eNSk/lUpx4cIF1exXhKbZbGZgYACn08nzzz/P8PAwgUBgo7+STwTZbJZkMsn4+DiHDh0iFApx4sQJkskk2Wz2pvsoyyjZbJbjx49TLBaJRCKk0+kbbnpzc3OqIA4EAng8HrZt29Zw2sknkWq1SjKZpFwuY7VaMZlMTE5OcvHiRUZGRnjjjTfI5/Pk83m8Xi8vvvgimzZtarhrcUOFpbKuODMzw+joKKFQiGQyycLCAsFgUDWp8/k8mUyGYrGIlFJdV7RarXR2drJjxw71mKVSiVQqdY3ptmnTJtra2tQ4S7fbTWtrKzqdjlwuhxCCarWKEAK73Y7T6SQQCLB161YtfGidUDSJSCTCuXPnWFxcJBqN3jYHWElhDIfDlMtltTTb9WQyGcrlMn6/n/n5eXQ6nZbu2CBUq1UymQyFQkF9LZFIMD8/r/5JKbFarVitVjZt2sTQ0FBDVUmHDRSWKysrRCIRlpaWCAaDTE9Po9Pp6OnpUdcsFxYWGBkZwePx0NLSgs1mQ6fTYTAYOHDgANu2bcPn8+H1etXjKuXdFNNNp9OpZrdyjCeffJKuri7Gx8c5cuQI8XicS5cuqUHpra2tOBwO7HZ7Qy0oPwgUi0VKpRITExOcOnWKy5cvMzo6SiaTuW2hi2KxqFaeMRgMH1nfVDG/p6am+NGPfsS2bdsYHh7Wbn4NQCaT4dVXX2V8fJxHH32UgYEB3nnnHd566y0WFhaoVqu4XC7V19Df3//JNsMrlQqJRIJoNEosFmNpaQmXy0VbWxt9fX089NBDTE1NMT09rYb+mEwm1SO+detWtm7dekfntFgsWCwWhoaGGBgY4PTp0wSDQcxmMxMTE+o2NpsNq9XacJPzIFAqlcjlckQiES5fvsz4+DjhcJhisbimfUul0prWHZWllVgsxtmzZ9HpdGs6h8b6k8/nee+99xgZGVGbkV25coULFy6o5djsdjtDQ0MEAgFcLlfDaZWwgcKyWCxy8eJFpqeniUajlMtlvF4vQ0NDDA4O0tHRQVNTk6qKd3Z2quuVH5dyuUwmk1FNwKWlJUqlkpYOt04o2TjKRXLu3DmCwaCaungvzGPF0dPT04Pb7cZsNmOz2TCZTNjtdgYGBjRnXZ0pFArMz88zNzdHPB4nk8kwOjpKPp9nfHycbDaLwWDA5XLR3d2tZl41qjWw4cJybGyMeDzOysoKPp+PnTt3smnTJvx+P52dnWzbtu2en1tJhYxEIoyOjqoZQ5qwXB+klCwvL5NIJDh8+DCvvfaa6ny5VyjOn4GBAYaGhmhtbaWtrQ2j0YjZbMbj8WjOnTpTKBSYmJhgZmZGrRg1OjrK3NwcsViMbDZLa2sr7e3t9Pb28sgjj+Byueo97Fuy7sIym80yPT1NOBxmbGyM6elpDAYDXq+Xvr4+hoaG8Pl89zzEQ0qphqVMT08zPj5OMBikv78fs9lMX18fDoeDrVu34nQ6Gyb/9H6mWq2STqcpFAqMjY0RCoXUXN97YRLr9Xo14WDPnj34fD61mZXNZlPDi5SGV0pdRI2NoVQqkc/nyeVyLC0tEY1GOXHihOrIVQqdKFZkU1OT+ptZWFjgwoUL+Hw+ent7G/JGt+7CMh6P84tf/ILp6WlOnDhBNBolEAjQ2dnJ3r17OXDgAHq9fl2E5fnz5xkZGeHkyZMcO3aMnp4eHn30Ufr6+vjyl79Me3u7as5pWubHR3HiJRIJ3nnnHcbGxrh06dI90yj1ej1utxuXy8VLL73Evn37aGlpobm5Gbi2WKwQQmtWtsFks1kWFxcJh8OcOXOG2dlZfvjDHxKPx8nlcuh0Onp7e9m+fTuxWIxoNMrMzAznz59nYmKCw4cP09vbq/osGo11F5aFQoHJyUmmp6fJ5XJq/rbi5b7XgcPValUtzjE7O6t627xeLz6fj46ODrxeL3a7HZvNpqY8ahfW3aN4olOpFOfPnycSiTA1NUU4HFarBt0pyj5Ka1TFEdfV1YXX68XpdGK327FarVoEQ51RWnvEYjEmJiYIhUKMjY0Ri8XUuatWq+h0OpxOJx0dHdhsNpqbmzEYDKTTaZqbmwkGg5TLZebn5wFoaWlpKCVm3X9lsViM119/nbm5OYrF4jVCSfF030uKxSLHjh1jbGyM0dFRZmZm6Ojo4JlnnqGjo4Ndu3bR3t6ueuU0Pj6FQoGZmRlmZ2f59re/zZUrV0gmkxQKhbty5lwvXJXSa263m89+9rN0d3erdQ61m1z9yWazpNNpzp49y8GDB5mbm+PUqVMYDAa1j1Y0GgVgx44d7N+/n2KxSLFYZHp6mp6eHubn53nzzTfVYigDAwPs3bsXn89X50/3IesuLCuVCtlsVs3SMBqN11QMqlQqH0uzU3KNlfMonf5mZ2eJx+Nks1lMJhOdnZ1q6S6Hw6HlD99DKpUKmUyGZDLJ0tISS0tL16Sg3gzFQaOU5YPVEJObxV4aDAba29txu934/X68Xi9Wq1UTlA3C8vIy4XCY+fl5wuGwmmzQ3NyM1+vFYrFgt9sRQqgx1EodgEKhQG9vL1JKNapBSYdNJpM0NzdjNpsbQrGpi/2iLARnMhmWl5exWCx3HS6Qz+eZn58nkUjw3nvvEYlEOHr0KDMzM2r1GbfbzWc+8xn1rqWY/xr3hnw+z9TUFMFgkHQ6TbFYvK1GqWRqKEUT9Ho9Z86cYXp6GiGEmrkF4HK5eOKJJ+ju7ubxxx/H7XY3bHjJJ41qtcrx48d54403CAaDjI6Oqtk43d3dPPfcc/j9flXo9fT00N7eTrlcplwu43a72bJlC8lkkv3797O8vMwHH3xAKBRSNdPNmzfT2dlZ749an0Ia5XKZYrFIJpMhkUiomTZrWbtUCjIovcaVFLpYLKZWXU8kEmo/H+Wu5nK5aG5upqmpSSuucI9RCvnm8/lrui4qmTdK1Xul7J5STcjn86kFXvV6PWNjY9cISmUfq9WKz+fD7/fjdDppaWmp8yfWgA+vxaWlJSYmJtRUZYvFgsvlwufz0dXVRWdnJ06nU1WKTCaTGrVgsViuyZ5bXFzk4sWLqkddCIHP51MjZuppTWy4sKxUKkSjUZaXlzl06BDz8/N8+tOf5nOf+9w11YZuRTKZZHl5mUgkwpUrVwiFQrz11lsUi0VaWlowGo08/PDDGAwGuru7cbvd7Ny5E4/Ho1WhqQOKQ62lpQWn04nX61VbeLjdboxGI0ajkWKxyMTEBBcvXlT3bWpqUjO89u3bh9frbZjmVRqrFkWhUGBubo7Lly9jt9sZHBxkx44dvPDCC7hcLjU5QLHylCUX5capoFh7Ho8Hl8vF8vIyr776Kj/72c/UGpetra11DfHbEGGpCCjlv1LMYnJyUs3kUWpUmkwmtdAF3LjYr1Q+D4fDauzkyMgIAHv27KGtrY3Ozk7a2toYGBigo6NDjevSWD8UZ93VSxxCCHXpw+1209HRQSAQ4IknnsBms6nWhJI0cH3LUyW/3+l04vP5VOGqUX8UC1GJgkgkEjQ1NdHe3k5/fz8HDhxQQ7puxvVdEZSSeooVmEgkSKfTXLp0iVAopC7X1ZN1F5ZGo5H29nay2SylUkkNM6hUKkQiEfL5PFarlUqlQm9vL/v37yeXy3Hy5EmSySTpdPqaRf90Ok0qlcJms9HW1samTZv45je/idlspru7m+bmZlwuFzabjZaWFjX4VWP9sNvt7Ny5k+7ubux2+zW1KRXHms1mw+Fw4HA4cDqdCCHUZZTXX3+dyclJRkdHAdTlkuHhYZ5++ml1nctisWiWQYNQqVTUOpSxWAxYXVveuXMngUDgYztQzWYzBw4coKOjA5/PRyQSqft1vO7CUqns43A4SKVS6HQ6qtUqlUqFZDJJMplECEE+n2fXrl0MDQ2RSCQ4evQoCwsLqkBVUEIOhoaGOHDgwDVmnVJdXQsL2lgsFgt9fX2Uy2V6enrUykBCCNra2tT1qKvjIVdWVtT5P3r0KGfOnGFxcVE9XmtrK5s3b+bpp59Whazm/W4cqtWqGk+bSqUQQtDS0kJfXx9er/djz5XRaGR4eJjOzk4ymQzxeByfz3eN42+jWXdh6fP5eOGFF1hYWOD9998nkUioVZPT6TSZTAaAmZkZtdRaoVDg3LlzpFIptU6h8gUpHRgHBwfp7e3F5XLhdruxWCxYrdZbdofUWD90Op0a3nF9HUmbzXbT6ua5XI4LFy4wNzdHOBxmeXlZTYns7Oxk586dDA4O0tLSgtVq1TTKBkMIgdlsVgvfWCwWkskk586do1gsEggEaGlpwe/335HiUi6X1d/DoUOHmJycVMOMdDodw8PDD66w7Orq4hvf+AaLi4vo9Xqmp6cxm83q44WFBaLRKOPj44yPj3PixAngwzL06kBrF9zg4CC7d++mv7+fTZs20drais/n0zTJOqLT6dT1pLU6YLLZLGfOnGFqaoqZmRnVlAMIBAJ85jOfYcuWLVpL2wbGbDbT1NSk5uXH43F++ctfkkql6O3tVaMX7uTaLBaLjI+PMzc3xw9+8ANOnTqlesF7enr46le/uo6f6KPZEAePcjH19fWpaU6K5ysWi3Hx4kW13YOiXV59gQgh6Orqor29nd27d7N79268Xq+aiaOZZ/cPlUqFUqlEMplkenqamZkZtWeOzWZTs3UGBgZwu92aRtmgKKmLUkr6+/sJhUIUCgU1HHBkZEQtt9be3q6mON6KXC7H/Pw88XickZERQqEQ8XgcKaWa3lrv38OGhQ7ZbDYee+wxisWi6oBJJBIkk0kOHjxIOBwml8uRTqdv6O9tMpnYv38/u3btYt++fezdu/cac1sTlvcP5XKZRCJBKBRiZGSEYDBIJpNBr9fj8XhwOp0MDw+zf//+G8JLNBoHvV5Pf38/vb29ZLNZWlpauHTpEiMjI8zOznLhwgU8Hg+JRILe3l6eeeaZjxSWsViMw4cPq8U3lNa4BoOBXbt2MTw8zNDQ0CdDWOp0OjWKXymztbKygl6vp6enh+HhYbW80/XhQgaDgS1bttDb26sGt2rcfyith8PhMAsLC6TTafL5vNq1MRAI0NfXR0dHR0MVUNC4OcrNzOl00tvbq1oMShy1lFJ1zt2q2Ek6nSYajapr15FIhEKhgJQSv9+PxWJhYGBAbSxYTzZMWCoVkRUTWwiBw+GgubmZp556iv3796sZAdcjhFCrYGutH+5PKpUKKysrasGEYDBIOBwmk8moFYRefPFFvvCFLzRkSwGNmyOEUNu2RKNRHn/8cU6dOsUrr7yCx+PhscceIxAIqO2rr+fSpUt8//vfJx6PMzk5qcZgu91uvvjFL7Jlyxb27NnD4OAgZrP5k6FZAjcs1CuBzMrdR+PBpVKpUCwWSafThEIhIpGI2l/H6XSqRWH9fn+9h6pxhyi9rhSLMBQK4fV68Xg8qkJ0vWappCwnk0lmZmbIZrNUq1XVl2G1WgkEAgQCAXw+3y2F7UaiFQLU2BDS6TThcJjR0VEOHz6smmlKn+jt27evS0sRjY1Dyfveu3cvX//617HZbLcUcqlUimQyyZUrVxgZGaG1tZWHH34Yp9PJnj17aG9vp6enB4fDUfdgdAVNWGpsCCsrK+TzebVPfC6XU9eulfYijaA9aNw9SuKB0+lk8+bNGAyGWy6bKZaG0sFTp9OpKbE7duzA6XSqUTONgiYsNeqCEAKTyYTFYlELbGiOuweDpqYmBgYGVF/D1TVLFRRB+PnPf56enh61o6vVasXr9aqx2I2EJiw1Noyb9chR0lM/quiCxv2FyWS6refabDZjNpvVpoH3A5qw1NgQTCYTDoeDtrY2PB4P6XRa652jcV+h/Vo1NgSTyYTdbqelpYX29nb0ev01BVI0NBodTVhqbAhGoxGbzUZ/fz/PPvss+XyeYrGIw+HA6/XWe3gaGrdFE5YaG4LJZMJkMrF792527tx5Q5EUDY1GR/uVamwo69H+WENjI9B+tRoaGhprQBOWGhoaGmtAXF/hZ807ChEFpu/tcBqeXimlu96D2Ci0OX7w0eZ47dy1sNTQ0ND4JKGZ4RoaGhprQBOWGhoaGmtAE5YaGhoaa+AjhaUQwimEOFv7WxBCzF/1fF3q/gshviKEGBVCXBBCvLqG7YNCiHNCiA+EEG8KIXwf49x/KoT4wzVs98dCiHEhxGUhxOfv9nyNQD3muHbe54QQUgixdw3bVmrjOS+E+BshxNpaSN78WN8VQjx/m23ahBCv1X5TvxJC7Ljb8zUCGz3HQogeIcRRIcSZ2nf49Br22eg5/s3a2M4JId4VQuy+7YGllGv6A/4U+MPrXjOsdf81nmMQOAO01Z571rBPEHDVHv9b4M+ve18Aurv9jDfZZhvwa8AM9AETgP5efg/1+tuIOa4d0w4cA94H9q5h+8xVj/8X8Ad3O0bgu8Dzt9nmz4B/WXu8FThS77m5n+YY+O/A79YebwOCDTjHj14lZ54Cfnm7496xGV6T2n8phPgl8B+v18Zqd4ZA7fFv1e7MZ4UQrwghbleg7uvAf5NSJgCklIt3OLxjwCYhRKCm9f0VcB7oFkL8kRDiZO1u8q+uGu8/F0KMCSHeAbas4Rx/F/jfUsqilHIKGAc+fYfjbGjWeY4B/g3wH4DCXQzvOKtz/IQQ4rgQ4sfAqBBCL4T4s6vm+B/XxieEEP+19ns4DHjWcI5twC8ApJSXgIAQ4oFKYF/nOZaA0iemBQjd4fDWfY6llO8qcobVm3bX7fa52zXLLuBRKeUf3GoDIcQQ8FXgMSnlMFABfrP23nduYX5tBjYLIU4IId4XQnzhDsf1ReBc7fEg8BdSyu2sCsFBVoXaMPApIcRvCCE+BXyt9trTwL6rxv8NIcQ3bnKOTmD2qudztdceNNZljoUQDwHdUsqf3emAhBAGVrUAZY4fAn5fSrkZeBlISin3sTqPXxdC9AHPsjr/24B/yKpGoRzvXwshvnSTU/0a+Pu1bT4N9LKGi+k+ZL2u4z8FfksIMQe8DnxrrQPawDm+mpeBg7cb293mhv+NlLJym22eBD4FnBSrRV+twCKAlPJ3PmI8g8ATrE7kMSHETinl8m3OdVQIUQE+AP4F0ApMSynfr73/t2t/Z2rPm2vnsQOvSSlzALU7GLUx/uVtzvmgc8/nWAihA/4z8Nt3OBarEOJs7fFx4H+wekH8qqbdw+r87rpqraqF1Tn+DeCva58lJIT4hXJQKeWf3OJ8/x74L7VznmP1d3O77+J+ZL2u4xeA70op/5MQ4m8B3xNC7JBS3ti69UM2eo4BEEIcYFVY7v+o7eDuhWX2qscrXKuhKr0BBPA/pZR/fAfHnWN17aAMTAkhxlj9Mk7eZr8DUsqY8kQI0XrdGAXw76SUr1y9kxDin97B2BTmge6rnnfVXnvQWI85tgM7gLdqF54P+LEQ4ktSypGP2C9f02pUavtfP8ffklIeum672zoXrkdKmQL+UW1/AUwBk3d6nPuA9bqOXwa+ACClfE8IYQFc1ITsLdjQOa7ttwv4DvCUlHLpdtvfi9ChIKuqsmJi9dVePwI8L4Tw1N5rF0L03uZYP2RVq0QI4WLVLJ+sPb/0McZ4CHhJCNFcO1ZnbVzHgL8nhLAKIezA31nDsX4MfE0IYa6ZAIPArz7G2O4HgtyDOZZSJqWULillQEoZYHWt6EtSypHanBz5GGM8BPyuEMJYG8tmIUQTq3P81dp6lx84cLsDCSFaxYde4t8BjtUE6INMkHt3Hc+wqpEqZrwFiDbYHPcAPwD+gZRybC0nvxfC8v8B7UKIC8DvAWMAUspRVk3iN4UQHwA/B/y1gd5qreMQsCSEGAWOAn8kpVyqCc677q4upXwTeBV4TwhxDvg+YJdSngb+D6trVAcvzLPAAAAA00lEQVS5SoO91ZqllPIC8H+BUeAN4J+swZS537mXc3wr/KxqN3fLd1idk9NCiPPAK6xaTq8BV2rv/RXwnrLDR6xnDQHnhRCXWV0/+/2PMa77hXs5x/+M1fXEXwN/Dfy2XHU7N9Ic/wngBP5CrDquPsqyWT1WzXXe0Aghvgj0Syn/vN5j0VgfhBC/B8xIKX9824017kvu9zm+L4SlhoaGRr3R0h01NDQ01oAmLDU0NDTWgCYsNTQ0NNaAJiw1NDQ01oAmLDU0NDTWgCYsNTQ0NNbA/wcTUCA88aNFvQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors(cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "This tutorial showed how to optimize the hyper-parameters of a neural network using Bayesian optimization. We used the scikit-optimize (`skopt`) library which is still under development, but it is already an extremely powerful tool. It was able to substantially improve on hand-tuned hyper-parameters in a small number of iterations. This is vastly superior to Grid Search and Random Search of the hyper-parameters, which would require far more computational time, and would most likely find inferior hyper-parameters, especially for more difficult problems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "These are a few suggestions for exercises that may help improve your skills with TensorFlow. It is important to get hands-on experience with TensorFlow in order to learn how to use it properly.\n",
    "\n",
    "You may want to backup this Notebook before making any changes.\n",
    "\n",
    "* Try and run 100 or 200 iterations of the optimization instead of just 40 iterations. What happens to the plotted landscapes?\n",
    "* Try some of the other optimization methods from scikit-optimize such as `forest_minimize` instead of `gp_minimize`. How do they perform?\n",
    "* Try using another acquisition function for the optimizer e.g. Probability of Improvement.\n",
    "* Try optimizing more hyper-parameters with the Bayesian optimization. For example, the kernel-size and number of filters in the convolutional-layers, or the batch-size used in training.\n",
    "* Add a hyper-parameter for the number of convolutional layers and implement it in `create_model()`. Note that if you have pooling-layers after the convolution then the images are downsampled, so there is a limit to the number of layers you can have before the images become too small.\n",
    "* Look at the plots. Do you think that some of the hyper-parameters may be irrelevant? Try and remove these parameters and redo the optimization of the remaining hyper-parameters.\n",
    "* Use another and more difficult dataset with image-files.\n",
    "* Train for more epochs. Does it improve the classification accuracy on the validiation- and test-sets? How does it affect the time-usage?\n",
    "* Explain to a friend how the program works."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016-2018 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}