Tennis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Collaboration and Competition\n",
    "\n",
    "---\n",
    "\n",
    "You are welcome to use this coding environment to train your agent for the project.  Follow the instructions below to get started!\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "Run the next code cell to install a few packages.  This line will take a few minutes to run!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mtensorflow 1.7.1 has requirement numpy>=1.13.3, but you'll have numpy 1.12.1 which is incompatible.\u001b[0m\r\n",
      "\u001b[31mipython 6.5.0 has requirement prompt-toolkit<2.0.0,>=1.0.15, but you'll have prompt-toolkit 3.0.18 which is incompatible.\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!pip -q install ./python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The environment is already saved in the Workspace and can be accessed at the file path provided below. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  A few **important notes**:\n",
    "- When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```\n",
    "- To structure your work, you're welcome to work directly in this Jupyter notebook, or you might like to start over with a new file!  You can see the list of files in the workspace by clicking on **_Jupyter_** in the top left corner of the notebook.\n",
    "- In this coding environment, you will not be able to watch the agents while they are training.  However, **_after training the agents_**, you can download the saved model weights to watch the agents on your own machine! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: TennisBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 8\n",
      "        Number of stacked Vector Observation: 3\n",
      "        Vector Action space type: continuous\n",
      "        Vector Action space size (per agent): 2\n",
      "        Vector Action descriptions: , \n"
     ]
    }
   ],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "from maddpg import maddpg_train\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Unity environment\n",
    "env = UnityEnvironment(file_name=\"/data/Tennis_Linux_NoVis/Tennis\")\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "states = env_info.vector_observations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 100\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "1631\n",
      "Episode 200\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "3283\n",
      "Episode 300\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "5010\n",
      "Episode 400\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "6650\n",
      "Episode 500\tAverage Score in 100 episodes: 0.01\t Episode score: 0.10000000149011612\n",
      "8287\n",
      "Episode 600\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "10023\n",
      "Episode 700\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "11474\n",
      "Episode 800\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "13151\n",
      "Episode 900\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "14571\n",
      "Episode 1000\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "15991\n",
      "Episode 1100\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "17411\n",
      "Episode 1200\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "18831\n",
      "Episode 1300\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "20251\n",
      "Episode 1400\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "21671\n",
      "Episode 1500\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "23091\n",
      "Episode 1600\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "24580\n",
      "Episode 1700\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "26076\n",
      "Episode 1800\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "27503\n",
      "Episode 1900\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "28923\n",
      "Episode 2000\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "30397\n",
      "Episode 2100\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "31965\n",
      "Episode 2200\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "33577\n",
      "Episode 2300\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "35016\n",
      "Episode 2400\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "36436\n",
      "Episode 2500\tAverage Score in 100 episodes: 0.00\t Episode score: 0.0\n",
      "37856\n",
      "Episode 2600\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "39418\n",
      "Episode 2700\tAverage Score in 100 episodes: 0.01\t Episode score: 0.0\n",
      "41105\n",
      "Episode 2800\tAverage Score in 100 episodes: 0.02\t Episode score: 0.0\n",
      "42991\n",
      "Episode 2900\tAverage Score in 100 episodes: 0.03\t Episode score: 0.0\n",
      "45058\n",
      "Episode 3000\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "47299\n",
      "Episode 3100\tAverage Score in 100 episodes: 0.04\t Episode score: 0.10000000149011612\n",
      "49589\n",
      "Episode 3200\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "51813\n",
      "Episode 3300\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "53950\n",
      "Episode 3400\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "56112\n",
      "Episode 3500\tAverage Score in 100 episodes: 0.03\t Episode score: 0.0\n",
      "58247\n",
      "Episode 3600\tAverage Score in 100 episodes: 0.04\t Episode score: 0.10000000149011612\n",
      "60398\n",
      "Episode 3700\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "62564\n",
      "Episode 3800\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "64713\n",
      "Episode 3900\tAverage Score in 100 episodes: 0.05\t Episode score: 0.10000000149011612\n",
      "66964\n",
      "Episode 4000\tAverage Score in 100 episodes: 0.04\t Episode score: 0.10000000149011612\n",
      "69125\n",
      "Episode 4100\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "71199\n",
      "Episode 4200\tAverage Score in 100 episodes: 0.06\t Episode score: 0.10000000149011612\n",
      "73627\n",
      "Episode 4300\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "75781\n",
      "Episode 4400\tAverage Score in 100 episodes: 0.05\t Episode score: 0.10000000149011612\n",
      "78015\n",
      "Episode 4500\tAverage Score in 100 episodes: 0.05\t Episode score: 0.0\n",
      "80332\n",
      "Episode 4600\tAverage Score in 100 episodes: 0.05\t Episode score: 0.10000000149011612\n",
      "82546\n",
      "Episode 4700\tAverage Score in 100 episodes: 0.04\t Episode score: 0.10000000149011612\n",
      "84675\n",
      "Episode 4800\tAverage Score in 100 episodes: 0.05\t Episode score: 0.0\n",
      "86875\n",
      "Episode 4900\tAverage Score in 100 episodes: 0.05\t Episode score: 0.10000000149011612\n",
      "89042\n",
      "Episode 5000\tAverage Score in 100 episodes: 0.05\t Episode score: 0.10000000149011612\n",
      "91262\n",
      "Episode 5100\tAverage Score in 100 episodes: 0.04\t Episode score: 0.10000000149011612\n",
      "93402\n",
      "Episode 5200\tAverage Score in 100 episodes: 0.06\t Episode score: 0.10000000149011612\n",
      "95770\n",
      "Episode 5300\tAverage Score in 100 episodes: 0.05\t Episode score: 0.0\n",
      "98172\n",
      "Episode 5400\tAverage Score in 100 episodes: 0.05\t Episode score: 0.0\n",
      "100000\n",
      "Episode 5500\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "100000\n",
      "Episode 5600\tAverage Score in 100 episodes: 0.03\t Episode score: 0.0\n",
      "100000\n",
      "Episode 5700\tAverage Score in 100 episodes: 0.05\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 5800\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "100000\n",
      "Episode 5900\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "100000\n",
      "Episode 6000\tAverage Score in 100 episodes: 0.06\t Episode score: 0.0\n",
      "100000\n",
      "Episode 6100\tAverage Score in 100 episodes: 0.05\t Episode score: 0.0\n",
      "100000\n",
      "Episode 6200\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "100000\n",
      "Episode 6300\tAverage Score in 100 episodes: 0.04\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 6400\tAverage Score in 100 episodes: 0.04\t Episode score: 0.0\n",
      "100000\n",
      "Episode 6500\tAverage Score in 100 episodes: 0.06\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 6600\tAverage Score in 100 episodes: 0.09\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 6700\tAverage Score in 100 episodes: 0.08\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 6800\tAverage Score in 100 episodes: 0.06\t Episode score: 0.0\n",
      "100000\n",
      "Episode 6900\tAverage Score in 100 episodes: 0.11\t Episode score: 0.09000000171363354\n",
      "100000\n",
      "Episode 7000\tAverage Score in 100 episodes: 0.10\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 7100\tAverage Score in 100 episodes: 0.11\t Episode score: 0.09000000171363354\n",
      "100000\n",
      "Episode 7200\tAverage Score in 100 episodes: 0.09\t Episode score: 0.09000000171363354\n",
      "100000\n",
      "Episode 7300\tAverage Score in 100 episodes: 0.09\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 7400\tAverage Score in 100 episodes: 0.17\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 7500\tAverage Score in 100 episodes: 0.24\t Episode score: 0.20000000298023224\n",
      "100000\n",
      "Episode 7600\tAverage Score in 100 episodes: 0.18\t Episode score: 0.19000000320374966\n",
      "100000\n",
      "Episode 7700\tAverage Score in 100 episodes: 0.17\t Episode score: 0.09000000171363354\n",
      "100000\n",
      "Episode 7800\tAverage Score in 100 episodes: 0.20\t Episode score: 0.20000000298023224\n",
      "100000\n",
      "Episode 7900\tAverage Score in 100 episodes: 0.16\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 8000\tAverage Score in 100 episodes: 0.12\t Episode score: 0.19000000320374966\n",
      "100000\n",
      "Episode 8100\tAverage Score in 100 episodes: 0.13\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 8200\tAverage Score in 100 episodes: 0.22\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 8300\tAverage Score in 100 episodes: 0.29\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 8400\tAverage Score in 100 episodes: 0.22\t Episode score: 0.10000000149011612\n",
      "100000\n",
      "Episode 8500\tAverage Score in 100 episodes: 0.75\t Episode score: 2.7000000402331352\n",
      "100000\n"
     ]
    }
   ],
   "source": [
    "state_size = states.shape[1]\n",
    "action_size = brain.vector_action_space_size\n",
    "agents_num=2\n",
    "scores, avg_scores =  maddpg_train(agents_num, action_size, state_size, env, \n",
    "                       brain_name,buffer_size = int(1e5),batch_size = 256, seed = 0, n_episodes=50000, \n",
    "                       max_t=4000, print_every=100, update_every = 4, tau = 1e-3, actorLr =1e-4,\n",
    "                       criticLr = 1e-3, weight_decay = 0)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd4VGXa+PHvPTNpkAIhoQYILk1KKFIVKaKAsiKuIqAi6K51FXV9VVwbq/jqru7ub1FfgV07grj2FVQERJBFqvQiCAihppCEQNrMPL8/pjBJJsmkTOr9uS4uZs555pxnDsPcc55yP2KMQSmllAKw1HQFlFJK1R4aFJRSSnlpUFBKKeWlQUEppZSXBgWllFJeGhSUUkp5aVBQqgEQkQtFJLWm66FqPw0KqlYRkUMiki8icUW2bxERIyKJPttmurcNKFJ2mog4RCTb/eegiLwpIp19yiS6X+spc1JEvhCRK/zUJ8enzJsiEumz/woR+VZEzohImruej4pIuJ/3NtznfGeLnD9bRFpX/gr6Z4zZbYyJK7ukaug0KKja6CAw2fNERHoCEb4FRESAKUA6MNXPMdYaYyKBGOByIAfYJCI9ipRr4i7XC/gG+EREphUpc7W7TF+gP/CEuw4TgA+BBUB7Y0wzYCKQALQtWiFjzEpjTKTP+fA8d/85VvplUSr4NCio2uhd4Baf51OBd4qUuRRoDdwPTBKRUH8HMsY4jDE/G2PuAb4DZpZQ7oQx5h/u/X8WkWL/N4wxR4EvgR7uoPQ34BljzD+NMenuMnuNMfcZY/YF/G59iEgzEZkvIidE5LCIPOE+FyJyr4gsFZFXRSRTRPaLyAif124UkSdFZL2IZLnvfGLc+3qISK5P2btE5Bf3Hc7PIvKbitRX1T8aFFRt9AMQ7W4Ht+L69T2/SJmpwH+ARe7nvw7guB/jCiZllWkOdCm6Q0TaAlcBP7r3JwAfBXDe8ngfSAE6AIOAG4AbffaPANYAscAc4J9FXn8jruvVGogH7it6AhGJB/4XGGGMiQKGAruq9F2oOkuDgqqtPHcLVwB7gKOeHSLSCJgALDDGFOBqwvHXhFTUMVxfpmWVoUi5T0UkA/ge193G/wKe9vkTPvV6X0QyROSciEwJoD6FiMivgIHAw8aYHHdz0svAJJ9iO40xC4wxDlx3T7/y7eMA5hpjDhpjsnEFrN5+TuUEBOguImHGmKPGmD3lra+qn2w1XQGlSvAusArXL+aiTUfXAnZgifv5e8AyEYk3xqSUcsw2uPogStPG/bdvufHGmGW+hUQkzf2wFa4+EIwxk9z7vges7sfZPi/rZow5XMq52wONgVR3ixG4frj5fmGf8Hl8zv13JJBdwn7fgIG7nmkiMhV4EHhXRL4D/mCM+bmUuqkGQu8UVK1kjPkF15ftVbiadHxNxfVld1hETgD/BkLw6ZwuwbXA6gDKnAL2llHOc/dSalt8kY7k0gICwBEgE2hqjGni/hNtjBlQxuvKzRjzuTHmMlxB8BjwSlWfQ9VNeqegarPf4vqCPCsins9qG2AkcCWwzafsA7iCxWzfA7j7JNoBfwCGA4P9nUhEWuBqknoauN8Y4yytYsYYIyIPAf8UkSxcTVgZQEegRTneo+8x94nIZuB/ReQ5XL/0OwLxxpg1FTmmP+6+kZ7At0AucBZwVNXxVd2mQUHVWiU0Z1wKbDHGLPXdKCKzgYd8hpwOdjfdCJAKrAT6G2N2Fzlehnt0z1lgIzDBGPNVgPVbJCKZwGPA34E84DAwD9fdS0VMBF7EdafSGPgZeLaCxyqJDXgcV6e2A9gE3FnF51B1lOgiO0oppTy0T0EppZSXBgWllFJeGhSUUkp5aVBQSinlVedGH8XFxZnExMSaroZSSgXV3jTXVJkuzYplXKmQTZs2pRpj4ssqV+eCQmJiIhs3bqzpaiilVFANf2s4ACunrayS44nIL4GU0+YjpZRSXhoUlFJKeWlQUEop5VXn+hT8KSgoIDk5mdzc3LILK9VAhYeHk5CQQEhISE1XRdVi9SIoJCcnExUVRWJiIj4ph5VSbsYY0tLSSE5OpkOHDjVdHVWL1Yvmo9zcXJo1a6YBQakSiAjNmjXTu+k66rWV+0icsZhL/7yCU2eC+29YL4ICoAFBqTLo/5G6689f/QTAkdM5zF5WoeW/A1Yvmo+UUqo++uFAGokzFhfaNn/dYeavO0yYzcLeWVdW+TnrzZ1CTXvuuefo3r07SUlJ9O7dm3Xr1tV0larNVVddRUZGRqWOsXLlSn79619XUY2Uqh/axTbyu717qyhWPzoiKOdssHcKp7JyuXfhj7xyYx+aR4VX6lhr167liy++YPPmzYSFhZGamkp+fn6ljmm327HZgv/PUxXnWbJkSdmFlFLl1rpJBIdOFN/ep13TSn9vlaTB3inMXr6PDYfSq6R97vjx48TFxREWFgZAXFwcrVu3BmDDhg1cfPHF9OrViwEDBnDmzBlyc3O59dZb6dmzJ3369OHbb78F4K233mLChAlcffXVjBo1CoAXX3yR/v37k5SUxNNPP+33/JGRkTz00EP07duXkSNHkpLiWrv+559/ZsyYMVx00UVceuml7NnjWv992rRp/OEPf2DEiBE8+uijhY7lcDh4+OGHveecO3cu4PolP3ToUK699lq6devGXXfdhdPpWrEyMTGR1NRUzp49y9ixY+nVqxc9evRg0aJFACxfvpw+ffrQs2dPbrvtNvLy8gD46quv6Nq1K0OGDOHjj88vw3z27Fluu+02+vfvT58+ffjss88q+S+kVP2Skp0XtGPXuzuFP/1nJ7uOZZW4f/2hdHwXm/O0z4nAgMRYv6/p1jqap6/uXuIxR40axTPPPEPnzp25/PLLmThxIsOGDSM/P5+JEyeyaNEi+vfvT1ZWFhEREfzjH/8AYPv27ezZs4dRo0bx00+ujqS1a9eybds2YmNjWbp0Kfv27WP9+vUYYxg3bhyrVq1i6NChhc5/9uxZ+vbty1//+leeeeYZ/vSnP/HKK69wxx13MGfOHDp16sS6deu45557WLFiBQA//fQTy5Ytw2q1FjrW66+/TkxMDBs2bCAvL49LLrnEG6DWr1/Prl27aN++PWPGjOHjjz/m+uuv9772q6++onXr1ixe7GoDzczMJDc3l2nTprF8+XI6d+7MLbfcwmuvvcZdd93F7bffzooVK+jYsSMTJ070Hue5557jsssu44033iAjI4MBAwZw+eWX07hx4xL/DZSqrw69MNbbr/Cr+MYsf2h4UM/X4O4Ueic0oVnjUCzugRgWgWaNQ+md0KTCx4yMjGTTpk3MmzeP+Ph4Jk6cyFtvvcXevXtp1aoV/fv3ByA6Ohqbzcb333/PlClTAOjatSvt27f3BoUrrriC2FhXcFq6dClLly6lT58+9O3blz179rBvX/E7G4vF4v1Svfnmm/n+++/Jzs7mv//9LxMmTKB3797ceeedHD9+3PuaCRMmFAsInnO+88479O7dm4EDB5KWluY954ABA7jggguwWq1MnjyZ77//vtBre/bsybJly3j00UdZvXo1MTEx7N27lw4dOtC5c2cApk6dyqpVq9izZw8dOnSgU6dOiAg333xzoTq88MIL9O7dm+HDh5Obm8vhw4cr9o+jVD1itQR/BFm9u1Mo7Re9x+OfbGfBelfvfb7DyZU9WjLr2p6VOq/VamX48OEMHz6cnj178vbbb9O3b1+/wwBLWxfb99ewMYbHHnuMO+8s35rqIoLT6aRJkyZs2bKlzPMUrdvLL7/M6NGjC21fuXJlsfdS9Hnnzp3ZtGkTS5Ys4bHHHmPUqFGMGzeu1HqWVIePPvqILl2qJmWwUoE6lZXLHe9uQgTmTrkoaO32nj7NmVd344+f7CCnwE7y6Rz+fddgurWKKfF1P53MZtfxzFLLVFaDu1MASM3O46aB7fnknku4aWD7SrfP7d27t9Av+C1bttC+fXu6du3KsWPH2LBhAwBnzpzBbrczdOhQ3nvvPcDVjHP48GG/X4CjR4/mjTfeIDs7G4CjR49y6tSpYuWcTicffvghAAsWLGDIkCFER0fToUMH/v3vfwOuL9qtW7eW+V5Gjx7Na6+9RkFBgbd+Z8+eBVzNRwcPHsTpdLJo0SKGDBlS6LXHjh2jUaNG3HzzzfzP//wPmzdvpmvXrhw6dIj9+/cD8O677zJs2DC6du3KwYMH+fnnnwFYuHBhoTq8/PLL3uD5448/lllvparC7OX72HIkgx8PZwR1PoCnT3P6wh/ZciSDvSeyOZvn4P6F/n/E+QqkTGXUuzuFQMyd0s/7eNb4HpU+XnZ2Nvfddx8ZGRnYbDY6duzIvHnzCA0NZdGiRdx3333k5OQQERHBsmXLuOeee7jrrrvo2bMnNpuNt956y9tJ7WvUqFHs3r2bwYMHA65mqvnz59O8efNC5Ro3bszOnTu56KKLiImJ8Xbwvvfee9x9993MmjWLgoICJk2aRK9evUp9L7/73e84dOgQffv2xRhDfHw8n376KQCDBw9mxowZbN++3dvp7Gv79u08/PDDWCwWQkJCeO211wgPD+fNN99kwoQJ2O12+vfvz1133UVYWBjz5s1j7NixxMXFMWTIEHbs2AHAk08+yQMPPEBSUhLGGBITE/niiy8q9o+jVAC6PPEleXZnoW3BmA9Q9Dz7U84W2r/vVLa3/yCxq/95Cr5lDr0wtkrq5UtKa8qojfr162eKLrKze/duLrzwwhqqUc2LjIz03k0Ey8qVK3nppZf0y7mOa+j/V0pyKiuXJz7bwTc7T+L5RrQKXNGtBc+M71FlzUinsnKZtWQ3X2w9hrOEr95WMeG8Pq0f93x9DXtPnCEs45liZcJDLHx8z8XlakYSkU3GmH5llWuQzUdKKeWreXQ48ZFh+H5POwzERYZVab9C8+hwosJsJQYEgMgwm/fLvkvLKL9l2jZtFLR+BQ0K9UCw7xIAhg8frncJql5Lzc6jSYQrrXj72AjaNo0IynyA1Ow8OreI9LuvcaiVzJyCMo8RSJmKapB9CkopVdTcKf144/uDPPPFLkZ0bcHMcWWPZKzoeZ78dAc/nSz+Y27nM2OKbfOdp9CvfVM+vPvioNTLI2h3CiLSVkS+FZHdIrJTRO73U2a4iGSKyBb3n6eCVR+llFJlC+adgh14yBizWUSigE0i8o0xZleRcquNMZoJTSmlaoGg3SkYY44bYza7H58BdgNtgnU+pZSq6xJnLKbLE1/WaB2qpaNZRBKBPoC/fNKDRWSriHwpIn4b8UTkDhHZKCIbPcneaqNPPvkEEfEmnlM15/HHH6dt27ZERhbu0MvLy2PixIl07NiRgQMHcujQIe++559/no4dO9KlSxe+/vrrKqnHnDlzeOeddyp9HE/SQVW/XdO7dakpsatjnaSgBwURiQQ+Ah4wxhTNVLcZaG+M6QW8DHzq7xjGmHnGmH7GmH7x8fHBrXAlLFy4kCFDhvD+++9XyfEcDkeVHKc62O32Gju3v+t09dVXs379+mLbX3/9dZo2bcr+/ft58MEHvVlid+3axfvvv8/OnTv56quvuOeee6rk+t91113ccsstlT6OahiiwmxBS60RqKAGBREJwRUQ3jPGfFx0vzEmyxiT7X68BAgRkbhg1ilYsrOzWbNmDa+//nqhoDBx4sRC6w1MmzaNjz76qNQU1SNGjODGG2+kZ09XPqbx48dz0UUX0b17d+bNm+c91uuvv07nzp0ZPnw4t99+O/feey8AKSkpXHfddfTv35/+/fuzZs2aYvXduXMnAwYMoHfv3iQlJXnTdLzzzjskJSXRq1cvb9K+X375hZEjR5KUlMTIkSO9yemKpuAOJOW1MYaHH36YHj160LNnT+/s66q4Tr4GDRpEq1atim3/7LPPmDp1KgDXX389y5cvxxjDZ599xqRJkwgLC6NDhw507NjRb1BZunQpgwcPpm/fvkyYMME7HDgxMZFHH32UAQMGMGDAAG9aj5kzZ/LSSy8BMHv2bLp160ZSUhKTJk0CID09nfHjx5OUlMSgQYPYtm0bAGlpaYwaNYo+ffpw5513FsqXNX/+fO+/3Z133onD4cDhcDBt2jTvdf373/9erO6qbDU9lbesIbDVMdc4aB3N4sp29jqw2xjztxLKtAROGmOMiAzAFaTSKnXiL2fAie2VOkQxLXvClS+UWuTTTz9lzJgxdO7cmdjYWDZv3kzfvn2ZNGkSixYt4qqrriI/P5/ly5fz2muvlZmieseOHXTo0AGAN954g9jYWHJycujfvz/XXXcdeXl5PPvss2zevJmoqCguu+wybwqL+++/nwcffJAhQ4Zw+PBhRo8eze7duwvVd86cOdx///3cdNNN5Ofn43A42LlzJ8899xxr1qwhLi6O9PR0AO69915uueUWpk6dyhtvvMH06dO9qS98U3D/8Y9/LDPl9ccff8yWLVvYunUrqamp9O/fn6FDh1bJdQrE0aNHadu2LQA2m42YmBjS0tI4evQogwYN8pZLSEjg6NGjhV6bmprKrFmzWLZsGY0bN+bPf/4zf/vb33jqKdeguejoaNavX88777zDAw88UGxexwsvvMDBgwcJCwvzrlT39NNP06dPHz799FNWrFjBLbfcwpYtW/jTn/7EkCFDeOqpp1i8eLH3x8Du3btZtGgRa9asISQkhHvuuYf33nuP7t27c/ToUW+qkMquhNfQ1dRy1r4peGpKMEcfXQJMAbaLiCeD0x+BdgDGmDnA9cDdImIHcoBJpq7l3XBbuHAhDzzwAACTJk1i4cKF9O3blyuvvJLp06eTl5fHV199xdChQ4mIiGDp0qVs27bNm8guMzOTffv2ERoayoABAwp90c2ePZtPPvkEgCNHjrBv3z5OnDjBsGHDvGm2J0yY4E2/vWzZMnbtOj/IKysrizNnzhAVdX525ODBg3nuuedITk7mN7/5DZ06dWLFihVcf/31xMW5btY8x167dq13EZwpU6bwyCOPeI/jm4J76dKlfP75595fxp6U175pFb7//nsmT56M1WqlRYsWDBs2jA0bNlTJdQqEv4+XiJS43dcPP/zArl27uOSSSwDIz8/35qUCmDx5svfvBx98sNjxkpKSuOmmmxg/fjzjx4/3Xo+PPvoIgMsuu4y0tDQyMzNZtWqV95qPHTuWpk2bAq4FizZt2uRNx56Tk0Pz5s25+uqrOXDgAPfddx9jx471Bk5VvxQ4nGUXqqSgBQVjzPdAqfHWGPMK8EqVnriMX/TBkJaWxooVK9ixYwcigsPhQET4y1/+Qnh4OMOHD+frr79m0aJF3i+O0lJU+/6yXrlyJcuWLWPt2rU0atTIu75AabHT6XSydu1aIiIiSixz4403MnDgQBYvXszo0aP517/+hTGmxHTWvnzLFE31XVbK65LqXdnrFKiEhASOHDlCQkICdrudzMxMYmNjvds9kpOTvavn+db9iiuuKJTR1ZfvdfF3HRcvXsyqVav4/PPPefbZZ9m5c2epwaiktOtTp07l+eefL7Zv69atfP3117z66qt88MEHvPHGGyVcBVVXHcvICfo5NM1FFfjwww+55ZZb+OWXXzh06BBHjhyhQ4cO3kVoJk2axJtvvsnq1au9X26lpaj2lZmZSdOmTWnUqBF79uzhhx9+AFwL3nz33XecPn0au93u/bUJruyqr7xyPtb6W1PhwIEDXHDBBUyfPp1x48axbds2Ro4cyQcffEBamqsFz9N8dPHFF3v7Sd57771iKbM9Akl5PXToUBYtWoTD4SAlJYVVq1YxYMCASl+nQI0bN463334bcP27XXbZZYgI48aN4/333ycvL4+DBw+yb98+b708Bg0axJo1a7z9BefOnfPenQHe/pFFixYVuoMAV6A+cuQII0aM4C9/+QsZGRlkZ2cXSqO+cuVK4uLiiI6OLrT9yy+/5PTp0wCMHDmSDz/80JtCPT09nV9++YXU1FScTifXXXedt1lR1X3rD6YXypKakp0f9GGrmuaiCixcuJAZM2YU2nbdddexYMECLr30UkaNGsUtt9zCuHHjCA0NBUpPUe1rzJgxzJkzh6SkJLp06eJt927Tpg1//OMfGThwIK1bt6Zbt27ExLgSZM2ePZvf//73JCUleddvmDNnTqHjLlq0iPnz5xMSEkLLli156qmniI2N5fHHH2fYsGFYrVb69OnDW2+9xezZs7ntttt48cUXiY+P58033/R7HQJJeX3ttdeydu1aevXq5b2batmyJUClrlNRjzzyCAsWLODcuXMkJCTwu9/9jpkzZ/Lb3/6WKVOm0LFjR2JjY73Brnv37txwww1069YNm83Gq6++Wmxluvj4eN566y0mT57sXWd61qxZ3lXl8vLyGDhwIE6ns9jdhMPh4OabbyYzMxNjDA8++CBNmjRh5syZ3HrrrSQlJdGoUSNvwHr66aeZPHkyffv2ZdiwYbRr1w6Abt26MWvWLEaNGoXT6SQkJIRXX32ViIgIbr31Vu+62f7uJFTgaksjdp92TbiocWs+33IMcK0UeXWv1jw+NniZbjV1dh2WnZ1NZGQkdruda6+9lttuu63YGgeqeiQmJrJx40Zvf0xt1VD/rwTKk/to2sWJQct9BPDkpzt494dfim33XR9h+FvDAbgk5mXeW3d+OdqbB7ar0EqRmjq7AZg5cya9e/emR48edOjQwdt5qZSqP1J9hqnGR4YGJXOrL20+qsM8o3xUzfOdGa1UVZo7pZ+3X6FDXGTQh63WmzuFutYMplR10/8jKhD1IiiEh4eTlpamH3qlSmCMIS0tjfDwmk2hUNuV5xvkVFYuN8xdy6kzuUGrT1GmGuZc14vmo4SEBJKTk6nNyfKUqmnh4eEkJCTUdDXqhEBmNM9evo8Nh9KZvWxfhTp+a6t6ERRCQkLKPbNVKaUqossTX5JnPz+zeP66w8xfd5gwm4W9s66swZpVjXrRfKSUUtVl9SMjGNe7NVaL63YixCplpryuKlJ6kogqoUFBKaXKoXl0OFFhNhxOV/u+3WFqRcrrqqJBQSmlyik1O49OzV0LOPVPjA363IHqpEFBKaXKae6Ufgzo4MoifHXv1rUi5XVV0aCglFLKS4OCUkrVFfVhjWallKprGvI8WA0KSinlVkOrcNYqGhSUUsqtQjcI1XlbUQ2n0qCglFJFBJLmIpAydZEGBaWUqiu0o1kpparHqaxc3v7voZquRo3ToKCUUriynh5JP1fT1ahx9SJLqlJKVVTRrKcAb645xIJ1h+tF1tPy0jsFpVSD5sl66ttcf0F842rJelobaVBQSjVonqynvqM9Q6xSK7OeVseAJw0KSqkGLzU7jxDL+a/cnHxHwK+tb5OfNSgopRq8uVP6ER5i9T4feWGLGqxNzQpaUBCRtiLyrYjsFpGdInK/nzIiIrNFZL+IbBORvsGqj1JKBepcvj3gstU5h6067kqCOfrIDjxkjNksIlHAJhH5xhizy6fMlUAn95+BwGvuv5VSqsZsPZJZ01WoMUG7UzDGHDfGbHY/PgPsBtoUKXYN8I5x+QFoIiKtglUnpZTyp8sTX3Im7/zdwZ4TZ0icsZguT3xZg7Uqrt50NItIItAHWFdkVxvgiM/zZIoHDkTkDhHZKCIbU1JSglVNpVQDtfqREdh8OpqtFuGa3q0b5LDUoAcFEYkEPgIeMMZkFd3t5yXFms2MMfOMMf2MMf3i4+ODUU2lVAPWPDq80JeRw2mICrPVymGpwRbUoCAiIbgCwnvGmI/9FEkG2vo8TwCOBbNOSinlj++v0S4tI0nJzquxutSkYI4+EuB1YLcx5m8lFPscuMU9CmkQkGmMOR6sOimlVEkiQs8PSb34V3HMndKvBmtTc4I5+ugSYAqwXUS2uLf9EWgHYIyZAywBrgL2A+eAW4NYH6WUqnLVOXmtOtZwCFpQMMZ8Txmd5cYYA/w+WHVQSqlgkXq6eKfOaFZKKeWlQUEppeqI6lgOWoOCUkoVUV+bhgKhQUEppeqI6uho1qCglFLKS4OCUkpVs+r4xV9RukazUkpVQkU6fwN5TYHDyb6T2Zw6k1v+E1SC3ikopVQFBPvXfvLpHLLyCpi9bF9wT1SE3ikopVQFVGZ46IJ1h/1u96TqzrM7ORnqukOY71O2OkZF6Z2CUkoVYcqRvKIidwzj+7T2u331oyNY/cgIxvU+vz88pHq/pvVOQSmlqlnjMP9fvZ5U3VE++/Pszmqpk4feKSilVDUrq+kp1Sdt900D2p1/XTWk39OgoJRSFM7eWdMzmn3Tdj87vke1nluDglJK1REbDqaz63hmUM+hQUEppeoIh4H7F24pu2AlaFBQSqlKqMjQ1PKMWOrw2JJCz/edyiZxxmISZywu/4kDoEFBKaUqoDKT18oTSBbfd0mxbQlNIlhy/5CKV6AUGhSUUqoW69Y6pti2iFAr3VoV314VdJ6CUkrVYr53FZ2bR4JAxrmCoJ1Pg4JSStURcVFhLLh9UFDPoc1HSilVzWpz6mwNCkopVc2qY63litKgoJRSRdTmX/LBpkFBKaWKKM8veVObf/ZXgAYFpZSqgPp6M6FBQSmlqlltbp7SoKCUUhVQmUaj2tzipEFBKaUqQYL8s983flTHHUbQgoKIvCEip0RkRwn7h4tIpohscf95Klh1UUopFZhgzmh+C3gFeKeUMquNMb8OYh2UUiogwf7FX1cE7U7BGLMKSA/W8ZVSqq6qzfEn4KAgIkNE5Fb343gR6VAF5x8sIltF5EsR6V7Kue8QkY0isjElJaUKTquUUjWnznc0i8jTwKPAY+5NIcD8Sp57M9DeGNMLeBn4tKSCxph5xph+xph+8fHxlTytUkqVrjy/5Bvq5LVrgXHAWQBjzDEgqjInNsZkGWOy3Y+XACEiEleZYyqlVHWpxS1AlRJoUMg3rnBoAESkcWVPLCItxd2zIyID3HVJq+xxlVJKVVygo48+EJG5QBMRuR24DfhnaS8QkYXAcCBORJKBp3E1O2GMmQNcD9wtInYgB5hk6tt9mFKqzqjOr5/yNE/tPpYZvIr4EVBQMMa8JCJXAFlAF+ApY8w3Zbxmchn7X8E1ZFUppRqU8sSfBz7YEryK+FFmUBARK/C1MeZyoNRAoJRSdVWNPjeEAAAgAElEQVRtmKeQOGPx+Sehrr/2nzrr3STV0JNRZp+CMcYBnBOR4KwSrZRStUxNNWRf07s18387gDZNImqmAgTep5ALbBeRb3CPQAIwxkwPSq2UUqoBigqzMaRTPI1CrXCuZuoQaFBY7P6jlFLKR0VuKkpqqUrJzgMgM6fAu61T80j2ncquwFkqJqAhqcaYt4GFwCb3nwXubUop1SAFow9i7pR+AKx//HLvti/vv7TKz1OagO4URGQ48DZwCNecjbYiMtWd30gppVQ51ObB94E2H/0VGGWM2QsgIp1x3TlcFKyKKaVUTakFA5FqTKAzmkM8AQHAGPMT7oloSinVEFXXZLfqvqkI9E5ho4i8Drzrfn4Trr4FpZSqFyp6d1CRl9XmO5FAg8LdwO+B6biuwSrg/4JVKaWUUjUj0KBgA/5hjPkbeGc5hwWtVkopVY9VtOWpNq3RvBzwnWIXASyr+uoopZSqSYEGhXDP2gcA7seNglMlpZSqO2rx6NIKCTQonBWRvp4nItIPV7prpZSqdxZvO8apM7mllqnM5LXa3NEcaFB4APi3iKwWkVXA+8C9wauWUkrVnBNZecxetq+mq1EjSg0KItJfRFoaYzYAXYFFgB34CjhYDfVTSqlqkXGuoNDz+esOkzhjMV2e+LLKz1WbZzSXdacwF8h3Px4M/BF4FTgNzAtivZRSqlpFRxQejBkeYuGa3q1Z/eiIGqqRS3UHkLKGpFqNMenuxxOBecaYj4CPRKR6lwNSSqkgshRp6M+zO4kKs9E8KryGalQzyrpTsIqIJ3CMBFb47At0joNSStU5Nw1s701lXdVqc0dzWV/sC4HvRCQV12ij1QAi0hGo3tWklVKqGs0a36Omq1AjSg0KxpjnRGQ50ApYas5ngLIA9wW7ckopVdtVpM2/pNecOpNb481VgazR/IMx5hNjjO8ynD8ZYzYHt2pKKdWw1IZhsNovoJRStcT8dYeZv+4wYTZL4LPIqpgGBaWUqkZdnviSPLvT777wEAuju7fk8bEX0u7Faq6YWw3FIqWUql2qa0DQ6kdG0CHOf+o4f8NgTTVnV9KgoJRS1ah5dDg2i/+v3mAOgw2UNh8ppVQlVGTOQW6Bw+/22jAMVu8UlFKqmg3v0rymq1CioAUFEXlDRE6JyI4S9ouIzBaR/SKyzTc1t1JK1SWnsnK5Ye7aMtNte9TmGc3BvFN4CxhTyv4rgU7uP3cArwWxLkopFRTGwOzl+9hwKL1WzDOorKD1KRhjVolIYilFrgHecc+S/kFEmohIK2PM8WDVSSmlqtozX+zyPvadZ7B31pUlvqakGc2JMxaX+trKLOwTqJrsU2gDHPF5nuzeVoyI3CEiG0VkY0pKSrVUTimlAtGjdbT3cWXTbdeGVN01GRT8hTy/8dMYM88Y088Y0y8+Pj7I1VJKqcCF2azex5VNt10bUnXXZFBIBtr6PE8AjtVQXZRSqkLO5tu9jwOdZ1BSK5C/11b3Ijs1GRQ+B25xj0IaBGRqf4JSqq65od/537azxvdg7pR+FT5WZV5bVYLW0SwiC4HhQJyIJANPAyEAxpg5wBLgKmA/cA64NVh1UUqpsjiL/CQvK431OfcdwonM88NQA019XdFf//kl5EyqSsEcfTS5jP0G+H2wzq+UUuWRk194lvHsZfuYdW3PEstvPZIBwMc/Jgf8msr6Je1s2YUqSdNcKKUatJKylpY0vLRo+dTs/DJfUxldn/zK+/h4Zm6Zw1YrS9NcKKUatNWPjCg0rNQjzOZ/eOnqR0Ywrndrv8eyCozp3qLMYaXlmW7w66RW3scWCf6wVQ0KSqkGrXl0OKG24l+F+Q7/w0ubR4cTFea/kcVhIC4yrEqHlUb6nMtpgj9sVYOCUqrBO5tnL7attOGlqSVsb9s0IqAhqeXpaPY9V+uY8KCn1tagoJRq8Cb0a1tsW2nDS0vavvrRy6p8WOkrN57PFdqpRVTQh61qUFBKKT8CzXha32hQUEopP4KZ8TSnoHhzVW2hQ1KVUg1a4ozFfrcHY3ipx7bkzCo9XlXSoKCUatCWTB/CDXPXkp1XePJamM3CmB4teXzshVV2rpLmRNQm2nyklGrQurWOweJn4kBJQ1IrwzPHwWqpvUuvaVBQSjV4+Y7iv94DzXhaHp45Dg5nNac+LQcNCkqpBu/aPsXX96psxtOSpGbn0aVFZLlfN9X6NZ3y/C55X6U0KCilVDWaO6UfAy9oVq7XCE6etL1LUs6GINXqPA0KSikVoFNZudwwd22JcxgCndtQ2oxmzzk8dh3LpCnZ2MRJprVpuepbERoUlFLK7+rAxc1evo8Nh9JLnMNQFXMbPOfweOTDbcSLK013VjUEBR2SqpRSZSg6lHT+usN+y1V2boO/ORP7U84yxOKa17DyqIUXZyzm0Atjy33sQOmdglJKlaFouuzwEP9fneEh/tNtF1VS6uyIEKvf7XG4gkKmtSlL7h8SQI0rToOCUqrBK2t9g6LpskuagJZnr9zchhCr/4p4mo9OW5rQrVVMhY4dKA0KSqkGL5BU1r4prG8a2N5vmUDnNpR0vtwSgk28ZJJjQsl0hJVd0UrSoKCUUgHwnbMwa3wPv2UqO7fhpxL6IeIkkxQTw8Ud4yt87EBpUFBKNShlDSv1liuy/1RW2cNNx85eTfenv+L7/Sl+z3EqK5fxr65h8bZj5apzPBmkEtxmIw8NCkqpBsXfsFJ/fQpFh5fOXl72cNOdx7I4m+fgnvmb/Q5dnb18H1uOZJB+rqBcdY6XTFJMk3K9pqLElGdduFqgX79+ZuPGjTVdDaVUHVNShtIwm4UCh5PakI4ozGbx1vFE6AwAWua/wKawO/nSMYAn7L+t8HBXEdlkjCmzbUvvFJRSDYJnWKknQWmIVbzDR8f3Lp77yCowpnsLlkwfUmg4KkCbJhEBnTPM5hqiumT6EEZ1bxFQHYuyYacp2aSYJsRFhpY53LWyNCgopRoEz7BSzx2B3WG8w0cjQovPD3AYiIsMo1vrmELDUQEa+Snvjyf9drfWMcRHlj1yqHl08aGssZzBIoZUYgi1Wqo0lbc/GhSUUg1GanYeCU1dv/KHdo7zDh/113LUtmmEd39qkWGmmTmB9Qn4DlEteoxAeeYopJgYvym+q5oGBaVUgzF3Sj+6tYoGYPKA9qUOH1396GXe/UXLrX/88jLPFRVmKzREtaJDVePFNZs5xTShZ5vgj0AKalAQkTEisldE9ovIDD/7p4lIiohscf/5XTDro5RSvjzDU3PzHWUX9tH9qa8qdK6y7DpWfO1m750CMdWylGfQEuKJiBV4FbgCSAY2iMjnxphdRYouMsbcG6x6KKWUL9+mIs/w1NPxJS96cyorl3sX/lho29lAgkiRYa6BDGm9//0txbbFu/MepZoYnGlnyz5vJQVtSKqIDAZmGmNGu58/BmCMed6nzDSgX3mCQlSHKHPR0xdVcW2VUg3F3pNnOH02P6CyFhHio8I4eSbXf8dDAK8HcFbgezbfcgCAzqYx8ZLBBmeXQscd0CG2XMf77tbvanxIahvgiM/zZPe2oq4TkW0i8qGItPV3IBG5Q0Q2isjGgoLyTfpQSilfnh/wZY0gEnF9mZ/MqlhAEBH6tGtCn3ZNiIsMKzPpXklCxE6Bu1FHxDUiqk+74E1kC+Z6Cv4uQdFL+x9goTEmT0TuAt4GLiv2ImPmAfPANXlt5bSVVVxVpVRDccc7G1m66yRdYiPZeyIbwf93/m/6tMFuDIu3HcdRgZltIVZhzW+vAuDxT7bzXglrMJTEM3ntnzRGMEzMfwqAm3u3Y9a1PctdH7k1sKgUzDuFZMD3l38CUCjhhzEmzRjjGaf1T0DbhZRS1SI339Vp2711NI393DWczbcTFWarUECAwplQU7PzaNoopMzXhNuKfyXHk0GKcY06ah0THlAW1soIZlDYAHQSkQ4iEgpMAj73LSAirXyejgN2B7E+SinlNfJC1wzjRqE2ru1bvGV77pR+pGbn8au4xhU6vm/z1Nwp/WjTtOxZ0D38DDmNk0xS3UGha6voSmVhDUTQgoIxxg7cC3yN68v+A2PMThF5RkTGuYtNF5GdIrIVmA5MC1Z9lFIK/DcVSQlrNM+d0o++7YO/LrK3HkWqYcEQI+eqLRkeBHmNZmPMEmBJkW1P+Tx+DHgsmHVQSjUsniGkr9zYx29KiCz3bOTdx11DPU9knmPz4fRi5W6Yu5Ypg9rx703JFapHVq6dXcczvSulFQQwG3nH0cLzFEKwA645CtVFZzQrpeqVf/hJje1ru/uLd+0BVyA4fDoXf3PCNhxK5w8fbK1UXe5feH7ewamssvsCcgoKVyRE3EGhGu8UNHW2UqpeKC019t5ZV5I4Y3EN1KriToTOoKlkc1zSuTpvFtvNBd59h14YW+7jaepspVSD4kmN7REeYvGmxgZYMn1IwCmvq1Kr6HAG/6p8E808vM1H7o7mcJuFJfcPqbK6+aNBQSlVL3hSY3vk2Z3e1NgA3VrHBJzyuipFhtu4IK7kNBqlCXUHhTR3n4LFIt4+imDRoKCUqjd801P7pq32CDTldVXp3CKSzJyCCqfNDsHOaRPpndFsr4bU2UEdfaSUUtVp7pR+3r6DWeN7FNu//vHLA+5baNIohIxyrqVc1NIHh3kfV6RPI0TspDjjvc+HdIovpXTV0DsFpVSt4UllfepM2Wmmyyp/w9y17DqWydjZq7nwqS8Z+ddv6f504CmvKxsQqkII9kIjj87kBr9OGhSUUrXG7DKGk5an/IZD6dz//hZ2HssiJ9/JzynnOJtXvnUTqkogayn4E4KdVJ85CruOZVVVlUqkQ1KVUjWurOGkgZav65pzmlM0AYQToTPob9nLVPtvmGWfUqxseYel6pBUpVSdsfqREVzRrYX3edHhpP7K+xt+umR6cIdrlpelHOmyL7bs4Iewe3nA9hEAVpxYcRabuBbbOCSow1I1KCilalzz6PBCmUqLDif1V97f8NNurasvHUQgxvduwyifYFeScPJ43vYvLGL4vfUzLpRffGYzF35PzRqHBXVYqgYFpVStkH7OtRpaXGSo3+GkRZU1/LRljP+AUp3O5tuJDCt7kOcDto9pbznFnfkPkEFjXgyZ652jkFok71Gwh9VqUFBK1QrP/yYJgBCrhVnje5SZItp3v7/yY7q3rFR9fiVHaScnK3WMuVP6kV7G0p/d5RC/sy7mfftwvnYO4MmC2+hhOUQ7OQUUznt0+YXNWf/45ZWqU1ka3DwF3wyKGErNpqgajrIya9aXc9ZmP508A8DxzFzvmP5wm4X2cY0AOJx+jnax5x9H2M43NyXOWExibARW6/lG/Lf+e6jCdUmQFD4JfRoHFsbnP8MvpmIB5rXv9vHDwdQS91tx8HzIPzlNFP9rvxGAr5wDWOwYQKT1W6BwUFh3II1TZ3KD+nlpcHcKvkPYyjv8TdVfNfFZ0M9fYc98vrPYtly7k70nstl7Ipuc/MKP04vMIziUnsPPKecqXQ8bdv4R8gqCwQBvhLxINNkVOtafv/yJ3IKSR3hOs35NkuUgMwumksX5VBhPF9yKHSsGSCfKu/1MniPon5cGMyQ1kCFsJQ1/U/VXeYdC1tVz1ma1LXvpQ7YPuM/2KdPz7+W4ieW90OfY4OzK1IJHsftpXLFhZ7x1DWsd3ThK4DOO48lgRdhDbHB24baChym6rH1u2H1EcY6CvNf9vr68nxcdklqEZwibZ4iY71Cxsoa/qfrL87nwrHgVbgv+Z6GsbJ4NzZLpQ4gMq/5Edf4Mtuzk99bP+MA+jM+dF7PBdGVGwe1cYt3Js7Y3EQoH8+5ykM9Dn+SlkLl8HvYE/WRPwOf6H9sHhJHPM/YpFA0IABmmMUeM/yAzvEt80D4vDaZPwTOEzbMGt+9a3GUNf1P1l+dz4blhznME/7NQVjbPhqZb6xhaxUSw71TFmmiqSlOy+H8hr3LQtORp+1Tv9o+dQ+lgP8F9tk+53LqZVc4kvnP0oovlMHdavyCdaB4r+C2/sy5hQehzPFZwOx85h5Z6rh5ygAnW7/in4yoOmVallvUnoUlE0D4vDSYogGsIW+MwK2fzHLSKCed4pmvq+U0D25MSYK4VVf+kZucRahXyHYYJFyWUORSyqs7poZ+/6s9e6s8fbQtoyhluLXiEHAp/4f7Nfj0/ORO43LqZyyw/cp11NQAf2Icxy34TWUSy2DGQ/wv5B38NnUNH+1H+Yp+I8dsYY3g65B3SiOIV+7Xlrmd4iCWon9EG06fgMfQv33I4/RzfPTycYS+uBCq2ipGqX5Jmfk1Wrp2tT40iplFItZzT05aunz+Xmuxb6Cs/8XHYTF6zX82f7ZNLLWvBSS/5GTvWQquhgat/YabtbW62LecLx0AeKribPEILlbna8l9eDn2FRwtuZ5Gj5CagE6EzAGiZ/4J3W2X6nQLtU2hQdwq+6lgsVEGmH4eGy4qDZ0Pe5JiJ5eUAfrk7sfCj6eR3nx0bT9hv4xfTgsdDFtBSTnN7/h84TTRguECOMyNkITucifzbMczvMUpzRQCzoyurQQWFXccyOZLuGrI2/KWV3u2X/3Ul91/eift8FtkOtUCBE2IbhZDmHvoWbrPQpFEIJ3wW4BaBd387gCEdz3cIfb8vhZtfXw/AY1d25s5h/j9AVeU/W49y38ItvHJjbwYkNmPSvLUcSD3HKzf25tdJbUp97a5jmUyc+wOvTenLU5/u4ECq6/p0jG+MwXiH+Pl7n6Ud8zf/t4Y8u6F9bATHMnPI90lO2ThUOJtfO7+Gez2ztNrPWdtG3zQ0N1mX0d3yC3fn3885qqKdXvin49ccNXH8PeQ1PgqdySpnEiMsW2hvOUWBsXJ/we9xVmCcz5fbj7PreGZQ01w0qOajy/+2kv2nzvrdF2IVChwVuxbR4Ta2zRztfe5pivAIdvNAp8eXUOAwhFiF6/om8P6GI4DrPe177qpSX+u5JtHhtkJ19qfo+yzxmH9dyf4U/9dZqarWVk5yh3UxOYRx0jThlGmKAwst5DQtJIOmnGGz6cSXjv6F5gIANCOTb8MeYqvzAqYUPIa/UUCV0Vd+4p+hf6UReaxxdmelszcrHH04RlyZr/XXfATQqXkk3/yh/HcZgTYfNYigUFt+iVV1cCjP+yp67speE3/vpbZcZ1X3xXOah20fcJpI/uMYzA7TAX9f2L1lP/8KfYlIcgAIl8Id1nnGRjYRNJMz5BsrK529+d7ZAwuGMAoYYtnOQMtursx/gZ9N6XfVFRWCHQvOYn0LZSkpKPgqz3eK9in4WDJ9CLe/s4mjGTlBO0eoTXjwik689OVP+FvG47ErO1f5OV+e3JsH3t9CaTc4FoHZk3sX217RaxJqFd64tb/ffUumD+Hmf633JjZTqiL6yR7+L3Q20ZzFgpM7bYs56GzBYucgvnP04kfTETs2xljW8/9CXuWkacoNBU9xwLQimnO0kNPYcHDCNOW0ezZwDznINdb/crV1LaOsm7znchrhL/aJQQsIgHd95arUPCqMt27z//+wshpEUOjWOoZGocGdHBNus3L3sE689u0Bv80wwehXuLpXG/7wwVYcpUQFq0X89itU9JqEh1hL7Ffo1jqGZpGhDSooCE4GWXaTYmLYbxJqujoAhJFPHiFUdVNI8BmmWb/mcdt7JJs4phTM4ISJZbR1A1db1nK39XPutX3GGRPBNucFDLbsYrPpxB35fyCdaACyaEyWaVzsyDvMBeywX8Dz9htpRib5hJDn/uN/2GjtFhMRErR+hQYRFCD446Bz3WkLcqt5NSi7zyw8ofAoGimyv6iKXJOy3l9tGG9eGdFkkygnSZST2LBzjDiSTTwnTNNCKQ48qQ3usv6HjpZjAGxzduBjx6Usc/alMbm0kNM0lwyac5qWcpoWcppYySLDRHLKNOWkacpxYkk28SSbOE6YZkSQ6y0bxTmyiSDLNOYMEQBEkEcE+YSInUzTmHQTTTpRtJNTXGHZxBXWTfSx7OeYiWWd80LWOS/kkGlJHJm0kHTiJYt9zjZ84+xbrH3dVyTnGGDZw2HTnAOmtd9O0WiyGWddy3XW1XSWI+w27dnu7MA25wXsM21INvFkEElpwSmMfC6y/MRQy3aGWbZyoeUw3zj68lDB3WTh+nL/wDGCDxwjiOYsgy27uNSyjUGW3XzoGMqT9lvL1SzjxEIKTQMuXxtZJbj/zxpEn4JHsNq81z52Ga1iIvye55N7LqZPu+B+CD3n2/PsGLo+eX5h8kDaG8tzTQIdw1+R6xxNNr0sB2gvJ+kgJ2gnJ2ki2TQmjwhyCZcCHFhwGAt2rGQTQaqJIcXEkEY0eSbUtR8LBdjIIYxzJoxcQomSczQji2aSRbxk0JwM7xd2KAXYsWLHigDR4j+hmsMIp4kizUSTbqJpZzlJG0ljt7Mdc+2/JlbO8BvranpYDvl9fYZpzAkTy2kTRRPJprmcppmcKfd1KstW5wWscfagnZxkoGU38VJ4TV+7sWATJwXGyhpnD5Y5+3LAtOKoieO4aUaS/Mwk20qusqyjkbhG2WWbcHaaRJJNPBac2HAQRQ6DLTsJEzu7nW3Z6OxCF8sResgh7+s8rz1lmhAqdhqRSyPysPqkirDixCKGfGNls+nMF45BvOcYWSO/3j3/X+5b+CP/2XqMf0zqzTW9i99llzS/xPdzf2WPlrx280WVqk/4Uz0BeGn4x7y09Ce/5yyPWtGnICJjgH8AVuBfxpgXiuwPA94BLgLSgInGmEPBqMv3+1KCcVgABj+/osR91/7ff4N23qJ8AwJUfRCsyuGajcilixxhiGU7w6zb6CP7sIrrB8o5E8YvpjnpJpoMojhLGHnOUNcXkjgIwUEkObSQ0/SwHCSWM4RI2QuynzNhpJloTtCU3aYdq5xJ5BKKDQc2HAiGoyaOX0wLDpmWFGCjjaTSWlJpI6nEkUWsuILLfmcbHnfcxkpnbzy/hN9wXEkXOUx/y15OmyhOmiacpCmnTFO/v2ZDKaCFpJMgqSRICq1JI5twTpmmnDCxZNGISHKIknNEcw6DuIIdYTiMlRjJJlbOEEsWp4lihaMPJ4n1OYNrXHwrSeOkcdXjDBEkyQGutK7nKss6ZoVsLVavMyaCTx2XsNg5kBacpqflIL0sPzPQshu7seLAQj42FjhG8qFjGDtNe+81sODkV3KMDnLc+77iJYM8E8o5E0YOoYXa2B1Y2eL8FeucF1bRcNCK+35/CkM6xpNb4PosZZXxa7y0FNZVmeI65cz5IBvs4agQxDsFEbECPwFXAMnABmCyMWaXT5l7gCRjzF0iMgm41hgzsbTjVvROoegw0TGW9cRJ5vn6Yogklyg5RxTnCMWOA8Hh/hVpx0oBNgqwYYBw8mlEHhHkcZxYNjs78aOzE5lE0ohcOksyXS2HseJkt7MdP5kEsmlU7nqXRxj5xHKGMMknBAch2HFg4ZRp4u5wK72NOZqz3l/qCXKKCPdxbNjJIYw9znbsNu04ZFqWOcY6nDzaSCoJkkpzOU0LXE0i7eQUHS1HaSNpgKujb5vpwHfOXvzg7MbPztbehcvLQ9zr2VpxEkYB4eQTIa6mlmwiSDNRxVIXKOP94k6QFNqQSrKJZ4lzQIO8Vp4h18Nf/JZDaee4pGMz3vvdoGLlPD+2bh7YjlnX9iy23aPo/vLy3Cl0D/kbae6Feio6HBVqwZBUERkMzDTGjHY/fwzAGPO8T5mv3WXWiogNOAHEm1IqVd6gUNKv5U9Dn6S35edi2/ONlTM0Ip8Q160tTkKwY3X/HeZeNzXXhHCOMPIIJZ4MbOK6JT5pmhBPJhYp/haSTRznTFjAdQ9UBPnEShaNpeR8KHnGRgpNCp3fipMwKaARuUSQT4QU7iB2GnEHQivh5HvfY44JJZPG2LGSb2w43HnfwfVV3lTOEFek2QIg3URy1MSx37Rhn7MNP5s2bHB28XYSKlUbeVJLBJJ+v7TXB8rznVXWkNTyNiXVhuajNsARn+fJwMCSyhhj7CKSCTQDCi1VJCJ3AHcAtGvXrlyVmP/bAd7Zxb6m5T+Czadt0wDZRAQwasO4F984/0s5glx6WQ7QV/bxK8sxDjlbsMe4flUbI3SxHKGrHKaT5SihVH0HUR6hpDujvJ2OOSYUOzbsWLHioLlkuCfynCaM81/8BiHXuNrezxFGuonikGnJIdOSw6Y5uZwPIGHk01GO0s3yC13kCJHkECIOd9NL4dFWmc7GJJt4jpo4jpo4TtCMFBNT7nHaSlU3EQixuJIjhodYGN29JY+PvRBwpTyftWQ3X20/Qb7D6d1/59ALmLPqAEu2HcM3ZoTZLIzpcf71gfIMFz9RwnpBCU0imDe1cv0VpQlmUPD3zVr053MgZTDGzAPmgetOoTyVGNIp3u9s3Qyf1YzKRzBFqp1DOD84u/ED3fA3SeGoM54V9PW7r67II5SdpgM7HR1quipKBY3NIhQ4DWE2S7GU5p6U5wVOZ6H93VrHEBVmw2Fc84KcxjVCKL+Cadi9w8VLCAoRodag9isEMygkA219nicAx0ook+xuPooB0qu6ItU9TFQpVb1CLIIT40106TTFh2gHwu403DSwPTcOaMeC9YeLpTRPzc7zu9+z/UBKNqnZeTSLDOVX8VEVTomemVNAo1ArbZo04twx1xv5x6Q+zF6xj4xzwR32Hcw+BRuujuaRwFFcHc03GmN2+pT5PdDTp6P5N8aYG0o7bmVTZyulVF0w/K3hAKyctrJKjlfjfQruPoJ7ga9xDUl9wxizU0SeATYaYz4HXgfeFZH9uO4QJgWrPkoppcoW1HkKxpglwJIi257yeZwLTAhmHZRSSgWu7iX9UEopFTQaFJRSSnlpUFBKKeWlQUEppZRXncuSKiIpwC8VfHkcRWZLq0L0+pROr0/J9NqUrjZcn/bGmDIXWa9zQaEyRGRjION0Gyq9PqXT61MyvTalq0vXR5uPlOeMaZIAAAZkSURBVFJKeWlQUEop5dXQgsK8mq5ALafXp3R6fUqm16Z0deb6NKg+BaWUUqVraHcKSimlSqFBQSmllFeDCQoiMkZE9orIfhGZUdP1qQ4i0lZEvhWR3SKyU0Tud2+PFZFvRGSf+++m7u0iIrPd12ibiPT1OdZUd/l9IjK1pt5TMIiIVUR+FJEv3M87iMg693tdJCKh7u1h7uf73fsTfY7xmHv7XhEZXTPvpGqJSBMR+VBE9rg/Q4P1s3OeiDzo/n+1Q0QWikh4vfjsGGPq/R9cqbt/Bi4AQoGtQLearlc1vO9WQF/34yhc61t0A/4CzHBvnwH82f34KuBLXOuTDALWubfHAgfcfzd1P25a0++vCq/TH4AFwBfu5x8Ak9yP5wB3ux/fA8xxP54ELHI/7ub+TIUBHdyfNWtNv68quC5vA79zPw4Fmuhnx3tt2gAHgQifz8y0+vDZaSh3CgOA/caYA8aYfOB94JoarlPQGWOOG2M2ux+fAXbj+jBfg+s/PO6/x7sfXwO8Y1x+AJqISCtgNPCNMSbdGHMa+AYYU41vJWhEJAEYC/zL/VyAy4AP3UWKXh/PdfsQGOkufw3wvjEmzxhzENiP6zNXZ4lINDAU15onGGPyjTEZ6GfHlw2IcC8o1gg4Tj347DSUoNAGOOLzPNm9rcFw3672AdYBLYwxx8EVOIDm7mIlXaf6fP3+H/AI4FmztRmQYYzxLOrt+16918G9P9Ndvj5enwuAFOBNd9Pav0SkMfrZAcAYcxR4CTiMKxhkApuoB5+dhhIUxM+2BjMWV0QigY+AB4wxWaUV9bPNlLK9ThORXwOnjDGbfDf7KWrK2Fcfr48N6Au8ZozpA5zF1VxUkoZ0bXD3pVyDq8mnNdAYuNJP0Tr32WkoQSEZaOvzPAE4VkN1qVYiEoIrILxnjPnYvfmk+9Ye99+n3NtLuk719fpdAowTkUO4mhQvw3Xn0MTdJACF36v3Orj3x+BaRrY+Xp9kINkYs879/ENcQUI/Oy6XAweNMSnGmALgY+Bi6sFnp6EEhQ1AJ/fIgFBcHT2f13Cdgs7dZvk6sNsY8zefXZ8DnlEgU4HPfLbf4h5JMgjIdDcRfA2MEpGm7l9Io9zb6jRjzGPGmARjTCKuz8QKY8xNwLfA9e5iRa+P57pd7y5v3NsnuUeYdAA6Aeur6W0EhTHmBHBERLq4N40EdqGfHY/DwCARaeT+f+a5PnX/s1PTvfjV9QfX6IifcPXuP17T9amm9zwE163oNmCL+89VuNoylwP73H/HussL8Kr7Gm0H+vkc6zZcnWD7gVtr+r0F4VoN5/zoowtw/cfcD/wbCHNvD3c/3+/ef4HP6x93X7e9wJU1/X6q6Jr0Bja6Pz+f4ho9pJ+d8+/rT8Ae+P/t3TGIFVcUh/HvHwMhIETWYKdpbEJgUQJpbIRUQbCIwhKU1IKQysKIolta2dhsqYmspQgB2UJJCCRRMCJibXohIhILWU6Ke50dln2r4FPR/X4wvHln5s28gcecufddzuUe8BNtBNE7/9uxzIUkabBRuo8kSS/BpCBJGpgUJEkDk4IkaWBSkCQNTAra8JIsJ7kzWtatopvkSJLvp3DeB0k+fdXjSNPkkFRteEmeVNXmt3DeB7Tx/A/f9LmlSWwpSBP0J/mzSW72ZWePn0lyrK//kOR+n0Pgco/NJLnSY38mme3xrUmWeoG5BUZ1b5Ic7ue4k2Qhyaa3cMmSSUGilT8edx/NjbY9rqqvgPO0ukirHQd2V9UscKTH5oG/e+wEcLHHTwO/VyswdxXYAZDkc2AO2FNVu4Bl4NB0L1F6OR++eBfpvfe034zXsjh6PbfG9rvApSRXaKUgoJUXOQBQVdd7C+ET2vwE3/b4L0n+7ft/DXwJ3GpldPiYlUJz0htlUpDWVxPWn9tHu9nvB04l+YL1yyGvdYwAF6rqx1f5otI02H0krW9u9PrHeEOSD4DtVXWDNlHPFmAz8Bu9+yfJXuBhtXksxvFvaAXmoBWWO5hkW982k+Sz13hN0kS2FKT+n8Lo/bWqej4s9aMkf9EeoL5b9blNwM+9ayjAuap6lOQMbcayu8B/rJRMngcWk9wGfqWVX6aq7ic5CSz1RPMMOAr8M+0LlV7EIanSBA4Z1UZk95EkaWBLQZI0sKUgSRqYFCRJA5OCJGlgUpAkDUwKkqTB//D8Vf4hmeU+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5d72be5f60>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot result\n",
    "import numpy as np\n",
    "episodes = list(range(1, len(scores)+1))\n",
    "plt.plot(episodes, scores,marker='*', label=\"Score per episode\")\n",
    "plt.plot(np.array(list(range(1,len(avg_scores)+1)))*100,avg_scores, label=\"Average score over 100 episodes\")\n",
    "plt.title(\"MADDPG-Tennis\")\n",
    "plt.xlabel(\"Episode\")\n",
    "plt.ylabel(\"Score\")\n",
    "plt.axvline(episodes[-1]-100,color='g')\n",
    "plt.axhline(0.5,color='g')\n",
    "plt.legend()\n",
    "plt.savefig('MADDPG_Tennis_score.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}