code-further-advanced.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(code-further-advanced)=\n",
    "# Advanced Coding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This chapter covers some slightly more advanced programming concepts you're likely to run into when coding in Python, in addition to some recommended extensions and options for advanced use of Visual Studio Code. You probably won't need much of this if you're using Python but if you start to develop packages yourself, there are a few bits—like classes and type hints—that are invaluable.\n",
    "\n",
    "This chapter has benefitted from the online book [*Research Software Engineering with Python*](https://merely-useful.github.io/py-rse/), the [official Python documentation](https://www.python.org/), the excellent [30 days of Python](https://github.com/Asabeneh/30-Days-Of-Python), and the [Hitchhiker's Guide to Python](https://docs.python-guide.org/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Higher order functions\n",
    "\n",
    "Functions are like any other variable in Python, which means you can do some interesting things with them and, well, it can get a bit *meta*. For example, a function can take one or more functions as parameters, a function can be returned as a result of another function, functions can be defined within functions, a function can be assigned to a variable, and you can iterate over functions (for example, if they are in a list).\n",
    "\n",
    "Here's an example that shows how to use a higher order function: it accepts a function, `f`, as an argument and then, using the splat operator `*`, it accepts all arguments of that function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def join_a_string(str_list):\n",
    "    return \" \".join(str_list)\n",
    "\n",
    "\n",
    "def higher_order_function(f, *args):\n",
    "    \"\"\"Lowers case of result\"\"\"\n",
    "    out_string = f(*args)\n",
    "    return out_string.lower()\n",
    "\n",
    "\n",
    "result = higher_order_function(join_a_string, [\"Hello\", \"World!\"])\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next example, we show how to return a function from a function (assigning a function, `result`, to a variable in the process):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def square(x):\n",
    "    return x ** 2\n",
    "\n",
    "\n",
    "def cube(x):\n",
    "    return x ** 3\n",
    "\n",
    "\n",
    "def higher_order_function(type):  # a higher order function returning a function\n",
    "    if type == \"square\":\n",
    "        return square\n",
    "    elif type == \"cube\":\n",
    "        return cube\n",
    "\n",
    "\n",
    "result = higher_order_function(\"square\")\n",
    "print(f\"Using higher_order_function('square'), result(3) yields {result(3)}\")\n",
    "result = higher_order_function(\"cube\")\n",
    "print(f\"Using higher_order_function('cube'), result(3) yields {result(3)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions within functions are allowed. They are known as *closures*. Here's a simple (if contrived) example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "\n",
    "def print_time_now():\n",
    "    def get_curr_time():\n",
    "        return datetime.now().strftime(\"%H:%M\")\n",
    "\n",
    "    now = get_curr_time()\n",
    "    print(now)\n",
    "\n",
    "\n",
    "print_time_now()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's see how to iterate over functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def square_root(x):\n",
    "    return x ** (0.5)\n",
    "\n",
    "\n",
    "functions_list = [square_root, square, cube]\n",
    "\n",
    "for func in functions_list:\n",
    "    print(f\"{func.__name__} applied to 4 is {func(4)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterators\n",
    "\n",
    "An iterator is an object that contains a countable number of values that a single command, `next()`, iterates through. Before that's possible though, we need to take a countable group of some kind and use the `iter()` keyword on it to turn it into an iterator. Let's see an example with some text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_lst = [\"Mumbai\", \"Delhi\", \"Bangalore\"]\n",
    "\n",
    "myiterator = iter(text_lst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, nothing has happened yet, but that's because we didn't call it yet. To get the next iteration, whatever it is, use `next()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "next(myiterator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "next(myiterator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "next(myiterator)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright, we've been through all of the values so... what's going to happen `next`!?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "next(myiterator)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "---------------------------------------------------------------------------\n",
    "StopIteration                             Traceback (most recent call last)\n",
    "<ipython-input-27-29fb3b4dbbec> in <module>\n",
    "----> 1 next(myiterator)\n",
    "\n",
    "StopIteration: \n",
    "```\n",
    "\n",
    "Iterating beyond the end raises a `StopIteration` error because we reached the end. To keep going, use `cycle()` in place of `iter()`. Note that you can build your own iterators (here we used a built-in object type, the `list`, to create an iterator of type `list_iterator`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generators\n",
    "\n",
    "Generator functions return 'lazy' iterators. They are lazy because they do not store their contents in memory. This has *big* advantages for some operations in specific situations: datasets larger than can fit into your computer's memory, or a complex function that needs to maintain an internal state every time it’s called.\n",
    "\n",
    "To give an idea of how and when they work, imagine that (exogenously) integers are really costly, taking as much as 10 MB of space to store (the real figure is more like 128 bytes). We will write a function, \"firstn\", that represents the first $n$ non-negative integers, where $n$ is large. The most naive possible way of doing this would be to build the full list in memory like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def first_n_naive(n):\n",
    "    \"\"\"Build and return a list\"\"\"\n",
    "    num, nums = 0, []\n",
    "    while num < n:\n",
    "        nums.append(num)\n",
    "        num += 1\n",
    "    return nums\n",
    "\n",
    "\n",
    "sum_of_first_n = sum(first_n_naive(1000000))\n",
    "sum_of_first_n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `nums` stores *every* number before returning all of them. In our imagined case, this is completely infeasible because we don't have enough computer space to keep all $n$ 10MB integers in memory.\n",
    "\n",
    "Now we'll rewrite the list-based function as a generator-based function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def first_n_generator(n):\n",
    "    \"\"\"A generator that yields items instead of returning a list\"\"\"\n",
    "    num = 0\n",
    "    while num < n:\n",
    "        yield num\n",
    "        num += 1\n",
    "\n",
    "\n",
    "sum_of_first_n = sum(first_n_generator(1000000))\n",
    "sum_of_first_n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, instead of creating an enormous list that has to be stored in memory, we `yield` up each number as it is 'generated'. The cleverness that's going on here is that  the 'state' of the function is remembered from one call to the next. This means that when `next()` is called on a generator object (either explicitly or implicitly, as in this example), the previously yielded variable `num` is incremented, and then yielded again.\n",
    "\n",
    "That was a fairly contrived example but there are plenty of practical ones. Working with pipelines that process very large datasets is a classic use case. For example, imagine you have a csv file that's far too big to fit in memory, i.e. open all at once, but you'd like to check the contents of each row and perhaps process them. The code below would `yield` each row in turn.\n",
    "\n",
    "```python\n",
    "def csv_reader(file_name):\n",
    "    for row in open(file_name, \"r\"):\n",
    "        yield row\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An even more concise way of defining this is via a *generator expression*, which syntactically looks a lot like a *list comprehension* but is a generator rather than a list. The example we just saw would be written as:\n",
    "\n",
    "```python\n",
    "csv_gen = (row for row in open(file_name))\n",
    "```\n",
    "\n",
    "It's easier to see the difference in the below example which clearly shows the analogy between *list comprehensions* and *generator comprehensions*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sq_nums_lc = [num ** 2 for num in range(2, 6)]\n",
    "sq_nums_lc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sq_nums_gc = (num ** 2 for num in range(2, 6))\n",
    "sq_nums_gc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The latter is a generator object and we can only access individual values calling `next()` on it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "next(sq_nums_gc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that for small numbers of entries, lists may actually be faster and more efficient than generators-but for large numbers of entries, generators will almost always win out."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decorators\n",
    "\n",
    "Decorators 'decorate' functions, they adorn them, modifying them as they execute. Let's say we want to run some numerical functions but we'd like to add ten on to whatever results we get. We could do it like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def multiply(num_one, num_two):\n",
    "    return num_one * num_two\n",
    "\n",
    "\n",
    "def add_ten(in_num):\n",
    "    return in_num + 10\n",
    "\n",
    "\n",
    "answer = add_ten(multiply(3, 4))\n",
    "answer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is fine for a one-off but a bit tedious if we're going to be using `add_ten()` a lot, and on many functions. Decorators allow for a more general solution that can be applied, in this case, to any `inner()` function that has two arguments and returns a numeric value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_ten(func):\n",
    "    def inner(a, b):\n",
    "        return func(a, b) + 10\n",
    "\n",
    "    return inner\n",
    "\n",
    "\n",
    "@add_ten\n",
    "def multiply(num_one, num_two):\n",
    "    return num_one * num_two\n",
    "\n",
    "\n",
    "multiply(3, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the same decorator for a different function (albeit one of the same form) now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@add_ten\n",
    "def divide(num_one, num_two):\n",
    "    return num_one / num_two\n",
    "\n",
    "\n",
    "divide(10, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But the magic of decorators is such that we can define them for much more general cases, regardless of the number of arguments or even keyword arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_ten(func):\n",
    "    def inner(*args, **kwargs):\n",
    "        print(\"Function has been decorated!\")\n",
    "        print(\"Adding ten...\")\n",
    "        return func(*args, **kwargs) + 10\n",
    "\n",
    "    return inner\n",
    "\n",
    "\n",
    "@add_ten\n",
    "def combine_three_nums(a, b, c):\n",
    "    return a * b - c\n",
    "\n",
    "\n",
    "@add_ten\n",
    "def combine_four_nums(a, b, c, d=0):\n",
    "    return a * b - c - d\n",
    "\n",
    "\n",
    "combine_three_nums(1, 2, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now see it applied to a function with a different number of (keyword) arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "combine_four_nums(3, 4, 2, d=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Decorators can be chained too (and order matters):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dividing_line(func):\n",
    "    def inner(*args, **kwargs):\n",
    "        print(\"\".join([\"-\"] * 30))\n",
    "        out = func(*args, **kwargs)\n",
    "        return out\n",
    "\n",
    "    return inner\n",
    "\n",
    "\n",
    "@dividing_line\n",
    "@add_ten\n",
    "def multiply(num_one, num_two):\n",
    "    return num_one * num_two\n",
    "\n",
    "\n",
    "multiply(3, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classes and objects\n",
    "\n",
    "Python is an object oriented programming language. Everything is an object (and every object has a type). A Class is an object constructor, a blueprint for creating objects. An object is a 'live' instance of a class. Objects are to classes what a yellow VW Beetle is to cars. The class defines the attributes and methods that the object can perform.\n",
    "\n",
    "Classes and instances of them are useful in certain situations, the most common being when you need something that has 'state', i.e. it can remember things that have happened to it, carry information with it, and change form.\n",
    "\n",
    "While you're quite unlikely to need to build classes in economics (unless you're doing something really fancy), some of the biggest Python packages are based around classes so it's useful to understand a bit about how they work, and especially how they have state.\n",
    "\n",
    "The syntax to create a class is\n",
    "\n",
    "```python\n",
    "class ClassName:\n",
    "  ...code...\n",
    "```\n",
    "\n",
    "But it's easiest to show with an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a class called Person\n",
    "\n",
    "\n",
    "class Person:\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "\n",
    "\n",
    "# Create an instance of the class\n",
    "p = Person(\"Adam\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we check `type()`, that's when it gets *really* interesting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Woah! We created a whole new data type based on the `Class` name. The class has a constructor method, `__init__`, that, in this case, takes an input variable `name` and assigns it to an *internal* object variable `name`. The `self` variable that you can also see is really saying 'generate an object of type this Class when called'. We can access any internal variables like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay but what's the point of all this? Well we can now create as many objects as we like of class 'Person' and they will have the same structure, but not the same state, as other objects of class 'Person'.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = Person(\"Ada\")\n",
    "m.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a very boring class! Let's add a method, which will allow us to change the state of objects. Here, we add a method `increment_age` which is also indented under the `class Person` header. Note that it takes `self` as an input, just like the constructor, but it only acts on objects of type person that have *already* been created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __init__(self, name, age):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "\n",
    "    def increment_age(self):\n",
    "        self.age = self.age + 1\n",
    "\n",
    "\n",
    "# Create an instance of the class\n",
    "p = Person(\"Adam\", 231)\n",
    "\n",
    "print(p.age)\n",
    "# Call the method increment_age\n",
    "p.increment_age()\n",
    "print(p.age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This very simple method changes the internal state. Just like class constructors and regular functions, class methods can take arguments. Here's an example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __init__(self, name, age):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "\n",
    "    def increment_age(self):\n",
    "        self.age = self.age + 1\n",
    "\n",
    "    def change_age(self, new_age):\n",
    "        self.age = new_age\n",
    "\n",
    "\n",
    "# Create an instance of the class\n",
    "p = Person(\"Adam\", 231)\n",
    "\n",
    "print(p.age)\n",
    "# Call the method increment_age\n",
    "p.change_age(67)\n",
    "print(p.age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can be tedious to have to initialise a class with a whole load of parameters every time. Just like with functions, we can define *default parameters* for classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person:\n",
    "    def __init__(self, name=\"default_name\", age=20):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "\n",
    "\n",
    "p = Person()\n",
    "p.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That covers a lot of the basics of classes but if you're using classes in anger then you might also want to look up [inheritance and composition](https://realpython.com/inheritance-composition-python/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataclassess\n",
    "\n",
    "The basic classes we created above come with a lot of 'boilerplate'; code we need but which is not very surprising. Dataclasses were introduced in Python 3.7 as a way to remove this boilerplate when the classes being created are quite simple. Think of dataclasses as a class with sensible defaults that is for light object-oriented programming.\n",
    "\n",
    "A simple example, with a `Circle` class, demonstrates why they are effective. First, the full class way of doing things:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "class Circle1:\n",
    "    def __init__(self, colour: str, radius: float) -> None:\n",
    "        self.colour = colour\n",
    "        self.radius = radius\n",
    "\n",
    "    def area(self) -> float:\n",
    "        return np.pi * self.radius ** 2\n",
    "\n",
    "\n",
    "circle1 = Circle1(\"red\", 2)\n",
    "circle1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We don't get a very informative message when we call `circle1`, as you can see. At least we can compute its area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "circle1.area()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll create the same object with dataclasses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataclasses import dataclass\n",
    "\n",
    "\n",
    "@dataclass\n",
    "class Circle2:\n",
    "    colour: str\n",
    "    radius: float\n",
    "\n",
    "    def area(self) -> float:\n",
    "        return np.pi * self.radius ** 2\n",
    "\n",
    "\n",
    "circle2 = Circle2(\"blue\", 2)\n",
    "circle2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Right away we get a much more informative message when we call the object, *and* the class definition is a whole lot simpler. Everything else is just the same (just try calling `circle2.area()`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Type annotations and type checkers\n",
    "\n",
    "Type annotations were introduced in Python 3.5 (these notes are written in 3.8). If you've seen more low-level languages, typing will be familiar to you. Python uses 'duck typing' (\"if it walks and quacks like a duck, it is a duck\") which means that if a variable walks like an integer, and talks like an integer, then it gets treated as if it is an integer. Ditto for other variable types. Duck typing is useful if you just want to code quickly and aren't writing production code.\n",
    "\n",
    "But... there are times when you *do* know what variable types you're going to be dealing with ahead of time and you want to prevent the propagation of the wrong kinds of variable types. In these situations, you can clearly say what variable types are supposed to be. And, when used with some other packages, typing can make code easier to understand, debug, and maintain.\n",
    "\n",
    "Note that it doesn't have to be all or nothing on type checking, you can just add it in gradually or where you think it's most important.\n",
    "\n",
    "Now it's important to be really clear on one point, namely that *Python does **not** enforce type annotations*. But we can use *static type checking* to ensure all types are as they should be in *advance* of running. Before we do that, let's see how we add type annotations. \n",
    "\n",
    "This is the simplest example of a type annotation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "answer: int = 42"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This explicitly says that `answer` is an integer. Type annotations can be used in functions too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def increment(number: int) -> int:\n",
    "    return number + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A static type checker uses these type annotations to verify the type correctness of a programme without executing it. [**mypy**](http://mypy-lang.org/) is the most widely used static type checker. After installing **mypy**, to run type checking on a file `code_script.py` use\n",
    "\n",
    "```bash\n",
    "mypy code_script.py\n",
    "```\n",
    "\n",
    "on the command line.\n",
    "\n",
    "What do you see when you run it? Let's say the content of your script is:\n",
    "\n",
    "```python\n",
    "# Contents of code_script.py\n",
    "def greeting(name: str) -> str:\n",
    "    return 'Hello ' + name\n",
    "\n",
    "\n",
    "greeting(3)\n",
    "```\n",
    "\n",
    "This would return:\n",
    "\n",
    "```bash\n",
    "Argument 1 to \"greeting\" has incompatible type \"int\"; expected \"str\"\n",
    "```\n",
    "\n",
    "Here are more of the type annotations that you might need or come across, courtesy of the [**mypy**](http://mypy-lang.org/) documentation:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "from typing import List, Set, Dict, Tuple, Optional\n",
    "\n",
    "# For simple built-in types, just use the name of the type\n",
    "x: int = 1\n",
    "x: float = 1.0\n",
    "x: bool = True\n",
    "x: str = \"test\"\n",
    "x: bytes = b\"test\"\n",
    "\n",
    "# For collections, the type of the collection item is in brackets\n",
    "# (Python 3.9+ only)\n",
    "x: list[int] = [1]\n",
    "x: set[int] = {6, 7}\n",
    "\n",
    "# In Python 3.8 and earlier, the name of the collection type is\n",
    "# capitalized, and the type is imported from 'typing'\n",
    "x: List[int] = [1]\n",
    "x: Set[int] = {6, 7}\n",
    "\n",
    "# Same as above, but with type comment syntax (Python 3.5 and earlier)\n",
    "x = [1]  # type: List[int]\n",
    "\n",
    "# For mappings, we need the types of both keys and values\n",
    "x: dict[str, float] = {'field': 2.0}  # Python 3.9+\n",
    "x: Dict[str, float] = {'field': 2.0}\n",
    "\n",
    "# For tuples of fixed size, we specify the types of all the elements\n",
    "x: tuple[int, str, float] = (3, \"yes\", 7.5)  # Python 3.9+\n",
    "x: Tuple[int, str, float] = (3, \"yes\", 7.5)\n",
    "\n",
    "# For tuples of variable size, we use one type and ellipsis\n",
    "x: tuple[int, ...] = (1, 2, 3)  # Python 3.9+\n",
    "x: Tuple[int, ...] = (1, 2, 3)\n",
    "\n",
    "# Use Optional[] for values that could be None\n",
    "x: Optional[str] = some_function()\n",
    "# Mypy understands a value can't be None in an if-statement\n",
    "if x is not None:\n",
    "    print(x.upper())\n",
    "# If a value can never be None due to some invariants, use an assert\n",
    "assert x is not None\n",
    "print(x.upper())\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I am the Walrus\n",
    "\n",
    "The Walrus operator, `:=` was introduced in Python 3.8 and, well, it's fairly complicated but it *does* have its uses. The main use case for the Walrus operator is when you want to both *evaluate an expression* and *assign a variable* in one fell swoop.\n",
    "\n",
    "Take this (trivial) example which involves evaluating an expression, `len(a) > 4`, that returns a boolean and then assigning that *same* expression to a variable `n`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [1, 2, 3, 4]\n",
    "if len(a) > 3:\n",
    "    n = len(a)\n",
    "    print(f\"List is too long ({n} elements, expected <= 3)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Walrus operator allows us to skip the clumsy use of `len(a)` twice and do both steps in one go. As noted, that's trivial here, but if evaluation were very computationally expensive, then this might save us some trouble. Here's the version with the Walrus operator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [1, 2, 3, 4]\n",
    "if (n := len(a)) > 3:\n",
    "    print(f\"List is too long ({n} elements, expected <= 3)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Map, filter, and reduce\n",
    "\n",
    "Map, filter, and reduce are built-in higher order functions. Lambda functions, featured in the basics of coding chapter, can be passed as into each of these as an argument and some of the best use cases of lambda functions are in conjunction with map, filter, and reduce.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Map\n",
    "\n",
    "`map()` takes a function and an iterable as arguments, ie the syntax is `map(function, iterable)`. An iterable is a type that is composed of elements that can be iterated over. The map essentially applies the function to each entry in the iterable. Here's an example where a list of strings is cast to integers via `map()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numbers_str = [\"1\", \"2\", \"3\", \"4\", \"5\"]\n",
    "mapped_result = map(int, numbers_str)\n",
    "list(mapped_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example with a lambda function. The benefit of using a lambda in this map operation is that otherwise we would have to write a whole function that simply returned the input with `.title()` at the end:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "names = [\"robinson\", \"fawcett\", \"ostrom\"]\n",
    "names_titled = map(lambda name: name.title(), names)\n",
    "list(names_titled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filter\n",
    "\n",
    "`filter()` calls a specified function and returns a boolean for each item of the specified iterable. It filters the items that satisfy the given boolean criteria. It uses the `filter(function, iterable)` syntax. In the example below, we take all the numbers from zero to five and filter them according to whether they are divisible by 2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numbers = list(range(6))\n",
    "fil_result = filter(lambda x: x % 2 == 0, numbers)\n",
    "list(fil_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reduce\n",
    "\n",
    "`reduce()` is defined in the built-in `functools` module. Like `map()` and `filter()`, `reduce()` takes two parameters, a function and an iterable. However, it returns a single value rather than another iterable. The way `reduce()` works is to apply operations successively so that the example below effectively first sums 2 and 3 to make 5, then 5 and 5 to make 10, then 10 and 15 to make 25, and, finally, 25 and 20 to make the final result of 45."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import reduce\n",
    "\n",
    "numbers = [2, 3, 5, 15, 20]\n",
    "\n",
    "reduce(lambda x, y: x + y, numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Non-local variables\n",
    "\n",
    "Non-local variables are used in nested functions as a means to say 'hey, this variable is not just local to this nested function, it exists outside it too'. Here's an example that prints \"world\" because we tell the inner function to use the same `x` as the outer function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d78cc72",
   "metadata": {},
   "outputs": [],
   "source": [
    "def outer_function():\n",
    "  x = \"hello\"\n",
    "  def nested_function():\n",
    "    nonlocal x\n",
    "    x = \"world\"\n",
    "  nested_function()\n",
    "  return x\n",
    "\n",
    "print(outer_function())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d95d5ec7",
   "metadata": {},
   "source": [
    "```{admonition} Exercise\n",
    "Re-write the above function without the `nonlocal` keyword. What does it print?\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple dispatch\n",
    "\n",
    "One can use object-oriented methods and inheritance to get different code objects to behave in different ways depending on the type of input. For example, a different behaviour might occur if you send a string into a function versus an integer. An alternative to the object-oriented approach is to use *multiple dispatch*. [**fastcore**](https://fastcore.fast.ai/) is a library that provides \"goodies to make your coding faster, easier, and more maintainable\" and has many neat features but amongst the goodies is multiple dispatch, with the `typedispatch` decorator. The example below doesn't execute but shows you how the library can be used to define different behaviours for inputs of different types."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "# fastcore is designed to be imported as *\n",
    "from fastcore.dispatch import *\n",
    "\n",
    "\n",
    "@typedispatch\n",
    "def func_example(x: int, y: float):\n",
    "    return x + y\n",
    "\n",
    "\n",
    "@typedispatch\n",
    "def func_example(x: int, y: int):\n",
    "    return x * y\n",
    "\n",
    "\n",
    "# Int and float\n",
    "print(func_example(5, 5.0))\n",
    "\n",
    "# Int and int\n",
    "print(func_example(5, 5))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we can see here is that we have the same function, `func_example`, used twice with *very* similar inputs. But the inputs are *not* the same; in the first instance it's an integer and a float while in the second it's two integers. The different inputs get routed into the different versions of the `@typeddispatch` function. This decorator-based approach is not the only way to use **fastcore** to do typed dispatch but it's one of the most convenient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fine-tuning Visual Studio Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This section has tips on making your IDE, *Visual Studio Code*, even more effective.\n",
    "\n",
    "VS Code can do a whole lot more with some extra add-ons. You can install these using the extensions tab on the left hand side of VS Code. Here are the ones this book recommends and why:\n",
    "\n",
    "- Markdown extensions - markdown is a simple text language that is often used to provide READMEs for code repositories. It comes with the file extension .md\n",
    "  - *Markdown All in One*, to help writing Markdown docs.\n",
    "  - *Markdown Preview Enhanced*, to view rendered markdown as you type it (right click and select 'Open Preview...').\n",
    "- Coding extensions\n",
    "  - *Jupyter* provides support for Jupyter Notebooks\n",
    "  - *indent-rainbow*, gives different levels of indentation different colours for ease of reading.\n",
    "  - *Path Intellisense*, autocompletes filenames in code.\n",
    "  - *autoDocstring*, automatically creates a basic doc string whenever you write a function.\n",
    "- Version control\n",
    "  - *Git History*, view and search your git log along and show a graph of git commits with details.\n",
    "  - *GitLens*, helps to visualise code authorship at a glance via 'Git blame' annotations, navigate and explore Git repositories, and more.\n",
    "  - *Git Graph*, view a graph of your commit history.\n",
    "  - *Code Spell Checker*, does exactly what it says, really useful for avoiding mangled variable name errors. If you need it to use, for example, 'British English', change the 'C Spell: Language' text from 'en' to 'en-GB' in VS Code's settings. Other languages are available as separate extensions.\n",
    "- General\n",
    "  - *Rainbow CSV*, uses colour to make plain old CSV files much more readable in VS Code.\n",
    "  - *vscode-icons*, intelligent icons for your files as seen in the VS Code file explorer, eg a folder called data gets an icon showing a disc drive superimposed onto a folder.\n",
    "  - *polacode*, take pictures of code snippets to share on social media\n",
    "  - *Excel viewer*, does what it says\n",
    "  - *Selection Word Count*, calculates and displays the word count of a document and, when there is a selection, the word count of a selection (both are shown in the status bar)\n",
    "  - *LiveShare*, to collaborate on code with someone else in real-time\n",
    "- LaTeX - it's a bit of surprise, but VS Code is one of the best LaTeX editors out there. You will need LaTeX installed already though and initial setup of a compilation 'recipe' is a bit fiddly (though, once it works, it's dreamy).\n",
    "  - *LaTeX Workshop*, provides core features for LaTeX typesetting with Visual Studio Code.\n",
    "  - *LaTeX Preview*, both in-line and side-by-side previews of LaTeX code. A really fantastic extension.\n",
    "\n",
    "There are some extensions that most people won't need but which experienced coders may find useful:\n",
    "\n",
    "- *Github Pull Request* — allows you to review and manage GitHub pull requests and issues in Visual Studio Code\n",
    "- *Remote development* — allows you to open any folder in: a container, a remote machine, or the Windows Subsystem for Linux (WSL)\n",
    "- *Remote - WSL* — run VS Code in the Windows Subsystem for Linux\n",
    "- *Remote - SSH* — run VS Code over an SSH connection, eg in the cloud\n",
    "- *Remote - Container* — run VS Code in a Docker container\n",
    "- *Docker* - makes it easy to build, manage, and deploy Docker containers from Visual Studio Code\n",
    "\n",
    "A useful setting (under the cog icon, then settings) for coding is to change the 'Editor: Render Whitespace', aka `editor.renderWhitespace`, from 'selection' to 'boundary'. This will now show any boundary whitespace, or more than one instance of whitespace contiguously, as a grey dot. This might seem odd but it's really useful because the wrong amount of whitespace can create problems with code."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3.10.12 ('codeforecon')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "c4570b151692b3082981c89d172815ada9960dee4eb0bedb37dc10c95601d3bd"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}