tutorial-1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# sat-search\n",
    "\n",
    "This notebook is a tutorial on how to use sat-search to search STAC APIs, save the results, and download assets.\n",
    "\n",
    "Sat-search is built using [sat-stac](https://github.com/sat-utils/sat-stac) which provides the core Python classes used to represent STAC catalogs: `Collection`, `Item`, and `Items`. It is recommended to review the [tutorial on STAC Classes](https://github.com/sat-utils/sat-stac/blob/master/tutorial-2.ipynb) for more information on how to use these objects returned from searching.\n",
    "\n",
    "Only the `search` module is in sat-search is used as a library, and it contains a single class, `Search`. The `parser` module is used for creating a CLI parser, and `main` contains the main function used in the CLI.\n",
    "\n",
    "**API endpoint**: Sat-search required an endpoint to be passed in or defined by the STAC_API_URL environment variable. This tutorial uses https://earth-search.aws.element84.com/v0 but any STAC endpoint can be used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initializing a Search object\n",
    "\n",
    "The first step in performing a search is to create a Search object with all the desired query parameters. Query parameters need to follow the querying as provided in the [STAC specification](https://github.com/radiantearth/stac-spec), although an abbreviated form is also supported (see below).\n",
    "\n",
    "Another place to look at the STAC query format is in the [sat-api docs](http://sat-utils.github.io/sat-api/), specifically see the section on [full-features querying](http://sat-utils.github.io/sat-api/#search-stac-items-by-full-featured-filtering-) which is what sat-search uses to POST queries to an API. Any field that can be provided in the [searchBody](http://sat-utils.github.io/sat-api/#tocssearchbody) can be provided as a keyword parameter when creating the search. These fields include:\n",
    "\n",
    "- bbox: bounding box of the form [minlon, minlat, maxlon, maxlat]\n",
    "- intersects: A GeoJSON geometry\n",
    "- time: A single date-time, a period string, or a range (seperated by /)\n",
    "- sort: A dictionary of fields to sort along with ascending/descending\n",
    "- query: Dictionary of properties to query on, supports eq, lt, gt, lte, gte\n",
    "\n",
    "Examples of queries are in the sat-api docs, but an example JSON query that would be POSTed might be:\n",
    "\n",
    "```\n",
    "{\n",
    "  \"bbox\": [\n",
    "    -110,\n",
    "    39.5,\n",
    "    -105,\n",
    "    40.5\n",
    "  ],\n",
    "  \"time\": \"2018-02-12T00:00:00Z/2018-03-18T12:31:12Z\",\n",
    "  \"query\": {\n",
    "    \"eo:cloud_cover\": {\n",
    "      \"lt\": 10\n",
    "    }\n",
    "  },\n",
    "  \"sort\": [\n",
    "    {\n",
    "      \"field\": \"eo:cloud_cover\",\n",
    "      \"direction\": \"desc\"\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "```\n",
    "\n",
    "### Simple queries\n",
    "\n",
    "In sat-search, each of the fields in the query is simply provided as a keyword argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bbox search: 10305 items\n",
      "time search: 474806 items\n",
      "cloud_cover search: 7359438 items\n"
     ]
    }
   ],
   "source": [
    "from satsearch import Search\n",
    "\n",
    "search = Search(bbox=[-110, 39.5, -105, 40.5])\n",
    "print('bbox search: %s items' % search.found())\n",
    "\n",
    "search = Search(datetime='2018-02-12T00:00:00Z/2018-03-18T12:31:12Z')\n",
    "print('time search: %s items' % search.found())\n",
    "\n",
    "search = Search(query={'eo:cloud_cover': {'lt': 10}})\n",
    "print('cloud_cover search: %s items' % search.found())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Complex query\n",
    "\n",
    "Now we combine all these filters and add in a sort filter to order the results (which will be shown further below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21 items\n"
     ]
    }
   ],
   "source": [
    "search = Search(bbox=[-110, 39.5, -105, 40.5],\n",
    "               datetime='2018-02-12T00:00:00Z/2018-03-18T12:31:12Z',\n",
    "               query={'eo:cloud_cover': {'lt': 10}},\n",
    "               collections=['sentinel-s2-l2a'])\n",
    "print('%s items' % search.found())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Intersects query\n",
    "\n",
    "The intersects query works the same way, except a geometry is provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "intersects search: 4815 items\n"
     ]
    }
   ],
   "source": [
    "geom = {\n",
    "    \"type\": \"Polygon\",\n",
    "    \"coordinates\": [\n",
    "      [\n",
    "        [\n",
    "          -66.3958740234375,\n",
    "          43.305193797650546\n",
    "        ],\n",
    "        [\n",
    "          -64.390869140625,\n",
    "          43.305193797650546\n",
    "        ],\n",
    "        [\n",
    "          -64.390869140625,\n",
    "          44.22945656830167\n",
    "        ],\n",
    "        [\n",
    "          -66.3958740234375,\n",
    "          44.22945656830167\n",
    "        ],\n",
    "        [\n",
    "          -66.3958740234375,\n",
    "          43.305193797650546\n",
    "        ]\n",
    "      ]\n",
    "    ]\n",
    "}\n",
    "\n",
    "search = Search(intersects=geom)\n",
    "print('intersects search: %s items' % search.found())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Alternate search syntax\n",
    "\n",
    "This all works fine, except the syntax for creating queries is a bit verbose, so sat-search allows an alternate syntax using simple strings of the property and equality symbols. \n",
    "\n",
    "A typical query is shown below for eo:cloud_cover, along with the alternate versions that use the alternate syntax."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7359438 items found\n",
      "7359438 items found\n"
     ]
    }
   ],
   "source": [
    "query = {\n",
    "  \"eo:cloud_cover\": {\n",
    "    \"lt\": 10\n",
    "  }\n",
    "}\n",
    "\n",
    "search = Search(query=query)\n",
    "print('%s items found' % search.found())\n",
    "\n",
    "search = Search.search(query=[\"eo:cloud_cover<10\"])\n",
    "print('%s items found' % search.found())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fetching results\n",
    "\n",
    "The examples above use the Search::found() function, but this only returns the total number of hits by performing a fast query with limit=0 (returns no items). To fetch the actual Items use the `Search::items()` function. This returns a sat-stac `Items` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15 items\n",
      "15 items\n",
      "2 collections\n",
      "[sentinel-s2-l2a, sentinel-s2-l1c]\n",
      "S2B_12TXK_20180204_0_L1C\n",
      "S2B_12TYK_20180204_0_L1C\n",
      "S2B_12TWK_20180204_0_L1C\n",
      "S2B_13TBE_20180204_0_L1C\n",
      "S2B_13TBE_20180204_0_L2A\n",
      "S2B_12TYK_20180204_0_L2A\n",
      "S2B_12TWK_20180204_0_L2A\n",
      "S2B_13TCE_20180204_0_L2A\n",
      "S2B_12TXK_20180204_0_L2A\n",
      "S2A_12SWJ_20180202_0_L1C\n",
      "S2A_12SXJ_20180202_0_L1C\n",
      "S2A_12SXJ_20180202_0_L2A\n",
      "S2A_12SWJ_20180202_0_L2A\n",
      "S2B_13SED_20180201_0_L1C\n",
      "S2B_13SED_20180201_0_L2A\n"
     ]
    }
   ],
   "source": [
    "search = Search(bbox=[-110, 39.5, -105, 40.5],\n",
    "               datetime='2018-02-01/2018-02-04',\n",
    "               property=[\"eo:cloud_cover<5\"])\n",
    "print('%s items' % search.found())\n",
    "\n",
    "items = search.items()\n",
    "print('%s items' % len(items))\n",
    "print('%s collections' % len(items._collections))\n",
    "print(items._collections)\n",
    "\n",
    "for item in items:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Limit\n",
    "\n",
    "The `search.items()` function does take 1 argument, `limit`. This is the total number of items that will be returned. Behind the scenes sat-search may make multiple queries to the API, up until either the limit, or the total number of hits, whichever is greater."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "There are more items found (15) than the limit (2) provided.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Items (2):\n",
      "date                      id                        \n",
      "2018-02-04                S2B_12TXK_20180204_0_L1C  \n",
      "2018-02-04                S2B_12TYK_20180204_0_L1C  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "items = search.items(limit=2)\n",
    "print(items.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Returned `Items`\n",
    "\n",
    "The returned `Items` object has several useful functions and is covered in detail in the [sat-stac STAC classes tutorial](https://github.com/sat-utils/sat-stac/blob/master/tutorial-2.ipynb). The `Items` object contains all the returned Items (`Items._items`), along with any Collection references by those Items (`Items._collections`), and the search parameters used (`Items._search`) Below are some examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Items (2):\n",
      "date                      id                        \n",
      "2018-02-04                S2B_12TXK_20180204_0_L1C  \n",
      "2018-02-04                S2B_12TYK_20180204_0_L1C  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(items.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Items (6):\n",
      "date                      id                        \n",
      "2018-02-07                S2B_12TWK_20180207_0_L2A  \n",
      "2018-02-07                S2B_12TXK_20180207_0_L2A  \n",
      "2018-02-07                S2B_12SWJ_20180207_0_L2A  \n",
      "2018-02-07                S2B_12SXJ_20180207_0_L2A  \n",
      "2018-02-02                S2A_12SXJ_20180202_0_L2A  \n",
      "2018-02-02                S2A_12SWJ_20180202_0_L2A  \n",
      "\n",
      "Items (6):\n",
      "date                      id                        eo:cloud_cover            \n",
      "2018-02-07                S2B_12TWK_20180207_0_L2A  2.45                      \n",
      "2018-02-07                S2B_12TXK_20180207_0_L2A  17.91                     \n",
      "2018-02-07                S2B_12SWJ_20180207_0_L2A  12.85                     \n",
      "2018-02-07                S2B_12SXJ_20180207_0_L2A  2.54                      \n",
      "2018-02-02                S2A_12SXJ_20180202_0_L2A  5.74                      \n",
      "2018-02-02                S2A_12SWJ_20180202_0_L2A  18.46                     \n",
      "\n"
     ]
    }
   ],
   "source": [
    "from satstac import ItemCollection\n",
    "\n",
    "search = Search.search(bbox=[-110, 39.5, -105, 40.5],\n",
    "               datetime='2018-02-01/2018-02-10',\n",
    "               query=[\"eo:cloud_cover<25\"],\n",
    "               collections=['sentinel-s2-l2a'])\n",
    "items = search.items()\n",
    "print(items.summary())\n",
    "\n",
    "items.save('test.json')\n",
    "items2 = ItemCollection.open('test.json')\n",
    "\n",
    "print(items2.summary(['date', 'id', 'eo:cloud_cover']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['downloads/2018-02-07/S2B_12TWK_20180207_0_L2A_metadata.xml', 'downloads/2018-02-07/S2B_12TXK_20180207_0_L2A_metadata.xml', 'downloads/2018-02-07/S2B_12SWJ_20180207_0_L2A_metadata.xml', 'downloads/2018-02-07/S2B_12SXJ_20180207_0_L2A_metadata.xml', 'downloads/2018-02-02/S2A_12SXJ_20180202_0_L2A_metadata.xml', 'downloads/2018-02-02/S2A_12SWJ_20180202_0_L2A_metadata.xml']\n"
     ]
    }
   ],
   "source": [
    "# download a specific asset from all items and put in a directory by date in 'downloads'\n",
    "filenames = items.download('metadata', filename_template='downloads/${date}/${id}')\n",
    "print(filenames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}