mapZ

A Geospatial Application, which I'm still working on.

Putting ⭐ might be a bit inspirational to me.

Documentation

👀 : So, you interested in building a mapping application, is it ?

👀 : Yes

👀 : Okay, I'm gonna take you through each and every step. All you need to do is to follow me. Ready ?

👀 : Definitely

Platform and Tools used

I'm going to use Fedora Linux for implementing this whole process.

For database implementation I'll use PostgreSQL.

 postgres (PostgreSQL) 10.7

GeoSpatial Data to be stored and processed using PostGIS.

 Name         : postgis
 Version      : 2.4.3

A lot of Python scripts are going to be used for automating tile generation and database population procedure.

Python 3.7.3

NPM used for installing different JavaScript dependencies.

 >> npm --version
 6.4.1

Express app to be written for implementing Tile Map Server.

 >> node --version
 v10.15.0
 >> npm info express
 [email protected]

Mapnik used for rendering map tiles.

>> mapnik --version
 3.0.20

Well that's it 😄.

Initial SetUp

First thing first, you need world map data ( here we'll be using shapefiles ) to build a map of world. We'll use GADM as map data source. Here is the data, which we'll require.

So I've written small bash script for you to download and unzip that data. Well you're free to do that on your own too.

#!/usr/bin/bash
# script downloads shape files from GADM, make sure you're connected to internet
wget https://biogeo.ucdavis.edu/data/gadm3.6/gadm36_levels_shp.zip
# unzips downloaded zip into multiple layered shape files, which will be later on used for inflating features into database
unzip gadm36_levels_shp.zip

Now let's setup a PostgreSQL database with postgis extension enabled. Make sure you've installed PostgreSQL database properly on your system. I found it helpful.

Login to PostgreSQL.

 >> psql --username=your-user-name-for-postgresql # which is generally postgres

Create a database named world_features.

 >> create database world_features;

Now simply quit i.e. logout from psql prompt.

 >> \q

Relogin to use newly created database.

 >> psql --username=your-user-name-for-postgresql --dbname=world_features

Let's enable postgis extension for this database. Make sure you've installed postgis first.

 >> create extension postgis;

And initial setup is done. Now we gonna automate things 😉.

Database Population

👀 : That shapefile we downloaded, has 6 layers. So we'll be creating 6 different tables, where we're going to store that huge map data.

👀 : 😟

👀 : Hey wait, we're going to automate that whole thing. Now happy ???

👀 : ☺️

Alright so let's automate. And don't forget to grab a cup of ☕, cause this gonna be a bit longer.

See we're going to process almost few hundreds of thousands features ( mostly polygon / multipolygon geometry ), using geo.Open('/path-to-gadm36_0.shp').GetLayer(0).GetFeature(j).GetGeometryRef().ExportToWkt(), where j is feature count. So of course this gonna be time consuming.

 def app(path='/path-to-file/gadm36_{}.shp', file_id=[0, 1, 2, 3, 4, 5]):
     # path, path to gadm shapefiles
     # gadm has 6 layers, shape files hold corresponding layer number too
     print('[+]Now grab a cup of coffee, cause this gonna be a little longer ...\n')
     for i in file_id:
         print('[+]Working on `{}`'.format(path.format(i)))
         datasource = geo.Open(path.format(i))  # datasource opened
         # layer fetched, only a single layer present in a shape file
         layer = datasource.GetLayer(0)
         tmp = []
         for j in range(layer.GetFeatureCount()):
             feature = layer.GetFeature(j)  # gets feature by id
             gid = 'NA'
             name = 'NA'
             # there might be some fields present in shapefile, which is None
             if(feature.items().get('GID_{}'.format(i)) is not None):
                 # To handle so, I'm adding these two checks, otherwise those might be causing problem during database population
                 gid = feature.items().get('GID_{}'.format(i))
             if(feature.items().get('NAME_{}'.format(i)) is not None):
                 name = feature.items().get('NAME_{}'.format(i))
             tmp.append([gid, name,
                     feature.GetGeometryRef().ExportToWkt()])
             # holds data in temp variable
             # data format -- [feature_id, feature_name, outline]
         if(inflate_into_db('world_features', 'username', 'password', {i: tmp})):
             # finally inflate into database
             print('[+]Success')
 return

Don't forget to change username and password, required for database login, before running this script.

 if(inflate_into_db('world_features', 'username', 'password', {i: tmp})):
        # finally inflate into database
        print('[+]Success')

Simply run this script and it'll be done.

 >>  python3 fetch_and_push.py 

And it's done. Wanna check ?

Login to postgresql.

 >> psql --username=your-user-name-for-postgresql --dbname=world_features

Type in psql prompt.

 >> select feature_id, feature_name from world_features_level_0 where feature_id = 'IND';
 feature_id | feature_name 
 ------------+--------------
 IND        | India
 (1 row)

And this is the structure of table, which the script built for us. All 6 tables has same structure.

 >> \d world_features_level_0;
                Table "public.world_features_level_0"
     Column    |       Type        | Collation | Nullable | Default 
 --------------+-------------------+-----------+----------+---------
 feature_id   | character varying |           | not null | 
 feature_name | character varying |           | not null | 
 outline      | geography         |           |          | 
 Indexes:
     "world_features_level_0_pkey" PRIMARY KEY, btree (feature_id)
     "world_features_level_0_index" gist (outline)

Tile Generation

This gonna be way more time consuming than previous one. I'm still working on it. I'll be back 😎.

Tile Map Server

The Tile Map Server, built using NodeJS i.e. ExpressJS Framework resides here.

Get into tms directory and run following command, which will download all dependencies, required for running this express app.

   >> npm install

This Express app will be working in local network. Make necessary changes, so that it can be discovered from Internet.

   app.listen(8000, '0.0.0.0', () => {
   // tms listens at 0.0.0.0:8000, so that it can be accessed via both localhost and devices present in local network
   console.log('[+]Tile Map Server listening at - `0.0.0.0: 8000`\n');
});

Tile Map Server will be accepting GET request in /tile/:zoom/:row/:col.png path, where zoom is Zoom Level value, row is Row ID( tile identifier along X-axis ) and col is Column ID( tile identifier along Y-axis ).

   app.get('/tile/:zoom/:row/:col.png', (req, res) => {
   . 
   . // lots of code
   .
   });

Ready to run Tile Map Server ?

>>  node index.js

As you can see on terminal its running. Just head to this url, and you get to see a tile, which you built during that very long running tile generation procedure.

Mapping Application

Let's first talk about server side.

Server Side

We gonna write an Express App, which will be serving web pages for displaying maps, and for client, will simply reply on browser, which makes this whole venture a bit platform independent.

Mapping application resides here. As I assume, you've already successfully completed previous steps, simply get into app directory and run app/index.js for launching mapping application server.

   >> cd app
   >> node index.js

Back to client.

Client Side

You might have already found that there's a static directory inside app, which holds a simple web page, index.html, which will be served by app/index.js, ( express app ) and browser will be using it for displaying map with the help of a great JavaScript library leaflet.

For using leaflet in our app, we're going to put following two tags in head tag of index.html.

<link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/leaflet.css"
     integrity="sha512-puBpdR0798OZvTTbP4A8Ix/l+A4dHDD0DGqYW6RQ+9jxkRFclaxxQb/SJAWZfWAkuyeQUytO7+7N4QKrDh+drA=="
     crossorigin="" />
 <script src="https://unpkg.com/[email protected]/dist/leaflet.js"
     integrity="sha512-QVftwZFqvtRNi0ZyCtsznlKSWOStnDORoefr1enyq5mVL4tmKB3S/EnC3rRJcxCPavG10IcrVGSmPh6Qw5lwrg=="
crossorigin=""></script>

For displaying a map we need a div element with id attribute set, in body of html. You need to specify height for this div element.

   <style>
     body {
         margin: 0;
         padding: 0;
     }
     html,
     body,
     #map {
         height: 100%;
     }
 </style>

Put this script with in body of html, which will display map on browser. Of course the heavy lifting is done by leaflet.

<script>
     var map = L.map('map', {
         maxZoom: 5,
         minZoom: 0,
     });
     L.tileLayer('http://localhost:8000/tile/{z}/{x}/{y}.png', {
         attribution: '&copy; 2019 Anjan Roy'
     }).addTo(map);
     map.setView([22, 83], 3);
 </script>

You see, I've created a map with maxZoom: 5 and minZoom: 0, which will be displayed with in a div element, identified by map, id attribute.

   var map = L.map('map', {
         maxZoom: 5,
         minZoom: 0,
     });

Next we're going to add a tileLayer, used for displaying tiles. And the url for tms is http://localhost:8000/tile/{z}/{x}/{y}.png, where z denotes Zoom Level, x denotes tile-id along X-axis and y denotes tile-id along Y-axis.

Well, the top-left most tile is identified as 0-0 tile. After that as you move towards right, x increases and moving downward increases y value.

   L.tileLayer('http://localhost:8000/tile/{z}/{x}/{y}.png', {
         attribution: '© 2019 Anjan Roy'
     }).addTo(map);
     

And we add tileLayer to map. and 💥 !!!

Was it hard ???

👀 -- Facing some problems ?

👀 -- Yes

👀 -- Alright, find me here

Courtesy

Thanking Tobaloidee ❤️ -ily for designing a logo for this project.