Tasks for future releases

[claudiodonofrio]

See here which tasks were included in release 0.1.18

• release 0.1.19
  • function icos_hist_L1_L2_sparql in pylib/icoscp/sparql/sparqls.py should lookup whether a station is labelled or not #61
  • remove all unused sparql queries from the pylib, add a new tool to provide access to all example queries from the sparql endpoint http....
  • ⛓️ ICOS stations: add column to .data() indicating if the data object can be loaded with the pylib or note
  • ⛓️ ICOS dobj: if a pid is set where no data is available, return a sensible error message to the user
  • abstract the backend server for Dobj. The goal is to read datasets from other RI like fieldsites
  • add .help() function returning the url to the documentation
  • Rework stiltstation.find(id='...') #76
  • Multiple PIs not supported; bad naming of PI-related props #24
  • Remove examples directory #132
  • Python main block #133
  • Rework the code from Jonathan here, which was included in release 0.1.18

---

• Major Next Release
  • Extend data access #128
  • find station method #2

---

• Issue or card?
  • Some issues in the station object #134 This could perhaps wait until we remove the sparqls from the station module.
  • Reporting data. __portalUse() currently reports data access. Maybe create a module that can be used from stilt and dobj or collections.
  • Add icos data link to stilt station. (https://stilt.icos-cp.eu/viewer/stationinfo) (Needs elaboration)
  • Consider renaming the cpb module.
  • Filter station.data() by product ('co2' for example).
  • Go through the code and check for strings that need to be added to constants.
  • Licence by pid is available from the metadata store. Check it out by sparqling any pid. License should be fetched from the metadata store and should not be set as a fixed string.
  • Automation of pylib. (🌿 $git checkout tests locally for Zois)
    • Test automation (pytest)
    • Force coding styles (flake8)
    • Automate distribution to PyPI (GitHub workflows, actions, e.t.c) (tox)
    • Zois & Claudio have both worked on this. No issue has yet been published.
  • Convert countries to module (🌿 Here's the branch published, although there isn't any actual work in it: https://github.com/ICOS-Carbon-Portal/pylib/tree/zz_countries_module)
    • Add country column to stilt csv file. (https://stilt.icos-cp.eu/viewer/stationinfo)
    • Remove stations.json file from STILT module.
    • Do we need geoinfo.py within STILT module?
  • Switch nominatim to something lighter independent of release (Probably make a card for this)
    • Include automation (see point below) (Learn how to automate)
    • Implement as webservice with docker.
    • Minimal python
    • Shapefiles
    • More info on slack.

---

• Anders's comments/ideas
  • There are several function calls of the pylib with the parameter ‘project’. Some of this code is implemented only for ‘project’ = ‘ICOS’ (which is fine), but some are related to 'NEON', 'INGOS' or, 'FLUXNET' which is confusing for a user who might compare the pylib to the data portal (which, by the time of writing, has these seven projects: European ObsPack, GCP, ICOS, InGOS, Miscellaneous (Various other data not associated with a specific project), SOCAT, Swedish National Network).
    Should we clarify or explain this somehow?
  • Some improvement suggestions for the Station object, of icoscp.station:
    1. The Station object has the property products. By default products() returns a pandas DataFrame (provided the station object has ‘data’), but it can deliver a dictionary, (which is just the dataframe converted into a dictionary with the key 0 (zero), and then products(‘dict’)[0] is a dictionary), a better choice would be to return:
       • A dictionary with useful keys. In the atmospheric case (we could have other implementations for ES, OS): product labels with sampling heights like: {'Atmospheric CO2 product': [2.5, 8, 33, 60, 150], 'ICOS ATC CO2 Release': [2.5, 10, 30, 60, 150],...}
       • Just a list with labels instead of a data-frame
    2. The Station object could have a method get_dobj() with parameters doi or a product-key, where the product-key, in the case of 'AS' could be a tuple like (‘ICOS ATC CO2 Release’, 60).
  • icoscp.station.fmap() – if provided a dataframe with product data, we could give much more specific info on the station – e.g. if the product_df contains the column 'samplingheight', 'height' or 'depth'. This could be displayed on the map. Here is an example where I renamed the column 'samplingheight' to 'elevation'