Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Further modularize #42

Open
tcmetzger opened this issue Dec 8, 2022 · 0 comments
Open

Further modularize #42

tcmetzger opened this issue Dec 8, 2022 · 0 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@tcmetzger
Copy link
Contributor

tcmetzger commented Dec 8, 2022

Refactor three functional areas:

  • The core functionality of processing the data into a cloud warehouse-compatible format happens in a core module (io.py). This core module always takes a NumPy array as input and only has minimal dependencies.
  • All handling of concrete input and output data happens in handler modules. For version 0.1, we will have a handler module for local TIF file input and a handler module to write data to BigQuery. In future version, we could add handler modules to use data from an S3 bucket as input, store data in Redshift, etc. These handler modules have their own dependencies (such as rasterio for TIF reading, or google.cloud.bigquery for writing to BigQuery). Those dependencies only get installed when the handler module is actually used.
  • Interaction with end users happens through the CLI module which wraps all functionalities of the handler modules and the core module. The CLI will also handle installation of additional dependencies (e.g. when the TIF reading handler module is used for the first time), and set up of credentials where necessary (e.g. ask for the location of the credentials file for BigQuery)
@tcmetzger tcmetzger added the enhancement New feature or request label Dec 8, 2022
@tcmetzger tcmetzger modified the milestones: 0.1, Next Dec 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

2 participants