-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Templates for data i/o #94
Comments
This was referenced Feb 20, 2019
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It will be helpful to have a set of templates for loading data into Orca and saving outputs back to disk. Like the model step templates, these will be implemented as Orca steps: running a data loading step will register a table, running a data output step will save a table to disk, etc.
Loading data:
urbansim_templates.data.LoadTable()
Earlier discussion in Issue #66.
How should we handle single columns of data?
Saving data:
urbansim_templates.data.SaveTable()
Registering data locations
Would it makes sense to create a stand-alone object to represent "data locations" as well? Data locations would be things like local directory paths, authenticated cloud storage locations, etc.
Representing this separately would allow multiple data i/o steps to refer to a single abstract location that could be easily swapped out. It would also support a data output workflow where a new directory is generated for each model run.
On the other hand, it might be difficult in practice to separate this from the code for reading and writing tables -- probably need to try it out to make sure it's a good idea.
Registering merge relationships
Earlier discussion in Issue #78 about Orca broadcasts vs. implicit join keys. Include some kind of template-based backward compatibility with Orca broadcasts? Probably not.
The text was updated successfully, but these errors were encountered: