Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature scheduling strikes back #564

Open
wants to merge 37 commits into
base: main
Choose a base branch
from

Conversation

iakov-aws
Copy link
Collaborator

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Iakov Gan and others added 27 commits May 15, 2023 09:57
Updated all tabs to include relative date range & as of date. Added new spend by calendar period visual to allow for quarterly analysis
Co-authored-by: Iakov Gan <[email protected]>
* initial for cur-aggregation.yaml

* security recommendations for cur-aggregator

---------

Co-authored-by: Iakov Gan <[email protected]>
Co-authored-by: Iakov Gan <[email protected]>
* added function which creates refresh schedules for new created dataset

* Several smaller bugfixes

* Set proper log levels
* Simplify if conditions
* Create schedule when dataset is updated

* Added schedules in resources.yaml

* Updated resources.yaml with schedules for kpi datasets
* Refresh schedule parameters are taken from config or from environment

* created function which generates random hh:mm time within provided time range

* bugfix when time_to is smaller than time_from, for example '21:00-01:01'

* disabling cron for quicksight refresh schedule

* removed unused functions. bugfix for default

* adjust naming to quicksight structure

* replacing regex solution with split function

* improves logic when creating refresh dataset

* Added schedule for all datasets

* Added schedule for all datasets
* Changed structure in resources.yaml in a way that for each dataset, schedules are at the same level as dependsOn

---------

Co-authored-by: Babic <[email protected]>
cid/helpers/randtime.py Fixed Show fixed Hide fixed
cid/helpers/randtime.py Fixed Show fixed Hide fixed
cid/common.py Fixed Show fixed Hide fixed
cid/helpers/randtime.py Fixed Show fixed Hide fixed
@@ -119,33 +127,45 @@ datasets:
cur: true
views:
- ${cur_table_name}
schedules:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schedule on customer_all?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants