Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[9.0] add ElasticPilotParametersDB #7164

Draft
wants to merge 2 commits into
base: integration
Choose a base branch
from

Conversation

fstagni
Copy link
Contributor

@fstagni fstagni commented Aug 8, 2023

For #6622

ToDo:

  • inject records
  • update wiki
  • update docs
  • provide Grafana dashboard def

How to populate this DB? Probably directly from the Pilot, but not only.

BEGINRELEASENOTES

*WMS
NEW: add ElasticPilotParametersDB

ENDRELEASENOTES

@fstagni fstagni force-pushed the 81_elasticPilotParameters branch 2 times, most recently from a48f466 to 7e13f8e Compare August 8, 2023 15:59
indexSplit = int(pilotID) // 1e6
return f"{self.indexName_base}_{indexSplit}m"

def _createIndex(self, indexName: str) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use index templates instead of creating the index manually?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add the index templates, but anyway the index will need to be created by hand, because the logic for the name depends from the JobID.

# The _id in ES can't exceed 512 bytes, this is a ES hard-coded limitation.

# If a record with this pilotID update and add parameter, otherwise create a new record
if self.existsDoc(self._indexName(pilotID), docID=str(pilotID)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we upsert instead?


self.log.debug("Inserting data in {self.indexName}:{data}")

# The _id in ES can't exceed 512 bytes, this is a ES hard-coded limitation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does anything need to be done about this?

@fstagni fstagni mentioned this pull request Sep 18, 2023
6 tasks
@fstagni fstagni changed the title [8.1] add ElasticPilotParametersDB [9.0] add ElasticPilotParametersDB Oct 31, 2023
@fstagni fstagni force-pushed the 81_elasticPilotParameters branch 3 times, most recently from e88e94d to 0d8b501 Compare April 23, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants