Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimised getLastSeenEventTimestamp function #20134

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

ronaldlangeveld
Copy link
Member

@ronaldlangeveld ronaldlangeveld commented May 6, 2024

ref ENG-832

depends on #20118 for migration column in emails table, will rebase accordingly.

  • every time handleDelivered, handleOpened and handlePermanentFailed fires, it will also update the latest_event_timestmap in the email table.
  • we then conditionally remove 3 email queries in favour of 1 query that will get the latest event timestamp
  • added additional tests

ref ENG-832

- Added migrations for latest_event_timestamp column in emails table.
- updated schema
- updated emails model
ref ENG-832

- added a new column in the email table called `latest_event_timestamp`.
- everytime `handleDelivered`, `handleOpened` and `handlePermanentFailed` fires, it will also update the `latest_event_timestmapi` in the email table.
- we then conditionally remove 3 email queries in favour of 1 query that
  will get the latest event timestamp
@github-actions github-actions bot added the migration [pull request] Includes migration for review label May 6, 2024
Copy link
Contributor

github-actions bot commented May 6, 2024

It looks like this PR contains a migration 👀
Here's the checklist for reviewing migrations:

General requirements

  • Satisfies idempotency requirement (both up() and down())
  • Does not reference models
  • Filename is in the correct format (and correctly ordered)
  • Targets the next minor version
  • All code paths have appropriate log messages
  • Uses the correct utils
  • Contains a minimal changeset
  • Does not mix DDL/DML operations

Schema changes

  • Both schema change and related migration have been implemented
  • For index changes: has been performance tested for large tables
  • For new tables/columns: fields use the appropriate predefined field lengths
  • For new tables/columns: field names follow the appropriate conventions
  • Does not drop a non-alpha table outside of a major version

Data changes

  • Mass updates/inserts are batched appropriately
  • Does not loop over large tables/datasets
  • Defends against missing or invalid data
  • For settings updates: follows the appropriate guidelines

@ronaldlangeveld ronaldlangeveld added migration [pull request] Includes migration for review and removed migration [pull request] Includes migration for review labels May 7, 2024
@ronaldlangeveld ronaldlangeveld requested a review from sagzy May 7, 2024 06:29
@ronaldlangeveld ronaldlangeveld marked this pull request as ready for review May 7, 2024 06:29
Copy link
Contributor

@sagzy sagzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍


if (maxTimestamp) {
if (!(maxTimestamp instanceof Date)) {
maxTimestamp = new Date(maxTimestamp);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe worth explaining why we need to convert to a Date here?

// SQLite returns a string instead of a Date

if (maxFailedAt && !(maxFailedAt instanceof Date)) {
// SQLite returns a string instead of a Date
maxFailedAt = new Date(maxFailedAt);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing we're keeping the old logic to smoothen the rollout, as existing emails won't have the latest_event_timestamp field. Shall we make a note to clean this up in a couple of weeks from now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also return early here, to separate the new logic from the old one, e.g.

if (maxTimestamp) {
    if (!(maxTimestamp instanceof Date)) {
        maxTimestamp = new Date(maxTimestamp);
     }
              
     return maxTimestamp;
}


// Legacy logic

...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes correct, I'll add a note in the comments.

@ronaldlangeveld ronaldlangeveld removed the migration [pull request] Includes migration for review label May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants