-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Atomic job store transactions #3993
Labels
comp/datastore
Related to job and execution store
Comments
This was referenced May 13, 2024
wdbaruni
added a commit
that referenced
this issue
May 17, 2024
This PR allows executing multiple job store write operations in a single atomic transaction. This done by optionally providing a transactional context to the operation, which will prevent automatic commit after each operation and delegates that control to the caller. ### Usage Examples #### A transaction per operation This example is for the same behaviour we have today where each operation is committed automatically, which will continue to work ``` ctx := context.Background() var err error // The job will be updated immediately if no error was returned if err = s.store.UpdateJobState(ctx, jobstore.UpdateJobStateRequest{}); err != nil { return err } // The execution will be updated immediately if no error was returned if err = s.store.UpdateExecution(ctx, jobstore.UpdateExecutionRequest{}); err != nil { return err } ``` #### A single transaction for both operations This is an example of the new behaviour where we can group multiple operations in a single atomic operation ``` var err error txCtx, err := s.store.BeginTx(context.Background()) if err != nil { return err } // make sure we rollback the transaction if there is an error defer func() { if err != nil { _ = txCtx.Rollback() } }() // updating the job won't be committed yet // Note we are passing the transaction context txCtx instead of a regular context if err = s.store.UpdateJobState(txCtx, jobstore.UpdateJobStateRequest{}); err != nil { return err } // updating the execution won't be committed yet if err = s.store.UpdateExecution(txCtx, jobstore.UpdateExecutionRequest{}); err != nil { return err } // commit both operations return txCtx.Commit() ``` Closes [3993](#3993)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Today we run job store operations in their isolation transaction, which prevents us from doing atomic transactions that span multiple operations, such as handling a single scheduler plan that can involve updating the job state, updating the state of multiple executions, enqueueing delayed evaluations and adding history events.
We need way to start a single transaction and have all these operations to be committed atomically or rolled-back
The text was updated successfully, but these errors were encountered: