Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Context is not preserved after execution returns from deferred state #39503

Closed
1 of 2 tasks
mxuan0 opened this issue May 8, 2024 · 2 comments · Fixed by #39671
Closed
1 of 2 tasks

Context is not preserved after execution returns from deferred state #39503

mxuan0 opened this issue May 8, 2024 · 2 comments · Fixed by #39671
Labels
good first issue kind:bug This is a clearly a bug pending-response provider:amazon-aws AWS/Amazon - related issues

Comments

@mxuan0
Copy link

mxuan0 commented May 8, 2024

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.7.2

What happened?

We are using 8.7.1. version of airflow provider package and set deferrable to True when using SageMakerTransformOperator. We set define the model name with something like "project-dag_name-date-uuid", where uuid is to deduplicate model names upon retry. However, when the execution gets back from deferred state, the original uuid is not preserved and there would be a new uuid so the operator cannot find a model with the new model name and it would fail.

What you think should happen instead?

The original UUID should be preserved

How to reproduce

In the operator config, set "ModelName": f"{name}-{str(uuid4())[:8]}". The run the DAG.

Operating System

amazon Linux AMI

Versions of Apache Airflow Providers

8.7.1

Deployment

Amazon (AWS) MWAA

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@mxuan0 mxuan0 added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels May 8, 2024
@rawwar rawwar added provider:amazon-aws AWS/Amazon - related issues and removed area:core labels May 9, 2024
@potiuk potiuk added pending-response good first issue and removed needs-triage label for new issues that we didn't triage yet labels May 9, 2024
@potiuk
Copy link
Member

potiuk commented May 9, 2024

Can you check if it's the same in latest provider please @mxuan0 ?

@vincbeck
Copy link
Contributor

Thanks @mxuan0 for posting the issue, it is indeed a bug, here is a fix: #39671

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue kind:bug This is a clearly a bug pending-response provider:amazon-aws AWS/Amazon - related issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants