New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
stepfunctions-tasks: emr-create-cluster - add support for AutoTerminationPolicy.IdleTimeout #29926
Labels
@aws-cdk/aws-emr
Related to Amazon EMR
@aws-cdk/aws-stepfunctions-tasks
effort/medium
Medium work item – several days of effort
feature-request
A feature should be added or improved.
p2
Comments
valentinlagunes
added
feature-request
A feature should be added or improved.
needs-triage
This issue or PR still needs to be triaged.
labels
Apr 22, 2024
valentinlagunes
changed the title
emr-create-cluster: add support for AutoTerminationPolicy.IdleTimeout
stepfunctions-tasks: emr-create-cluster - add support for AutoTerminationPolicy.IdleTimeout
Apr 22, 2024
Corresponding CloudFormation property: |
@nmussy I was just about to check that. Thank you! @valentinlagunes Yes we should support that if CFN has the relevant property. Are you interested to submit a PR for that? You can reach out to me on cdk.dev slack if you need me to help you get started. Cheers! |
pahud
added
p2
effort/medium
Medium work item – several days of effort
and removed
needs-triage
This issue or PR still needs to be triaged.
labels
Apr 23, 2024
1 task
This was referenced Apr 26, 2024
+1 |
+1 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
@aws-cdk/aws-emr
Related to Amazon EMR
@aws-cdk/aws-stepfunctions-tasks
effort/medium
Medium work item – several days of effort
feature-request
A feature should be added or improved.
p2
Describe the feature
This feature currently exists in the SDK RunJobFlow EMR API for creating EMR clusters.
"AutoTerminationPolicy: { "IdleTimeout": number }
This feature will terminate the cluster if it has been idle for a specified number of seconds.
Right now the CDK emrCreateCluster step function task is missing this option, but I believe it is possible to do if the entire step function is written in ASL JSON. There is a similar missing parameter issue for the ManagedScalingPolicy parameter as well.
Use Case
This feature is very useful for terminating clusters when no job has run in a while. Many times, I'll forget to terminate a cluster and this will cause a large bill. The auto termination policy helps prevent that. It is also useful for keeping a cluster available for a while if another job might be submitted later, instead of terminating it immediately and then having to create a new one. It also helps simplify my state machines by not having to add a terminateCluster step.
Proposed Solution
Add to the emrCreateCluster cdk construct. One workaround is by using ASL JSON directly with
DefinitionBody.fromString
.Other Information
Related Issue (Missing ManagedScalingPolicy)
Acknowledgements
CDK version used
2.138.0
Environment details (OS name and version, etc.)
MacOS
The text was updated successfully, but these errors were encountered: