We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
If you pass a large arg if fn_args or fn_kwargs, Ray stores a copy of that object in heap memory for each task. So, if Ray Data launches many tasks, this can cause heap OOMs. See Anti-pattern: Passing the same large argument by value repeatedly harms performance
fn_args
fn_kwargs
To avoid users shooting themselves in the foot with this, we should raise a warning.
UX
The text was updated successfully, but these errors were encountered:
bveeramani
No branches or pull requests
Description
If you pass a large arg if
fn_args
orfn_kwargs
, Ray stores a copy of that object in heap memory for each task. So, if Ray Data launches many tasks, this can cause heap OOMs. See Anti-pattern: Passing the same large argument by value repeatedly harms performanceTo avoid users shooting themselves in the foot with this, we should raise a warning.
Use case
UX
The text was updated successfully, but these errors were encountered: