New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Warn about mutable objects in Request.cb_kwargs
when JOBDIR
is set
#6120
Comments
In addition, I would be good to note, that to any custom object which cannot or is not meant to be serializable one should add pickle-specific method that immediately throws an error an pickle attempt.
W/o Defining |
Summary
Any serializable object inside
Request.cb_kwargs
whenJOBDIR
is set, is not passed to request's callable. Instead, its deep copy is passed. This is the result of the serialization and deserialization process. This behavior should be noted in the documentation.Suggestion
As I suggested in #6119, the next note (or its variation) can added to jobs documentation:
Additionally, note can be added to
cb_kwargs
as well:Motivation
Making deep copies of
cb_kwargs
content whenJOBDIR
is set, is not obvious at a glance. EnablingJOBDIR
is easy, and one can set it without realization that previously working code will stop working.Additional context
In #6119 I also suggested that when
JOBDIR
is set, a warning can be emitted if there is a common mutable object (such as lists, dicts, sets, bytearrays and array from Pythonarray
module) insidecb_kwargs
values. However, now I think this is unnecessary and can lead to redundant warnings. Especially in cases when mutable object is passed, but there is no intent in mutating it inside request's callback; for example, dictionaries does not have immutable analogue andtypes.MappingProxyType
is not pickleable.The text was updated successfully, but these errors were encountered: