-
Notifications
You must be signed in to change notification settings - Fork 149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Caching of previous_state non-compatible with Multi-Agent #80
Comments
Yes, everything you said is absolutely correct
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I discovered while digging through the code that a certain state value called
previous_state
of the DQN algorithm (and possibly some others) is being cached on theact()
andaction_distribution
methods of the class.From the little digging that I did, it seems to be related to the side-panel of the rendering, which showcases extra information about the attention heads of the controller vehicles.
Only that, when there are more than one controller vehicles, it seems to be redefined n+1 times, where n is the number of vehicles, during each
act()
call: once as the tuple of observations of all agents, and once as the observation of each agent, until it gets redefined as the observation of the last controlled vehicle.Snippet from
rl_agents/agents/deep_q_network/abstract.py
:It does not seem like the most pressing issue, but I am just putting it here, in case anyone has a decent idea on how to deal with this. Or for a clearer explanation as to why this variable is important, as I only gave one example of its usefulness.
Thanks!
The text was updated successfully, but these errors were encountered: