Some issues regarding attention mechanisms

[yshichseu]

Dear author, thank you for providing such a lightweight reinforcement learning library. Currently, I am hoping to integrate your attention mechanism into other reinforcement learning algorithms. I encountered an issue while modifying the network. Through the configuration file of the rlagent, I noticed that due to out being set to None, the last layer of the egoattionnetwork output has a dimension of 64 (if I overlooked it, please forgive me!), but the dimension of the action space is 3. How do these two establish a mapping relationship? I used a linear layer to implement it, but it seems to be ineffective. Therefore, I need to deepen my understanding of how the attention mechanism works. I should set up an attention network between observation and reinforcement learning algorithms for feature extraction. Should the final output of the attention network be the corresponding value of the action? Thank you for your answer! Thank you!

[eleurent]

Hi,

This function gets called when instantiating the network with the knowledge of the env and its observation and action spaces, which are used to set the input and output sizes:

https://github.com/eleurent/rl-agents/blob/84df15ea977271e6a4d015f10f9f355f7e866890/rl_agents/agents/common/models.py#L404C5-L404C22