In MultiHeadAttention
, let num_heads=1
by default
#3833
Labels
Priority: P2 - no schedule
Best effort response and resolution. We have no plan to work on this at the moment.
MultiHeadAttention
, let num_heads=1
by default
#3833
Minor suggestion: In
nn.MultiHeadAttention
, letnum_heads=1
by default, which yields single-head attention. This seems like the most natural default value. I can submit a PR for this.The text was updated successfully, but these errors were encountered: