Question about answer ranking

[dhansmair]

Hi there, I see that in line

ALBEF/models/model_vqa.py

Line 198 in b9727e4

log_probs_sum = log_probs.sum(1)

you are using a sum to accumulate the loss for the tokens in the answer sequence. How does this behave if the possible answers have varying lengths? Shouldn't the loss be divided by the sequence length to get the average loss per token? Otherwise, won't the ranking be biased towards shorter sequences?

[LiJunnan1992]

Hi, sum of log_probs = log of the multiplication of probs = log of the sequence prob

[MLAlex1]

I think @dhansmair makes a good point - indeed I also think it will be biased if we do not divide by the length of each answer sequence.