You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, so I was training 345m GPT2 using your example scripts examples/pretrain_gpt.sh. The validation loss and PPL, however, keep going up, while the training loss decreases as expected.
My hyperparameters are shown here:
Hi, so I was training 345m GPT2 using your example scripts
examples/pretrain_gpt.sh
. The validation loss and PPL, however, keep going up, while the training loss decreases as expected.My hyperparameters are shown here:
Can anyone please tell me what is wrong? Should not the PPL decreases? Thanks!
The text was updated successfully, but these errors were encountered: