New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
loss 下降到2.7就不下降了 #744
Comments
The loss seems large. I guess there are pseudo-negative examples in the training data (some samples in the negative sample list |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
你好,我在做一个QA的数据集的任务,其中query是question,pos是answer,neg是其他的选项,但是在finetune
的时候,loss下降到2.7就不再下降了,并且最后accuracy也大概只有20-30%,请问大佬知道这种事什么问题么?
脚本:torchrun --nproc_per_node 1 -m FlagEmbedding.baai_general_embedding.finetune.run --output_dir finetune_model --model_name_or_path BAAI/bge-small-en-v1.5 --train_data fine_tune_data_10.jsonl --learning_rate 1e-5 --bf16 --num_train_epochs 5 --per_device_train_batch_size 16 --dataloader_drop_last True --normlized True --temperature 0.02 --query_max_len 82 --passage_max_len 56 --train_group_size 10 --negatives_cross_device --logging_steps 10 --save_steps 10000 --query_instruction_for_retrieval ""
The text was updated successfully, but these errors were encountered: