Release v0.3.0

Changes

• Upgraded the PyTorch NGC container to version 24.02 (@openllmai0, Xianyu)
• Upgraded DeepSpeed to version 0.14.0 (@openllmai0, Xianyu)
• Fixed the vLLM version to 0.4.2 (Xianyu)
• Cleaned up the codebase (@openllmai0, Xianyu)

Release v0.2.9

Changes

• Fixed OOM for --colocate_critic_reward and --colocate_actor_ref @openllmai0 (Xianyu)

Release v0.2.8

Changes

• Fixed DPO loss mask @openllmai0 (Xianyu)
• Fixed vLLM generation corner case @openllmai0 (Xianyu)
• Upgraded Ray and Transformers @openllmai0 (Xianyu)
• Fixed typos in README.md @KT313
• Added system prompt in datasets @hijkzzz

Release v0.2.7

Changes

• Added support for vLLM-v0.4.2 @hijkzzz
• Added support for Jamba-v0.1 (Incompatible with vLLM-v0.4.2 now) @hijkzzz
• Added LoRA configs (--lora_dropout, --target_modules) @hijkzzz

Release v0.2.6

Changes

• Upgraded vLLM to v0.4.1 @mgerstgrasser @wuxibin89 @hijkzzz
• Upgraded Transformers to v4.40.1 and DeepSpeed to v0.14.0 @hijkzzz
• Fixed typo in train_ppo_ray.py @mickelliu
• Fixed mismatch size output_state_dict(148) and state_dict(149) in model saving @hijkzzz
• Added support for --colocate_actor_ref and --colocate_critic_reward in train_ppo_ray.py @hijkzzz
• Added support for Ray PPO reward ref models offloading @hijkzzz

Release v0.2.5

Changes

• Added Chinese README.md @khazic
• Added KD Trainer and Loss @ifromeast
• Fixed num_training_steps @wuxibin89
• Updated requirements.txt @kfertakis
• Fixed error due to 'margin' variable type being list in rm_trainer.py @StwayneXG

Release v0.2.4

Changes

• Fixed DPO masked loss function @hijkzzz
• Fixed Yi-34B tokenizer (--disable_fast_tokenizer) #240 @hijkzzz
• Supported wandb.login() (--wandb True) #231 @mgerstgrasser

Release v0.2.3

Changes

• Fixed #191 "deepspeed.zero.Init causes very strange spikes in PPO policy_loss" @hijkzzz
• Added dockerfile for vLLM @hijkzzz

Release v0.2.2

Changes

• Fixed LlamaRotaryEmbedding for Transformers v4.38.1 @hijkzzz
• Use lazy vLLM engine @wuxibin89
• Added Chinese PR docs @catqaq
• Fixed tensor shape docs @Thecats-Jfm

Release v0.2.1

Changes

• Fixed position_ids for left padding #217 @hijkzzz
• Supported input_key for custom dataset @hijkzzz