Adapter name for SFT trainer

[para-zhou]

Hi, I appreciate that the PPOTrainer is enabled to assign ''adapter_name'' to trainable adapters, would you please also enable SFTTrainer with this function?

[younesbelkada]

Hi @para-zhou
Thanks for the issue ! adapter_name is used on our AutoModelForxxx classes that are internal to TRL. In order to switch between adapters in SFTTrainer you should first create your PeftModelForxxx and call model.set_adapter(adapter_name)