Replies: 3 comments 2 replies
-
You can adapt our ControlNet training script and refer to the LoRA training scripts to do this. For example, after initializing he from peft import LoraConfig
lora_config = LoraConfig(
r=args.rank,
lora_alpha=args.rank,
init_lora_weights="gaussian",
target_modules=["to_k", "to_q", "to_v", "to_out.0"],
)
controlnet.add_adapter(lora_config) Does that help? |
Beta Was this translation helpful? Give feedback.
-
Hello @sayakpaul, I am trying to do a similar thing. I have managed to add the adapters with |
Beta Was this translation helpful? Give feedback.
-
I have tried to train a controlnet with lora, but after I merge the controlnet model and lora, the weight of controlnet seems not changed, and the output is sd, not controlnet |
Beta Was this translation helpful? Give feedback.
-
train full controlnet need much resource and time, so how to train controlnet with lora?
Beta Was this translation helpful? Give feedback.
All reactions