Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No matter what I put: ValueError: The initializer token must be a single token. #270

Open
openSourcerer9000 opened this issue Apr 7, 2024 · 0 comments

Comments

@openSourcerer9000
Copy link

openSourcerer9000 commented Apr 7, 2024

So I'm trying to finetune an SD model using some images. I want to add the 2 keywords (tokens?) pmauritania and cafrica, letting the model associate all images with both, and using mauritania and africa as vector points to start from when learning these words. I want to be able to prompt my model with these and have it produce results associated with these initializer words, as well as the images I'm giving it. I added both of these keywords at the beginning of the image captions, which I wrote to .txt files in the same folder, with the same name as the corresponding jpgs:

image


SET MODEL_NAME="C:\Users\seanrm100\Desktop\Docs\Art\StableDiffuzion\finechune\vodka.pt"
SET INSTANCE_DIR="temp"
SET OUTPUT_DIR="loras"

lora_pti ^
  --pretrained_model_name_or_path=%MODEL_NAME%  ^
  --instance_data_dir=%INSTANCE_DIR% ^
  --output_dir=%OUTPUT_DIR% ^
  --train_text_encoder ^
  --resolution=512 ^
  --train_batch_size=1 ^
  --gradient_accumulation_steps=4 ^
  --scale_lr ^
  --learning_rate_unet=1e-4 ^
  --learning_rate_text=1e-5 ^
  --learning_rate_ti=5e-4 ^
  --color_jitter ^
  --lr_scheduler="linear" ^
  --lr_warmup_steps=0 ^
  --placeholder_tokens="cafrica|pmauritania" ^
  --initializer_tokens="africa|mauritania" ^
  --save_steps=100 ^
  --max_train_steps_ti=1000 ^
  --max_train_steps_tuning=1000 ^
  --perform_inversion=True ^
  --clip_ti_decay ^
  --weight_decay_ti=0.000 ^
  --weight_decay_lora=0.001^
  --continue_inversion ^
  --continue_inversion_lr=1e-4 ^
  --device="cuda:0" ^
  --lora_rank=1 ^
@REM  --use_face_segmentation_condition^
pause

I get

PTI : Placeholder Tokens ['<pmauritania>']
PTI : Initializer Tokens ['<mauritania>']
...
raise ValueError("The initializer token must be a single token.") ValueError: The initializer token must be a single token.

So I try using only 1 token
--placeholder_tokens="pmauritania" ^
--initializer_tokens="mauritania" ^

same thing.

--placeholder_tokens="" ^
--initializer_tokens="mauritania" ^
and
--placeholder_tokens="" ^
--initializer_tokens="" ^

same thing. Am I understanding this usage correctly? Documentation is lacking on how to caption the images, how exactly to use these args, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant