How to run InstructBLIP with other LLM model

[zhangzitenga]

I modified llm_model in blip2_instructBLIP_vicuna7b.yaml to llama-7b model, which originated from https://huggingface.co/baffo32/decapoda-research-llama-7B-hf, but the inference result was null when I rerun InstructBlip, and there will be no error in the middle. If I use vicuna-7b-v1.1, I can reason normally. Does anyone know why? Thanks.