You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just want to ask that for the pruned model, how can we deploy it using MLC-LLM? Since the qkv dimensions in each layer are different, the model is stored using torch.save rather than save_pretrained. So I'm a little confused about how to use MLC-LLM with this model? Could you please give me some tips or advice?
Thanks!
The text was updated successfully, but these errors were encountered:
❓ General Questions
Hi there,
I just want to ask that for the pruned model, how can we deploy it using MLC-LLM? Since the qkv dimensions in each layer are different, the model is stored using torch.save rather than save_pretrained. So I'm a little confused about how to use MLC-LLM with this model? Could you please give me some tips or advice?
Thanks!
The text was updated successfully, but these errors were encountered: