Issue with HuggingFaceHubExperiment: Unable to Use Models Other Than gpt2

[shrijayan]

⁉️ Discussion/Question

I'm currently facing an issue while attempting to use the HuggingFaceHubExperiment for testing models from Hugging Face. Specifically, it seems that only the gpt2 model is functional in this experiment, while other models from Hugging Face do not work. I am trying to use other models available from HuggingFace.

I've tried inputting Model IDs from Hugging Face, and I've also explored the available models on https://huggingface.co/api/models. Unfortunately, none of the models, aside from gpt2, seem to be compatible with the HuggingFaceHubExperiment.

Is there a specific reason for this limitation, or am I overlooking something in the setup process? I would greatly appreciate any guidance or insights you could provide to help resolve this issue.

[steventkrawczyk]

Thanks for bringing this issue up, I'm looking into it

[shrijayan]

Thanks for bringing this issue up, I'm looking into it

Thanks for looking up!

[shrijayan]

Thanks for bringing this issue up, I'm looking into it

Are there other possibilities to use all other huggingface models?

[shrijayan]

@steventkrawczyk I think its all because of the endpoint where it looks!

[steventkrawczyk]

Which model in particular are you trying to use? The huggingface hub experiment uses the Inference API, so it will only support models which that API supports. Is there a model you can call through the inference API, but are running into problems experimenting with prompttools? Can you post your error here?

[shrijayan]

Which model in particular are you trying to use? The huggingface hub experiment uses the Inference API, so it will only support models which that API supports. Is there a model you can call through the inference API, but are running into problems experimenting with prompttools? Can you post your error here?

I have also searched on https://huggingface.co/api/models, and it seems that the model present there, including GPT-2, does not support my requirements. I am certain that GPT-2 fetches the model from this link. However, any other model from the same link, such as GPT-2 Large, is also not working.