Could not initialize the JudgeOpenAI model and openi import error

[lewtun]

FYI I'm seeing a lot of these "errors" in my logs when running lighteval

Could not initialize the JudgeOpenAI model:
[Errno 2] No such file or directory: 'src/lighteval/tasks/extended/mt_bench/judge_prompts.jsonl'

This doesn't cause an exception, but maybe good to clean up from the logs?

I also learned that you need the openai dep even if you don't run mt_bench due to an import error from this line:

lighteval/src/lighteval/metrics/llm_as_judge.py

Line 30 in fc428f6

from openai import OpenAI

I would recommend either promoting the openai dep to a core dep or having a checker like is_openai_available() then only importing the method if it is

[NathanHB]

thanks for the catch ! Did not pay attention to those on my local install, I will solve this asap :)