What was the intention of image_path in the model files such as gpt.py?

[mattmazzola]

The get_response function takes image_path but the variable is unused.

I assumed it would be useful if targeting another LMM like GPT4V; however, the code to set the image path as a user message is not there so I wanted to confirm.

What was the intention of the image_path variable in the models?

I see the image_path is actually used for Bard,

MathVista/models/bard.py

Lines 28 to 36 in 82f68d0

	def get_response(self, image_path, input_text):
	patience = self.patience
	while patience > 0:
	patience -= 1
	# print(f"Patience: {patience}")
	try:
	assert os.path.exists(image_path)
	image = open(image_path, 'rb').read() # (jpeg, png, webp) are supported.
	response = self.model.ask_about_image(input_text, image)

but not used for Claude or GPT
Is this an oversight and it should be used or was this intentional simply to keep the signature of functions the same?

MathVista/models/claude.py

Lines 17 to 39 in 82f68d0

	def get_response(self, image_path, user_prompt):
	patience = self.patience
	while patience > 0:
	patience -= 1
	try:
	# configure the default for all requests:
	anthropic = Anthropic(
	max_retries=0,
	api_key=self.api_key,
	)
	# update prompt
	if "python" in user_prompt:
	_HUMAN_PROMPT = HUMAN_PROMPT + "Generate the runnable python code only."
	else:
	_HUMAN_PROMPT = HUMAN_PROMPT
	# configure per-request options
	completion = anthropic.with_options(max_retries=5).completions.create(
	prompt=f"{_HUMAN_PROMPT} {user_prompt}{AI_PROMPT}",
	max_tokens_to_sample=self.max_tokens,
	model=self.model,
	)

MathVista/models/gpt.py

Lines 16 to 40 in 82f68d0

	def get_response(self, image_path, user_prompt):
	patience = self.patience
	max_tokens = self.max_tokens
	messages = [
	{"role": "user", "content": user_prompt},
	]
	while patience > 0:
	patience -= 1
	try:
	# print("self.model", self.model)
	response = openai.ChatCompletion.create(model=self.model,
	messages=messages,
	api_key=self.api_key,
	temperature=self.temperature,
	max_tokens=max_tokens,
	n=self.n
	)
	if self.n == 1:
	prediction = response['choices'][0]['message']['content'].strip()
	if prediction != "" and prediction != None:
	return prediction
	else:
	prediction = [choice['message']['content'].strip() for choice in response['choices']]
	if prediction[0] != "" and prediction[0] != None:
	return prediction