fix(agenthub) : limit context size send to llm api

[coffeecupjapan]

What?

When observation becoming bigger like more than 16385 words like when developing environment, the LLM API (in my case chatGPT-3.5) throws error like below image.

I think If we can limit the context size send to LLM api, we don't have to worry about the above 400 error.
I did't try using Gemini or Vertex LLM, and even for GPT I didn't try using GPT4, so maybe there are improvement to have conditional branch that supports each LLM.

Thank you for reading my PR.

[iFurySt]

should we dense the messages or swap a portion to long-term memory instead of truncating them directly?

[rbren]

@xingyaoww I vaguely remember you doing something like this already

@coffeecupjapan we should at least extract out the magic constant, so we can tune this a bit

[rbren]

See also: #1421

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

❗ No coverage uploaded for pull request base (main@31c1a2d). Click here to learn what that means.

Files	Patch %	Lines
agenthub/codeact_agent/codeact_agent.py	0.00%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1485   +/-   ##
=======================================
  Coverage        ?   58.52%           
=======================================
  Files           ?       83           
  Lines           ?     3479           
  Branches        ?        0           
=======================================
  Hits            ?     2036           
  Misses          ?     1443           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[xingyaoww]

@rbren Yes! We already have this in-place in #1494

OpenDevin/agenthub/codeact_agent/codeact_agent.py

Lines 37 to 44 in 24750ba

	def truncate_observation(observation: str, max_chars: int=5000) -> str:
	"""
	Truncate the middle of the observation if it is too long.
	"""
	if len(observation) <= max_chars:
	return observation
	half = max_chars // 2
	return observation[:half] + '\n[... Observation truncated due to length ...]\n' + observation[-half:]

[rbren]

@coffeecupjapan I'm going to close this one, but feel free to open up an issue where we can discuss a more long-term solution!