A timeless take on benchmarking LLMs, just like grounding people after waking up from stasis in deep space
Similar to most interstellar sci-fi stories, when a crew aboard interstellar vehicles is awakened from stasis, they "have to" start vomiting and need grounding about their location, time, and the status of online/offline systems. I'm currently halfway through Alien: Cold Forge where this is a trope for the franchise
The same could be applicable to LLMs as well, if we are to treat them as cognition engines instead of being relegated as a bunch of stochastic parrots. This also means that benchmarks like GAIA & others need to be re-worked from a timeless being's perspective. Where all notion of current space-time, tools available, data sources, localization etc. is jettisoned & needs to be re-introduced/affirmed back to them.
This project also assumes English to somehow become the timeless language or atleast interpreted without any formal rules being shared between LLM & benchmark (unifying translation is for another project), thereby banking on the pre-training to have resolved this. Just like we bootstrap & learn basic phrases in a language before formally being taught the same.