limit / decrease tokens fed to model for each query

[JonasDoesThings]

Hi, currently evaluating danswer. Is there currently a way to limit the input-tokens fed into the LLM to decrease cost per query?

I already searched GitHub and saw mentions of an env var called NUM_DOCUMENT_TOKENS_FED_TO_GENERATIVE_MODEL, but apparently that got removed? I couldn't find it in danswer's code anymore.

I tried to tweak-down GEN_AI_MAX_TOKENS but that lead to errors.

So is there currently some way to decrease the tokens spent on a query by reducing the amount of context being fed into the model? Maybe some limit I can tweak-down regarding the selection of relevant chunks?

Thanks in advance and greetings from Austria :)