Ability to track costs without using langchain or openai

[agamm]

Feature request

I have a project using llm calls that are abstracted away. I'd love to be able to add costs metadata to the RunTree object with something like: tokencost manually.

Motivation

This is what I'm currently doing:

 rt = RunTree(
        run_type="llm",
        name=f"LLM Call - {model}",
        inputs={"messages": chat_messages},
        extra={
            "metadata": {
                "input_size": count_message_tokens(chat_messages, model),
                "input_price": calculate_prompt_cost(chat_messages, model),
                "output_size": count_string_tokens(response, model),
                "output_price": calculate_completion_cost(response, model),
            }
        },
    )
    rt.end(...)
    rt.post()

This doesn't aggregate the whole run costs or shows the data nicely either.

[hinthornw]

https://docs.smith.langchain.com/how_to_guides/tracing/log_llm_trace#manually-provide-token-counts

[agamm]

Yeah but how do I add that to the RunTree instance? I can't use the tracable decorator with the library that I'm using.

[hinthornw]

Also out of curiosity - what motivates using the run trees directly instead of the @traceable or trace helpers?

The output format is the same as in the docs. outputs would contain an openai-compatible format, inputs contain messages, and if you want to add custom usage, you add that to extra directly