How to get, Prompt, Output, and Assertion value into Python

[P-Sood]

I am using a Python script as my prompt, and I was wondering how I could get the final LLM response from my provider and the assertion response.

I am using the llm-rubric assert type, and I will probably keep it that way.

I've seen the "Provider" Python script, but I can't tell how to get the Assertion response back because I can't figure out how to get the test case's assertion value to compare the original outputs against.

[typpo]

Finding it a little hard to follow - can you give me an example of what you're trying to do?

[P-Sood]

Hi @typpo, I got caught up with something. What I am asking for is a way to get all 4:

1. original prompt,
2. the llm responses to that original prompt,
3. the test case,
4. and the llm-rubric assessment response comparing the test case to the original llm response

Having all 4 at once in Python would be cool, and I'm wondering if there is any way to achieve that, because I want to use all 4 into a different script.