New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] code review found 17 items only commented 3 on PR #328
Comments
I suggest generating a github artifact If there are more than 3, the action could generate the artifact, and add a comment: More comments are available in your github artifactsPlease follow these steps to view the artifact:
|
Sounds like a great addition. When I built this I wanted to reduce the cognitive load on a human reviewer hence the whole ranking and limiting to 3 comments. Happy to generate an artifacts file that makes a lot of sense. |
Current Behavior:
I added LangSmith tracing to my github action.
In looking over the traces outputs for my large PR, I see 17 comments in 5 or 6 groups. However the Summary chain to generate emojis) only has 3 of these comments as input. These are the ones that go to the PR comments. The other 15 seem to be discarded.
Expected Behavior:
The PR comments should contain ALL the Risk items found in code review. Perhaps the emoji summarizer needs to randomly select the comments to add, or add a finite number of comments prioritized by risk level.
Steps To Reproduce:
Hmm. Tough. Big PR. Check the tracing.
Suggestions on the root cause
I am guessing the root cause is the finite context window for the summary. You can still limit the comments used in the summary, but add ALL comments to the PR.
The text was updated successfully, but these errors were encountered: