Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] New notebook for detecting python code vulnerabilities using GPT4 #1100

Open
mhbuehler opened this issue Mar 13, 2024 · 2 comments

Comments

@mhbuehler
Copy link

Is there any interest in a notebook contribution that demonstrates a few ways of prompting GPT4 to detect insecure code snippets? The solution would include prompt templates that use techniques like few-shot learning, KNN-based few-shot learning (from this paper), and asking for a code fix (proposed by this paper) to increase prediction accuracy.

Is your feature request related to a problem? Please describe.
This is a proposal for a new use case. It would help users who are looking for examples that:

  • Use the OpenAI API to perform binary classification
  • Apply the above prompt engineering techniques and evaluate their impacts
  • Identify and/or correct software vulnerabilities

Describe the solution you'd like
I've written a draft notebook based on my experiments and would be happy to submit a PR. I'm an experienced software engineer and have recently been applying AI to topics in software security.

Additional context
There would be a new data file based on the published dataset from this paper which has this license. The notebook would cite all sources and include some evaluation metrics. Thanks for your consideration.

Copy link

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added the Stale label May 12, 2024
@mhbuehler
Copy link
Author

Commenting so this issue stays open. Can maintainers take a look at the notebook in #1112 and let me know if it can be merged or not? Thank you.

@github-actions github-actions bot removed the Stale label May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant