SecGPT - LlamaIndex Integration #13127

Yuhao-W · 2024-04-26T16:23:28Z

Description

SecGPT is an LLM-based system that secures the execution of LLM apps via isolation. The key idea behind SecGPT is to isolate the execution of apps and to allow interaction between apps and the system only through well-defined interfaces with user permission. SecGPT can defend against multiple types of attacks, including app compromise, data stealing, inadvertent data exposure, and uncontrolled system alteration. We develop SecGPT using LlamaIndex because it supports several LLMs and apps and can be easily extended to include additional LLMs and apps. We implement SecGPT as a personal assistant chatbot, which the users can communicate with using text messages.

New Package?

Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?

Yes
[x ] No

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

Yes
[x ] No

Type of Change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
[x ] New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Added new unit/integration tests
[x ] Added new notebook (that tests end-to-end)
I stared at the code and made sure it makes sense

Suggested Checklist:

[x ] I have performed a self-review of my own code
[x ] I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added Google Colab support for the newly added notebooks.
[x ] My changes generate no new warnings
[x ] I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
I ran make format; make lint to appease the lint gods

review-notebook-app · 2024-04-26T16:23:34Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

nerdai

Hey @Yuhao-W,

I notice that we haven't used the standard process for creating a llama pack here. Could you follow the instructions at the link provided to get in the standard format? In particular, we use poetry for package dep manager as well as for building our python packages. For convenience we have a cli tool that helps you created these packs:

https://github.com/run-llama/llama_index/blob/main/CONTRIBUTING.md#2--contribute-a-pack-reader-tool-or-dataset-formerly-from-llama-hub

nerdai · 2024-04-29T15:04:04Z

@Yuhao-W submitted a PR to your fork/main branch. It brings in the necessary pants build files to pass our checks.

llm-platform-security#1

…ld-files add pants build files

nerdai · 2024-04-29T18:13:49Z

@Yuhao-W looks like lint/fmt checks are failing. Can you please run:

make lint and make format then commit and push?

Yuhao-W · 2024-04-29T20:59:50Z

@Yuhao-W looks like lint/fmt checks are failing. Can you please run:

make lint and make format then commit and push?

@nerdai Thanks, Andrei. I just fixed this.

Yuhao-W · 2024-05-01T17:22:39Z

@nerdai Hi, Andrei. I see that some checks failed. Is there anything that needs to be changed?

nerdai · 2024-05-02T04:54:00Z

@nerdai Hi, Andrei. I see that some checks failed. Is there anything that needs to be changed?

Hey @Yuhao-W sorry for the troubles. I took a look at the logs and couldn't find anything. Tagging @logan-markewich who is quite good at figuring out this stuff when it seems like all is lost. lol

Yuhao-W · 2024-05-03T19:41:49Z

I think I figured out the errors and updated the package by mainly including dependency information in the pyproject.toml file under our package path. I also set up a unit test environment and ran it locally. The unit test was passed on my end. @nerdai and/or @logan-markewich, would appreciate if you can review the changes!

nerdai · 2024-05-06T19:25:27Z

I think I figured out the errors and updated the package by mainly including dependency information in the pyproject.toml file under our package path. I also set up a unit test environment and ran it locally. The unit test was passed on my end. @nerdai and/or @logan-markewich, would appreciate if you can review the changes!

thanks, lets run the checks and see what happens!

Yuhao-W · 2024-05-06T20:25:59Z

I think I figured out the errors and updated the package by mainly including dependency information in the pyproject.toml file under our package path. I also set up a unit test environment and ran it locally. The unit test was passed on my end. @nerdai and/or @logan-markewich, would appreciate if you can review the changes!

thanks, lets run the checks and see what happens!

Thanks @andrei, it failed again : ( this time because the requirements.txt was named requirements.tx

I have made a new commit. Not sure but we may still see errors after, would appreciate a deeper look if it fails. Thank you!

nerdai · 2024-05-07T16:23:45Z

@logan-markewich we're still running into some errors here. Perhaps we need to add a dependency in pants? This is the error we're seeing in the tests:

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/sandbox.py:6: in <module>
    import tldextract
E   ModuleNotFoundError: No module named 'tldextract'

But tldextract is indeed included in the pyproject.toml as a dep for the project.

(CC @Yuhao-W)

fix tests

nerdai · 2024-05-11T22:36:52Z

@Yuhao-W got checks to pass 🥳. Needed to remove the requirements.txt file as it was tripping up pants having deps listed in both requirements.txt and pyproject.toml

nerdai · 2024-05-11T23:05:40Z

llama-index-packs/llama-index-packs-secgpt/examples/SecGPT.ipynb

@@ -0,0 +1,741 @@
+{


I'm a bit confused as to how these tools actually provide the fare price of both ride-sharing apps? Is this purely for illustration? In other words, is this notebook not actually functional?

Reply via ReviewNB

nerdai · 2024-05-11T23:05:41Z

llama-index-packs/llama-index-packs-secgpt/examples/SecGPT.ipynb

@@ -0,0 +1,741 @@
+{


Line #4. # A benign ride-sharing app - quick_ride
this one isn't benign is it?

Reply via ReviewNB

nerdai · 2024-05-11T23:05:41Z

llama-index-packs/llama-index-packs-secgpt/examples/SecGPT.ipynb

@@ -0,0 +1,741 @@
+{


Line #20. # A malicious ride-sharing app - metro hail
I think it was mentioned in the writeup before this cell block that QuickRide was the malicious app.

Reply via ReviewNB

nerdai · 2024-05-11T23:05:41Z

llama-index-packs/llama-index-packs-secgpt/examples/SecGPT.ipynb

@@ -0,0 +1,741 @@
+{


Is there a way to show the case when not using SecGPT or having these measures turned off? In other words, can we see the case when the attack is successful?

Reply via ReviewNB

nerdai

@Yuhao-W Thanks for this contribution! I'm really excited about this :)

I left some comments on your PR. As another blanket comment, I do think your pack would greatly improve if you were able to include some doc/class strings throughout your code (i.e., quick descriptions of funcs/classes and its params/args).

nerdai · 2024-05-11T23:07:19Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/planner.py

+from llama_index.core import ChatPromptTemplate
+
+
+class HubPlanner:


Since there is a prompt here, I think we should subclass PromptMixin:

https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/prompts/mixin.py

nerdai · 2024-05-11T23:08:18Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/planner.py

+ lc_output_parser = JsonOutputParser()
+ self.output_parser = LangchainOutputParser(lc_output_parser)
+
+ self.query_engine = QueryPipeline(


this may be a bit of a name clash with llama-index ecosystem. As this is not really a QueryEngine but rather a QueryPipeline. If possible, would suggest using a different name: query_pipeline.

nerdai · 2024-05-11T23:09:51Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/message.py

+
+
+class Message:
+ def function_probe_request(self, spoke_id, function):


suggestion: maybe should this be a staticmethod?

similarly for all other funcs?

nerdai · 2024-05-11T23:11:34Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/message.py

Out of curiosity: have you considered using Pydantic BaseModel to represent Message? You can then subclass a BaseMessage. Pydantic can be helpful for validaton.

nerdai · 2024-05-11T23:12:29Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/spoke.py

+ verbose=verbose,
+ )
+
+ def chat(


I think since this is a light wrapper on our React class, I think you can support stream_chat and its async versions as well.

nerdai · 2024-05-11T23:13:41Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/spoke.py

+from llama_index.core.tools import FunctionTool
+
+
+def add_numbers(x: int, y: int) -> int:
+ """
+ Adds the two numbers together and returns the result.
+ """
+ return x + y
+
+
+if __name__ == "__main__":
+ llm = OpenAI(model="gpt-4-turbo", temperature=0.0, additional_kwargs={"seed": 0})
+ function_tool = FunctionTool.from_defaults(fn=add_numbers)
+ print(function_tool.metadata)
+ print(function_tool.metadata.get_parameters_dict())
+ spoke = Spoke(
+ tools=[function_tool],
+ collab_functions=["send_email", "draft_email", "read_email"],
+ llm=llm,
+ verbose=True,
+ )
+ spoke.chat("send a email to [email protected], subject: hello, body: hello world")


This looks like it was used perhaps for testing while developing? I would suggest converting this into an acutal unit test and using mocking of LLMs.

For inspiration, see here: https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/agent/llama-index-agent-introspective/tests/test_self_reflection.py

nerdai · 2024-05-11T23:14:21Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/spoke_operator.py

+ # Format and send the app request message to the hub
+ def make_request(self, functionality: str, request: dict):
+ # format the app request message
+ app_request_message = Message().app_request(


if these are staticmethods then you should be able to do Message.app_request(...) instead.

nerdai · 2024-05-11T23:16:55Z

llama-index-packs/llama-index-packs-secgpt/pyproject.toml

+HubOperator = "Yuhao-W"
+HubPlanner = "Yuhao-W"
+Message = "Yuhao-W"
+Socket = "Yuhao-W"
+Spoke = "Yuhao-W"
+SpokeOperator = "Yuhao-W"
+SpokeOutputParser = "Yuhao-W"
+TIMEOUT = "Yuhao-W"
+ToolImporter = "Yuhao-W"
+VanillaSpoke = "Yuhao-W"
+create_function_placeholder = "Yuhao-W"
+create_message_spoke_tool = "Yuhao-W"
+drop_perms = "Yuhao-W"
+get_user_consent = "Yuhao-W"
+set_mem_limit = "Yuhao-W"


all of these will show up in llamahub.ai which i don't think is ideal. Probably only makes sense to have the main one be discoverable i.e., Hub.

nerdai · 2024-05-11T23:19:00Z

llama-index-packs/llama-index-packs-secgpt/llama_index/packs/secgpt/hub.py

+from .hub_operator import HubOperator
+
+
+class Hub(BaseLlamaPack):


The naming convention for packs is typically XXXPack. I would suggest Renaming this to:

SecGPTPack and then just create the alias for Hub i.e., add the below to the end of this file.

Hub = SecGPTPack

Yuhao-W · 2024-05-12T00:01:34Z

@nerdai Thanks for the feedback on the PR! I will address your comments and get back to you with an update soon.

Include SecGPT

adbce3f

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Apr 26, 2024

nerdai self-requested a review April 27, 2024 05:33

nerdai requested changes Apr 27, 2024

View reviewed changes

Yuhao-W and others added 2 commits April 29, 2024 01:11

Update SecGPT pack

cec0733

add build files

60b96de

Merge pull request #1 from llm-platform-security/nerdai/add-pants-bui…

a8da8a1

…ld-files add pants build files

make SecGPT

eb97e7b

Merge branch 'run-llama:main' into main

c86f4ae

Yuhao-W and others added 3 commits May 2, 2024 12:44

Include unit test for SecGPT

fca7c87

Merge branch 'run-llama:main' into main

f9e4870

Update SecGPT package

0e85f72

Yuhao-W and others added 2 commits May 6, 2024 15:21

Merge branch 'run-llama:main' into main

673d001

fix typo

68b1d7a

nerdai and others added 2 commits May 11, 2024 18:18

fix tests

fad838a

Merge pull request #2 from llm-platform-security/fix-tests

7e529dc

fix tests

nerdai reviewed May 11, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SecGPT - LlamaIndex Integration #13127

SecGPT - LlamaIndex Integration #13127

Yuhao-W commented Apr 26, 2024 •

edited

review-notebook-app bot commented Apr 26, 2024

nerdai left a comment

nerdai commented Apr 29, 2024

nerdai commented Apr 29, 2024

Yuhao-W commented Apr 29, 2024

Yuhao-W commented May 1, 2024

nerdai commented May 2, 2024

Yuhao-W commented May 3, 2024

nerdai commented May 6, 2024

Yuhao-W commented May 6, 2024

nerdai commented May 7, 2024

nerdai commented May 11, 2024

nerdai May 11, 2024 •

edited

nerdai May 11, 2024 •

edited

nerdai May 11, 2024 •

edited

nerdai May 11, 2024 •

edited

nerdai left a comment

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

nerdai May 11, 2024

Yuhao-W commented May 12, 2024

		from llama_index.core import ChatPromptTemplate


		class HubPlanner:



		class Message:
		def function_probe_request(self, spoke_id, function):

		from .hub_operator import HubOperator


		class Hub(BaseLlamaPack):

SecGPT - LlamaIndex Integration #13127

Are you sure you want to change the base?

SecGPT - LlamaIndex Integration #13127

Conversation

Yuhao-W commented Apr 26, 2024 • edited

Description

New Package?

Version Bump?

Type of Change

How Has This Been Tested?

Suggested Checklist:

review-notebook-app bot commented Apr 26, 2024

nerdai left a comment

Choose a reason for hiding this comment

nerdai commented Apr 29, 2024

nerdai commented Apr 29, 2024

Yuhao-W commented Apr 29, 2024

Yuhao-W commented May 1, 2024

nerdai commented May 2, 2024

Yuhao-W commented May 3, 2024

nerdai commented May 6, 2024

Yuhao-W commented May 6, 2024

nerdai commented May 7, 2024

nerdai commented May 11, 2024

nerdai May 11, 2024 • edited

Choose a reason for hiding this comment

nerdai May 11, 2024 • edited

Choose a reason for hiding this comment

nerdai May 11, 2024 • edited

Choose a reason for hiding this comment

nerdai May 11, 2024 • edited

Choose a reason for hiding this comment

nerdai left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Yuhao-W commented May 12, 2024

Yuhao-W commented Apr 26, 2024 •

edited

nerdai May 11, 2024 •

edited

nerdai May 11, 2024 •

edited

nerdai May 11, 2024 •

edited

nerdai May 11, 2024 •

edited