You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently have three specialized assistants that insert or edit code (insert-code-chunk, insert-code-expression, edit-code-chunk; there is no particular reason that edit-code-expression has not been written yet, it just hasn't).
We are in the process of improving these assistants by injecting a lot of contextual information into the system prompt (#2105). In addition to this, we know from our own and other's experience, that feeding back errors from executing code (e.g. non-existent libraries, or methods) can help a lot in producing correct code.
Implementing the basics of this should be fairly straightforward. In instruction_block.rs we can execute the CodeChunk in the suggestion and if there are any errors in its executionMessages add those to the messages of the InstructionBlock (perhaps with a prefix like "There was an error executing that code") and execute the instruction again.
The big issue with that is that code generated by AI assistants should be treated as potentially harmful (e.g unintentionally or maliciously (e.g. via prompt injection attacks) altering a dataframe in the kernel session, deleting files, exfiltrating data). So far we see two approaches to "reducing the blast radius" of executing untrusted, AI generated code:
"Forking" kernels i.e. creating a temporary clone of the kernel state so that code can be executed in that without affecting the state of the "main" kernel
"Sandboxing" kernels i.e. running kernels with restricted capability (e.g. only able to access certain dirs; read-only access to filesystem, no network access)
We already have Kernel::supports_forks() and KernelInstance::fork() methods which allow us to declare whether a kernel supports forks, and to create a fork, respectively. I propose to add two similar methods:
Kernel::supports_sandboxing(&self) -> KernelSandboxing where KernelSandboxing is a struct which specifies the types of sandboxing that a kernel supports (e.g. filesystem root, read/write, network access).
Kernel::create_sandbox(&self, sandbox: KernelSandbox) -> Result<Box<dyn KernelInstance>> where KernelSandbox is a struct specifying the sandbox e.g. which folders can be read, which can be written to
Most of the embedded kernels (e.g RhaiKernel, QuickJsKernel (not yet implemented but being considered)) are inherently sandboxed. The microkernels (e.g. PythonKernel, RKernel) which spawn an external process will require us to use nsjail on Linux and sandbox-exec on MacOS (at this stage I'm not aware of equivalent on Windows; users there may have to use Windows Subsystem for Linux).
Note that none of this prevents someone putting the stencila binary and Python etc in a Docker container or VM and isolating everything that way. This sandboxing is meant to provide some process isolation for local desktop use without having to run a Docker container or VM.
The text was updated successfully, but these errors were encountered:
Summary
We currently have three specialized assistants that insert or edit code (
insert-code-chunk
,insert-code-expression
,edit-code-chunk
; there is no particular reason thatedit-code-expression
has not been written yet, it just hasn't).We are in the process of improving these assistants by injecting a lot of contextual information into the system prompt (#2105). In addition to this, we know from our own and other's experience, that feeding back errors from executing code (e.g. non-existent libraries, or methods) can help a lot in producing correct code.
Implementing the basics of this should be fairly straightforward. In
instruction_block.rs
we can execute theCodeChunk
in thesuggestion
and if there are any errors in itsexecutionMessages
add those to themessages
of theInstructionBlock
(perhaps with a prefix like "There was an error executing that code") and execute the instruction again.The big issue with that is that code generated by AI assistants should be treated as potentially harmful (e.g unintentionally or maliciously (e.g. via prompt injection attacks) altering a dataframe in the kernel session, deleting files, exfiltrating data). So far we see two approaches to "reducing the blast radius" of executing untrusted, AI generated code:
We already have
Kernel::supports_forks()
andKernelInstance::fork()
methods which allow us to declare whether a kernel supports forks, and to create a fork, respectively. I propose to add two similar methods:Kernel::supports_sandboxing(&self) -> KernelSandboxing
whereKernelSandboxing
is a struct which specifies the types of sandboxing that a kernel supports (e.g. filesystem root, read/write, network access).Kernel::create_sandbox(&self, sandbox: KernelSandbox) -> Result<Box<dyn KernelInstance>>
whereKernelSandbox
is a struct specifying the sandbox e.g. which folders can be read, which can be written toMost of the embedded kernels (e.g
RhaiKernel
,QuickJsKernel
(not yet implemented but being considered)) are inherently sandboxed. The microkernels (e.g.PythonKernel
,RKernel
) which spawn an external process will require us to usensjail
on Linux andsandbox-exec
on MacOS (at this stage I'm not aware of equivalent on Windows; users there may have to use Windows Subsystem for Linux).Note that none of this prevents someone putting the
stencila
binary and Python etc in a Docker container or VM and isolating everything that way. This sandboxing is meant to provide some process isolation for local desktop use without having to run a Docker container or VM.The text was updated successfully, but these errors were encountered: