Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Tool to display/play media #508

Open
shhlife opened this issue Dec 15, 2023 · 0 comments
Open

Add a Tool to display/play media #508

shhlife opened this issue Dec 15, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@shhlife
Copy link
Collaborator

shhlife commented Dec 15, 2023

Is your feature request related to a problem? Please describe.
When working with Image Generation Drivers (and eventually video, audio, and any other media), there's no built in way to display that media.

Describe the solution you'd like
I would like a tool that I can add to a Griptape Structure that will display the media. For example, I could have an agent generate an image, and then ask it to display the image and it would open it in an appropriate application on my computer. If it's an audio file, then it could play the audio file for me.

Describe alternatives you've considered
Because we're using Python, I have created my own tool to do this for displaying images, but it'd be better if it were a part of Griptape so others could do it, and if it were extended to handle all media types.

Additional context
An example of a ViewImage tool

from schema import Schema, Literal
from attr import define
import os, subprocess, sys

from griptape.artifacts import TextArtifact, ErrorArtifact
from griptape.utils.decorators import activity
from griptape.tools import BaseTool

def open_image(image_path):
    if sys.platform == "win32":
        os.startfile(image_path)
    elif sys.platform == "darwin":  # macOS
        subprocess.run(["open", image_path])
    else:  # linux variants
        subprocess.run(["xdg-open", image_path])


@define
class ViewImage(BaseTool):
    @activity(
        config={
            "description": "View the image.",
            "schema": Schema(
                {
                    Literal(
                        "filename", description="The filename of the image to view."
                    ): str,
                }
            ),
        }
    )
    def view_image(self, params: dict) -> TextArtifact | ErrorArtifact:
        try:
            filename = params["values"]["filename"]
            agent_response = ""

            # Make sure the path exists
            agent_response = os.path.exists(filename)
            if agent_response:
                open_image(filename)

            return TextArtifact(agent_response)

        except Exception as e:
            return ErrorArtifact(f"Error retrieving projects: {e}")
@shhlife shhlife added the enhancement New feature or request label Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant