Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Streaming Support for Nvidia's Triton Integration #13135

Merged
merged 8 commits into from Apr 30, 2024
Merged

Streaming Support for Nvidia's Triton Integration #13135

merged 8 commits into from Apr 30, 2024

Commits on Apr 27, 2024

  1. Update base.py 

    Implemented Streaming capabilities for completion and chatting
    @Rohith-2
    Rohith-2 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    c312329 View commit details
    Browse the repository at this point in the history

  2. Update nvidia_triton.ipynb 

    Added examples for streaming the generation from LLM's complete functionality.
    @Rohith-2
    Rohith-2 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    a32f9c2 View commit details
    Browse the repository at this point in the history

  3. Update nvidia_triton.ipynb 

    Updated example to utilise the streaming capability of Triton with complete and chat functions.
    @Rohith-2
    Rohith-2 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    7b84550 View commit details
    Browse the repository at this point in the history

  4. Update nvidia_triton.ipynb 

    Updated example to utilise the streaming capability of Triton with complete and chat functions.
    @Rohith-2
    Rohith-2 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    4d89dfd View commit details
    Browse the repository at this point in the history

  5. Update pyproject.toml

    @Rohith-2
    Rohith-2 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    93bdef8 View commit details
    Browse the repository at this point in the history

  6. Added Tests

    @Rohith-2
    Rohith-2 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    2d2891a View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2024

  1. Update base.py 

    Reformatted with lint
    @Rohith-2
    Rohith-2 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    a0f62ee View commit details
    Browse the repository at this point in the history

  2. Merge branch 'run-llama:main' into patch-3

    @Rohith-2
    Rohith-2 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    4f8fabc View commit details
    Browse the repository at this point in the history