LLama-v2-Text-Gen-API

This project aims to develop an API backend using Flask to support Text Generation Inference with Llama v2 Models

Results

Swagger-UI

Text Generation Endpoint

API Call and Response

Streaming API

Usage

Build APP Image

docker compose build

Get everything up and running

docker compose down && docker compose up -d

Have fun

Visit http://localhost:7860/swagger to access the Swagger API Docs.

Contributing

Installing pre-commit

Pre-commit is already part of this project dependencies. If you would like to install it as a standalone run:

pip install pre-commit

To activate pre-commit run the following commands:

• Install Git hooks:

pre-commit install

• Update current hooks:

pre-commit autoupdate

To test your installation of pre-commit run:

pre-commit run --all-files