v0.1.8

Installation

pip install openllm==0.1.8

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.8

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Find more information about this release in the CHANGELOG.md

Full Changelog: v0.1.7...v0.1.8

v0.1.7

Features

OpenLLM now seamlessly integrates with HuggingFace Agents. Replace the HfAgent endpoint with a running remote server.

import transformers

agent = transformers.HfAgent("http://localhost:3000/hf/agent")  # URL that runs the OpenLLM server

agent.run("Is the following `text` positive or negative?", text="I don't like how this models is generate inputs")

Note
only starcoder is currently supported for agent feature.

To use it from the openllm.client, do:

import openllm

client = openllm.client.HTTPClient("http://123.23.21.1:3000/")

client.ask_agent(
    task="Is the following `text` positive or negative?",
    text="What are you thinking about?",
)

Installation

pip install openllm==0.1.7

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.7

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Find more information about this release in the CHANGELOG.md

What's Changed

• integration: HfAgent by @aarnphm in #29
• fix(tools): adhere to style guidelines by @aarnphm in #31

Full Changelog: v0.1.6...v0.1.7

v0.1.6

Features

Quantization now can be enabled during serving time:

openllm start stablelm --quantize int8

This will loads the model in 8-bit mode, with bitsandbytes

For CPU machine, don't worry, you can use --bettertransformer instead:

openllm start stablelm --bettertransformer

Roadmap

• GPTQ is being developed, will include support soon

Installation

pip install openllm==0.1.6

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.6

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Find more information about this release in the CHANGELOG.md

What's Changed

• refactor: toplevel CLI by @aarnphm in #26
• docs: add LangChain and BentoML Examples by @parano in #25
• feat: fine-tuning [part 1] by @aarnphm in #23
• feat: quantization by @aarnphm in #27
• perf: build quantization and better transformer behaviour by @aarnphm in #28

Full Changelog: v0.1.5...v0.1.6

v0.1.5

Installation

pip install openllm==0.1.5

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.5

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Full Changelog: v0.1.4...v0.1.5

v0.1.4

Installation

pip install openllm==0.1.4

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.4

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Full Changelog: v0.1.3...v0.1.4

v0.1.3

Installation

pip install openllm==0.1.3

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.3

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Full Changelog: v0.1.2...v0.1.3

v0.1.2

Installation

pip install openllm==0.1.2

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.2

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

What's Changed

• chore: metadata by @aarnphm in #19

Full Changelog: v0.1.1...v0.1.2

v0.1.1

🎉 Hello world, OpenLLM

OpenLLM version 0.1.1 brings initial support for SOTA LLMs (more to come!!):

Model	CPU	GPU	Installation	Model Ids
flan-t5	✅	✅	pip install "openllm[flan-t5]"	google/flan-t5-small google/flan-t5-base google/flan-t5-large google/flan-t5-xl google/flan-t5-xxl
dolly-v2	✅	✅	pip install openllm	databricks/dolly-v2-3b databricks/dolly-v2-7b databricks/dolly-v2-12b
chatglm	❌	✅	pip install "openllm[chatglm]"	thudm/chatglm-6b thudm/chatglm-6b-int8 thudm/chatglm-6b-int4
starcoder	❌	✅	pip install "openllm[starcoder]"	bigcode/starcoder bigcode/starcoderbase
falcon	❌	✅	pip install "openllm[falcon]"	tiiuae/falcon-7b tiiuae/falcon-40b tiiuae/falcon-7b-instruct tiiuae/falcon-40b-instruct
stablelm	✅	✅	pip install openllm	stabilityai/stablelm-tuned-alpha-3b stabilityai/stablelm-tuned-alpha-7b stabilityai/stablelm-base-alpha-3b stabilityai/stablelm-base-alpha-7b

Quickly startup falcon locally, with openllm start falcon:

openllm start falcon

Easily bundle this LLM into Bento, a portable format that can be easily deployed everywhere:

openllm build falcon

Refers to the README.md for more details

Installation

pip install openllm==0.1.1

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.1

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Full Changelog: v0.1.0...v0.1.1

v0.1.0

Installation

pip install openllm==0.1.0

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.0

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

What's Changed

• docs: add --model-id example command by @parano in #18
• tests: general testing by @aarnphm in #17

Full Changelog: v0.0.34...v0.1.0

v0.0.34

Installation

pip install openllm==0.0.34

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.0.34

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Full Changelog: v0.0.33...v0.0.34