EasieST

Easy-to-use SillyTavern Starter, based on Docker Compose.

Usage

clone

git clone https://github.com/moeru-ai/easiest.git
cd easiest

config

cp intel.docker-compose.yml docker-compose.yml # Intel oneAPI SYCL
# cp rocm.docker-compose.yml docker-compose.yml # AMD ROCm (TODO)
# cp cuda.docker-compose.yml docker-compose.yml # NVIDIA CUDA (TODO)
# cp vulkan.docker-compose.yml docker-compose.yml # Vulkan (TODO)

nano docker-compose.yml # edit config

up / down

sudo docker compose up -d
# podman compose up -d # if you use podman

sudo docker compose down
# podman compose down # if you use podman

FAQ

Recommended Models (2024-04-06)

If this section hasn't been updated in a long time, I recommend looking for a new model.

For GGUF format, I recommend Q5_K_M or Q4_K_M (and imatrix).

• 7B: Lewdiculous/KukulStanta-7B-GGUF-IQ-Imatrix
  • SillyTavern Presets:
    • Lewdicu-Context-3.0.2-eros.json => ./sillytavern/config/context
    • Lewdicu-Instruct-Alpaca-3.0.2-tentative.json => ./sillytavern/config/instruct
    • Lewdicu-Samplers-3.0.2.json => ./sillytavern/config/TextGen Settings
• 11B: mradermacher/Fimbulvetr-11B-v2-i1-GGUF
• 70B: mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF
  • SillyTavern Presets:
    • prompting-tips => ./sillytavern/config/context
    • instruct-formats => ./sillytavern/config/instruct
    • sampler-tips => ./sillytavern/config/TextGen Settings

Why llama.cpp?

llama.cpp provides the official docker image for Intel Arc Graphics.

I may change to ollama or koboldcpp later.

License

Unlicense