Creating a formula for "whisper.cpp"

[slhck]

Output of brew config

HOMEBREW_VERSION: 4.0.4-118-g10dd6a3
ORIGIN: https://github.com/Homebrew/brew
HEAD: 10dd6a30cac4b6ddc77f46963ad3c9ce14957fc3
Last commit: 19 hours ago
Core tap origin: https://github.com/Homebrew/homebrew-core
Core tap HEAD: f8ba616a4ef7661ff2116414bc1973dce49279f5
Core tap last commit: 5 weeks ago
Core tap branch: master
Core tap JSON: 02 Mar 13:09 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.1JfHzhpUpn/org.xquartz:0
HOMEBREW_EDITOR: vim
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 14.0.0 build 1400
Git: 2.39.2 => /opt/homebrew/bin/git
Curl: 7.79.1 => /usr/bin/curl
macOS: 12.6.3-arm64
CLT: 14.2.0.0.1.1668646533
Xcode: N/A
Rosetta 2: false

Output of brew doctor

Irrelevant

Description of issue

I'd like to have a formula for https://github.com/ggerganov/whisper.cpp — but the program does not provide a way to run make install. Also, it relies on some models being downloaded after the installation. I'm not sure what to do here as I have little experience with formulas.

class WhisperCpp < Formula
  desc "High-performance inference of OpenAI's Whisper automatic speech recognition (ASR) model"
  homepage "https://github.com/ggerganov/whisper.cpp"
  url "https://github.com/ggerganov/whisper.cpp/archive/refs/tags/v1.2.1.tar.gz"
  version "v1.2.1"
  sha256 "69d47fc2ba982c42bd5bade4b7c827d5b3ed74b9c59323991c45a9f0f24eb6ed"
  license "MIT"

  def install
    system "./make"
  end

  test do
    system "false"
  end
end

What do I do after system "./make"? Where should the main binary go?

Then, to download the actual language models, a utility shell script is provided: https://github.com/ggerganov/whisper.cpp#quick-start — I'm not sure how that could be part of the Homebrew installation. Ideally users would see a caveat that the models have to be downloaded manually, and get a pointer to the Homebrew-bundled model download script.
Alternatively, the users could download the models completely manually, but that'd be quite cumbersome.

Would anyone be interested in helping create a formula for this program?

[jamescridland]

I'd also like to see this as a formula. For the standard language model ("small") - which gives excellent results - it should be as "simple" as this:

  def install
    system "./make"
    system "cd /models"
    system "bash ./download-ggml-model.sh small"
  end

... I'd probably then suggest to the user how they might download other language models.

I don't know enough about Homebrew to help understand where the output of "./make" would go, sadly - but this is a perfect opportunity to use Homebrew, particularly since most users will also want to download ffmpeg to produce a version of the audio in 16kHz samplerate (which is all that Whisper.cpp deals with).

[crudalex]

supports the proposal. whisper.cpp is so powerful a transcription utility.

[jgillman]

It exists now! https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/w/whisper-cpp.rb

After manually downloading a model and pointing whisper-cpp to the right place it works fine but it doesn't use metal without some unintuitive trickery. I'm using an M1 Max and without metal it takes about 60 seconds to process a 1 min file. WITH metal the same task only takes 12 seconds!

In order to get the brew installed version to use metal I had to add the following env var:

GGML_METAL_PATH_RESOURCES="/opt/homebrew/Cellar/whisper-cpp/1.5.4/share/whisper-cpp/" whisper-cpp "sample.wav" -m ~/Code/whisper.cpp/models/ggml-medium.en.bin

I'm not sure what the best practice would be to automatically set that env var for the user (or prompt them to on install and upgrade), but it would be cool if it could be super fast straight out of the box.

[slhck]

I'm not sure what the best practice would be to automatically set that env var for the user (or prompt them to on install and upgrade), but it would be cool if it could be super fast straight out of the box.

That would be something to put in your .bash_profile or .zshrc, and Homebrew should ideally advise the user to do so via a caveat message.

You can always get a link to the latest installed version with brew --prefix, so:

export GGML_METAL_PATH_RESOURCES="$(brew --prefix whisper-cpp)/share/whisper-cpp"

Indeed this is much faster than the default variant:

➜ hyperfine --warmup 1 'whisper-cpp samples/jfk.wav -m models/ggml-base.en.bin' 'GGML_METAL_PATH_RESOURCES="$(brew --prefix whisper-cpp)/share/whisper-cpp" whisper-cpp samples/jfk.wav -m models/ggml-base.en.bin'
Benchmark 1: whisper-cpp samples/jfk.wav -m models/ggml-base.en.bin
  Time (mean ± σ):      1.618 s ±  0.040 s    [User: 5.203 s, System: 0.255 s]
  Range (min … max):    1.575 s …  1.682 s    10 runs

Benchmark 2: GGML_METAL_PATH_RESOURCES="$(brew --prefix whisper-cpp)/share/whisper-cpp" whisper-cpp samples/jfk.wav -m models/ggml-base.en.bin
  Time (mean ± σ):     468.6 ms ±  11.1 ms    [User: 156.4 ms, System: 85.3 ms]
  Range (min … max):   453.7 ms … 493.0 ms    10 runs

Summary
  GGML_METAL_PATH_RESOURCES="$(brew --prefix whisper-cpp)/share/whisper-cpp" whisper-cpp samples/jfk.wav -m models/ggml-base.en.bin ran
    3.45 ± 0.12 times faster than whisper-cpp samples/jfk.wav -m models/ggml-base.en.bin