-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
refactor gortana directory and readmes to be clearer in their intent
- Loading branch information
1 parent
4d5f4a4
commit 642fcc8
Showing
8 changed files
with
37 additions
and
26 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,36 +1,25 @@ | ||
# diy-jarvis | ||
|
||
~~~~bash | ||
# Check if pulse audio is running on the host | ||
pulseaudio --check -v | ||
~~~~ | ||
This repo contains various scripts and notebooks I've come across for processing audio in general and speech in particular. The overall goal of all this is to create a simple yet flexible system for responding to audio events. | ||
|
||
~~~~bash | ||
# Start pulse audio daemon on the host allowing anonymous connections from the docker ip range | ||
pulseaudio --load="module-native-protocol-tcp auth-ip-acl=127.0.0.1;172.17.0.0/24 auth-anonymous=1" --exit-idle-time=-1 --daemon | ||
~~~~ | ||
|
||
~~~~bash | ||
# mic check - 2 second delay from default in to default out | ||
pacat -r | pacat -p --latency-msec=2000 | ||
~~~~ | ||
|
||
~~~~bash | ||
# run container with diy_jarvis code and pulseaudio TCP connection to host | ||
docker run -it --rm -e PULSE_SERVER=docker.for.mac.localhost -v ~/.config/pulse:/home/pulseaudio/.config/pulse -v ~/code/deproot/src/github.com/kai5263499/diy-jarvis:/go/src/github.com/kai5263499/diy_jarvis pocketsphinx-go bash | ||
~~~~ | ||
## Containerized development | ||
|
||
~~~~bash | ||
# run pocketsphinx without keywords | ||
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /usr/share/pocketsphinx/model/en-us/en-us.lm.bin -dict /usr/share/pocketsphinx/model/en-us/cmudict-en-us.dict -inmic yes | ||
~~~~ | ||
I've found that working in a containerized development environment helps me make my finished product more portable. In order to do that, we need to run pulseaudio on the host and connect it to the container. | ||
|
||
~~~~bash | ||
# run pocketsphinx with keywords | ||
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /go/src/github.com/kai5263499/diy_jarvis/commands/6087.lm -dict /go/src/github.com/kai5263499/diy_jarvis/commands/6087.dic -keyphrase "JARVIS" -kws_threshold 1e-20 -inmic yes | ||
# First, we need to check if pulse audio is running on the host | ||
pulseaudio --check -v | ||
|
||
# Most likely, its not running so we'll need to start pulse audio daemon on the host allowing anonymous connections from the docker ip range, assuming it's 172.17.0.0/24 which appears to be the default for Docker Desktop on my mac | ||
pulseaudio --load="module-native-protocol-tcp auth-ip-acl=127.0.0.1;172.17.0.0/24 auth-anonymous=1" --exit-idle-time=-1 --daemon | ||
|
||
# Now we can run a basic container that has pulseaudio installed to test our audio setup | ||
docker run -it -e PULSE_SERVER=docker.for.mac.localhost -v ~/.config/pulse:/home/pulseaudio/.config/pulse --entrypoint bash --rm jess/pulseaudio | ||
|
||
# We then need to set the default source and sink to and run a mic check with a 2 second delay from our selected default source (in) to default sink (out) to make sure everything's in order | ||
pacat -r | pacat -p --latency-msec=2000 | ||
~~~~ | ||
|
||
~~~~bash | ||
# run gortana with keywords | ||
go run src/github.com/kai5263499/diy_jarvis/gortana/main.go --hmm=/usr/share/pocketsphinx/model/en-us/en-us --dict=/go/src/github.com/kai5263499/diy_jarvis/commands/6087.dic --lm=/go/src/github.com/kai5263499/diy_jarvis/commands/6087.lm --stdout | ||
~~~~ | ||
At this point your environment should be configured to run audio processing scripts and applications using pulseaudio inside of a docker container. This will make development easier and more reproducable when we go to transfer our final application to an embedded system such as a Raspberry Pi. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# gortana | ||
|
||
Gortana is a shameless clone of the example code included in the golang pulseaudio repo. It's based on the pocketsphinx library which is targeted to work on embedded systems. Because these embedded systems have limited resources, we also use a keyword list to reduce the search space that pocketsphinx needs to search through. | ||
|
||
~~~~bash | ||
# Start by running an image that contains pocketsphinx and it's golang wrapper along with the pulseaudio subsystem | ||
docker run -it --rm -e PULSE_SERVER=docker.for.mac.localhost -v ~/.config/pulse:/home/pulseaudio/.config/pulse -v ~/code/deproot/src/github.com/kai5263499/diy-jarvis:/go/src/github.com/kai5263499/diy_jarvis pocketsphinx-go bash | ||
|
||
# Then, run pocketsphinx without keywords. This should illustrate how inefficient pocketsphinx is when searching through the entire english dictionary. | ||
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /usr/share/pocketsphinx/model/en-us/en-us.lm.bin -dict /usr/share/pocketsphinx/model/en-us/cmudict-en-us.dict -inmic yes | ||
|
||
# Next, run pocketsphinx with a limited set of keywords | ||
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /go/src/github.com/kai5263499/diy_jarvis/commands/6087.lm -dict /go/src/github.com/kai5263499/diy_jarvis/commands/6087.dic -keyphrase "JARVIS" -kws_threshold 1e-20 -inmic yes | ||
|
||
# Finally, run gortana using our limited set of keywords | ||
go run ~/dep-root/src/github.com/kai5263499/diy-jarvis/gortana/main.go --hmm=/usr/share/pocketsphinx/model/en-us/en-us --dict=~/dep-root/src/github.com/kai5263499/diy-jarvis/commands/6087.dic --lm=~/dep-root/src/github.com/kai5263499/diy-jarvis/commands/6087.lm --stdout | ||
|
||
# TODO: Get this to cross-compile properly for ARM (Raspberry Pi) | ||
cd ~/dep-root/src/github.com/kai5263499/diy-jarvis/gortana | ||
GOOS=linux GOARCH=arm GOARM=5 go build | ||
|
||
~~~~ |
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.