Skip to content

Commit

Permalink
refactor gortana directory and readmes to be clearer in their intent
Browse files Browse the repository at this point in the history
  • Loading branch information
kai5263499 committed Jul 4, 2019
1 parent 4d5f4a4 commit 642fcc8
Show file tree
Hide file tree
Showing 8 changed files with 37 additions and 26 deletions.
41 changes: 15 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,25 @@
# diy-jarvis

~~~~bash
# Check if pulse audio is running on the host
pulseaudio --check -v
~~~~
This repo contains various scripts and notebooks I've come across for processing audio in general and speech in particular. The overall goal of all this is to create a simple yet flexible system for responding to audio events.

~~~~bash
# Start pulse audio daemon on the host allowing anonymous connections from the docker ip range
pulseaudio --load="module-native-protocol-tcp auth-ip-acl=127.0.0.1;172.17.0.0/24 auth-anonymous=1" --exit-idle-time=-1 --daemon
~~~~

~~~~bash
# mic check - 2 second delay from default in to default out
pacat -r | pacat -p --latency-msec=2000
~~~~

~~~~bash
# run container with diy_jarvis code and pulseaudio TCP connection to host
docker run -it --rm -e PULSE_SERVER=docker.for.mac.localhost -v ~/.config/pulse:/home/pulseaudio/.config/pulse -v ~/code/deproot/src/github.com/kai5263499/diy-jarvis:/go/src/github.com/kai5263499/diy_jarvis pocketsphinx-go bash
~~~~
## Containerized development

~~~~bash
# run pocketsphinx without keywords
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /usr/share/pocketsphinx/model/en-us/en-us.lm.bin -dict /usr/share/pocketsphinx/model/en-us/cmudict-en-us.dict -inmic yes
~~~~
I've found that working in a containerized development environment helps me make my finished product more portable. In order to do that, we need to run pulseaudio on the host and connect it to the container.

~~~~bash
# run pocketsphinx with keywords
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /go/src/github.com/kai5263499/diy_jarvis/commands/6087.lm -dict /go/src/github.com/kai5263499/diy_jarvis/commands/6087.dic -keyphrase "JARVIS" -kws_threshold 1e-20 -inmic yes
# First, we need to check if pulse audio is running on the host
pulseaudio --check -v

# Most likely, its not running so we'll need to start pulse audio daemon on the host allowing anonymous connections from the docker ip range, assuming it's 172.17.0.0/24 which appears to be the default for Docker Desktop on my mac
pulseaudio --load="module-native-protocol-tcp auth-ip-acl=127.0.0.1;172.17.0.0/24 auth-anonymous=1" --exit-idle-time=-1 --daemon

# Now we can run a basic container that has pulseaudio installed to test our audio setup
docker run -it -e PULSE_SERVER=docker.for.mac.localhost -v ~/.config/pulse:/home/pulseaudio/.config/pulse --entrypoint bash --rm jess/pulseaudio

# We then need to set the default source and sink to and run a mic check with a 2 second delay from our selected default source (in) to default sink (out) to make sure everything's in order
pacat -r | pacat -p --latency-msec=2000
~~~~

~~~~bash
# run gortana with keywords
go run src/github.com/kai5263499/diy_jarvis/gortana/main.go --hmm=/usr/share/pocketsphinx/model/en-us/en-us --dict=/go/src/github.com/kai5263499/diy_jarvis/commands/6087.dic --lm=/go/src/github.com/kai5263499/diy_jarvis/commands/6087.lm --stdout
~~~~
At this point your environment should be configured to run audio processing scripts and applications using pulseaudio inside of a docker container. This will make development easier and more reproducable when we go to transfer our final application to an embedded system such as a Raspberry Pi.
22 changes: 22 additions & 0 deletions gortana/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# gortana

Gortana is a shameless clone of the example code included in the golang pulseaudio repo. It's based on the pocketsphinx library which is targeted to work on embedded systems. Because these embedded systems have limited resources, we also use a keyword list to reduce the search space that pocketsphinx needs to search through.

~~~~bash
# Start by running an image that contains pocketsphinx and it's golang wrapper along with the pulseaudio subsystem
docker run -it --rm -e PULSE_SERVER=docker.for.mac.localhost -v ~/.config/pulse:/home/pulseaudio/.config/pulse -v ~/code/deproot/src/github.com/kai5263499/diy-jarvis:/go/src/github.com/kai5263499/diy_jarvis pocketsphinx-go bash

# Then, run pocketsphinx without keywords. This should illustrate how inefficient pocketsphinx is when searching through the entire english dictionary.
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /usr/share/pocketsphinx/model/en-us/en-us.lm.bin -dict /usr/share/pocketsphinx/model/en-us/cmudict-en-us.dict -inmic yes

# Next, run pocketsphinx with a limited set of keywords
pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/en-us/en-us -lm /go/src/github.com/kai5263499/diy_jarvis/commands/6087.lm -dict /go/src/github.com/kai5263499/diy_jarvis/commands/6087.dic -keyphrase "JARVIS" -kws_threshold 1e-20 -inmic yes

# Finally, run gortana using our limited set of keywords
go run ~/dep-root/src/github.com/kai5263499/diy-jarvis/gortana/main.go --hmm=/usr/share/pocketsphinx/model/en-us/en-us --dict=~/dep-root/src/github.com/kai5263499/diy-jarvis/commands/6087.dic --lm=~/dep-root/src/github.com/kai5263499/diy-jarvis/commands/6087.lm --stdout

# TODO: Get this to cross-compile properly for ARM (Raspberry Pi)
cd ~/dep-root/src/github.com/kai5263499/diy-jarvis/gortana
GOOS=linux GOARCH=arm GOARM=5 go build

~~~~
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit 642fcc8

Please sign in to comment.