Skip to content

instrumentbible/solfege.ai

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

31 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

solfege.ai โœ‹ ๐ŸŽน

Detect solfege hand signs using machine learning (live demo)

a project by Ryan Kemmer and Josh Stovall

About

We have developed a system that allows the user to 'play' solfege hand signs as a MIDI instrument. This technology could have practical applications both in music education and creation.

Solfege hand signs are often used in school choirs to teach music theory concepts. Since vocalists cannot actually see the notes on their instrument, hand signs have been developed as a teaching aid.

This system allows students to play the hand signs as an instrument -- recieving auditory feedback. Previously, a student learning solfege would have to sing the note or play another instrument to 'hear' their hand signs.

Libraries

This project uses TensorFlow, MediaPipe, and JZZ.js.

Setup

cd solfege
php -S 0.0.0.0:1234

go to http://localhost:1234

Use

creating data sets

  • make a hand sign
  • click the button to create training data
  • it will wait 1 second, start recording data for 30 seconds, then save a file with the data

Hand Signs

hand sign name pitch class
(distance from C)
equivalent note
do 0 C
di 1 C#
ra 1 Db
re 2 D
ri 3 D#
me 3 Eb
mi 4 E
fa 5 F
fi 6 F#
se 6 Gb
so 7 G
si 8 G#
le 8 Ab
la 9 A
li 10 A#
te 10 Bb
ti 11 B

Contributing

Any contributions you make are greatly appreciated. Any bugs and change requests are to be reported on the issues tab. If you don't like coding, you can contribute by becoming a sponsor.

GitHub Sponsors Patreon Square Venmo PayPal

Questions?

Please write to [email protected] or visit instrument.bible.

Discord Twitter YouTube Facebook LinkedIn Instagram