Skip to content

Machine Learning applied to sound

Notifications You must be signed in to change notification settings

jonnor/machinehearing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Machine Hearing

Machine Hearing, or Machine Listening, is the use of Machine Learning and audio sensors to derive meaningful information from sound. This include listening for and diagnosing problems in machinery, understanding events and activities that cause noise, and estimation of how humans perceive certain sounds.

Here you can find some notes on the topic compiled by Jon Nordby.

Soundsensing logo

This research is sponsored by Soundsensing, a provider of IoT audio sensors with built-in Machine Learning, used for Noise Monitoring and Condition Monitoring. The sensors are ideal for continious monitoring of audible noises and events, and can perform tasks such as Audio Classification, Audio Event Detection and Acoustic Anomaly Detection. Their sensors can transmit compressed and privacy-preserving spectrograms, allowing Machine Learning to be done in the cloud using familiar tools like Python. Or models can be deployed onto the sensor itself, for a highly efficient on-edge ML solution.

Pages

Some information is found in sub-pages.

Recent work

EuroPython 2021: Sound Event Detection with Machine Learning

Youtube: Sound Event Detection with Machine Learning (EuroPython 2021)

July 26, 2021. Presented at EuroPython 2021. Video recording, slides, notes.

TinyML EMEA 2021: Perfect coffee roasting with TinyML sound sensing

Perfect coffee roasting with TinyML sound sensing

June 7, 2021. Presented at tinyML EMEA Technical Forum 2021. Video recording coming, slides, notes.

TinyML Summit 2021: Environmental Sound Classification on microcontrollers

Environmental Sound Classification on microcontrollers

March 25, 2021. Video recording, slides, notes.

Classifying sound using Machine Learning

Youtube: Classifying sound using Machine Learning

At KnowIt Oslo, 2020. Video recording, slides, notes

Environmental Sound Classification on Microcontrollers using Convolutional Neural Networks

Github: jonnor/ESC-CNN-microcontroller

Master thesis. Report and code available in the Github repository.

EuroPython2019: Audio Classification using Machine Learning

Youtube: Audio Classification using Machine Learning by Jon Nordby, EuroPython 2019

Presentation at EuroPython2019. Video recording, notes

PyCode2019: Recognizing sounds with Machine Learning and Python

Slides

Presentation at PyCode Conference 2019 in Gdansk. Slides, notes

Video recording. Coming, maybe in November.

SenseCamp2019: Classification of Environmental Sound using IoT sensors

Slides

Presentation at SenseCamp 2019 hosted by FORCE Technology Senselab. Slides: web, .PDF

NMBU lecture on Audio Classification

Report and lecture at NMBU Data Science.

Report | Slides

Stack Overflow answers

With example code in Python

Notes

Rough notes on various topics.

Resources

Useful resources to learn more.

Presentations

Books

  • Computational Analysis of Sound Scenes and Events. Tuomas Virtanen, Mark D. Plumbley, Dan Ellis. 2018.
  • Human and Machine Hearing - Extracting Meaning from Sound. Richard F. Lyon. 2017, revised 2018.
  • An Introduction to Audio Content Analysis - Applications in Signal Processing and Music Informatics. Alexander Lerch. 2012. Companion website: https://www.audiocontentanalysis.org/
  • Machine Learning for Audio, Image and Video Analysis: Theory and Applications (Advanced Information and Knowledge Processing). Francesco Camastra, 3 sections. From Perception to Computation, Machine Learning, Applications.

Online courses

Software

Feature extraction

  • librosa. The go-to Python module.
  • essentia. C++ library, with Python bindings. Lots of Music Analysis extractors. Used by FreeSound and Acousticbrainz.
  • kapre. On-demand GPU computation of melspectrograms, for Keras
  • torchaudio. Audio processing in PyTorch

Data Augmentation

Lecture notes

Competions

  • Kaggle Whale detection
  • Kaggle FreeSound tagging 2018
  • Kaggle FreeSound
  • DCASE2014
  • DCASE2018
  • DCASE2019
  • DCASE2020
  • DCASE2021

Datasets

Online Communities

Lists