Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] refactor filtering and scaling of spectrograms #405

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

superbock
Copy link
Collaborator

@superbock superbock commented Dec 31, 2018

Changes proposed in this pull request

This PR refactors the filtering and scaling of spectrograms to be more memory efficient. Memory savings are mostly due to casting several objects to numpy arrays earlier than before and hence freeing memory by discarding references.

Additional options for stft() allow further memory savings, e.g. by computing only real-valued magnitude spectrograms (via complex=False) or filtering the magnitude spectrogram by passing a filterbank and perform this filtering in blocks (by passing block_size).

The changes come with an API change of FilterbankProcessor (which was never used anywhere in our own code).

This pull request fixes #248, and mitigates #404.

Remaining TODOs before this pull request can be merged

  • add literal options for filterbank (e.g. Mel, Bark, logarithmic, semitone)
  • add more scaling options (e.g. dB, perceptual weighting)
  • rethink default values of ScalingProcessor
  • decide if complex and circular_shift could/should be unified
  • block_size is used already by StrideLayer, but this layer does not accept additional kwargs, thus there is no direct naming conflict, still it is not optimal

@codecov
Copy link

codecov bot commented Jul 23, 2019

Codecov Report

Merging #405 into master will decrease coverage by 0.02%.
The diff coverage is 96.01%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #405      +/-   ##
==========================================
- Coverage   94.61%   94.59%   -0.03%     
==========================================
  Files          93       93              
  Lines       15107    15202      +95     
==========================================
+ Hits        14294    14380      +86     
- Misses        813      822       +9
Impacted Files Coverage Δ
madmom/features/key.py 100% <100%> (ø) ⬆️
madmom/features/beats.py 90.78% <100%> (+0.03%) ⬆️
bin/GMMPatternTracker 96.36% <100%> (+0.06%) ⬆️
madmom/features/notes.py 96.55% <100%> (+0.25%) ⬆️
tests/test_audio_filters.py 100% <100%> (ø) ⬆️
madmom/features/downbeats.py 95.52% <100%> (+0.1%) ⬆️
madmom/features/chords.py 100% <100%> (ø) ⬆️
bin/LogFiltSpecFlux 94.11% <100%> (+0.17%) ⬆️
bin/SuperFluxNN 95.23% <100%> (+0.11%) ⬆️
madmom/audio/spectrogram.py 92.36% <100%> (+0.27%) ⬆️
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c34a40...c970d7b. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rewrite FilterbankProcessor
1 participant