memory optimizations for pyannote.audio.core.inference.Inference.aggregate() #1713
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
benniekiss
force-pushed
the
inference_aggregate_memory_opt
branch
2 times, most recently
from
efd21e0
to
34f4f7e
Compare
|
rebased the changes onto most recent develop, and then fixed an incorrect git authorship config on my end |
hbredin
reviewed
May 19, 2024
since we are overwriting scores.data with an augmented scores.data, just operate on the array in place. * this results in ~8GB of memory savings on a 9 hour recording
storing ALL calculated masks in a tensor can consume a lot of RAM, about 4GB for ~9hr audio. so we save calculating the mask until the loop. because the mask is now calculated in the loop, we have to wait until after calculating the mask in order to np.nan_to_num(score)
benniekiss
force-pushed
the
inference_aggregate_memory_opt
branch
from
64a3351
to
0b4dbd3
Compare
|
rebased and added back the |
|
Merged! 🎉 Thanks a lot for your contribution. Will be part of next release. |
|
Awesome! I really appreciate your work. pyannote has become an invaluable tool, so I'm glad I can give back in my small way. |
|
I'd love to know more about how pyannote impacts your work. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While diarizing long audio recordings (>6 hours), I noticed very high memory usage, upwards of 30GB.
I tracked the spike to
pyannote.audio.core.inference.Inference.aggregate(), which was initializing several very large tensors.With this PR, RAM usage is reduced by 10 - 15 GB for long audio files in my tests. I have not tested extensively, but I do not believe this impacts accuracy or speed.
I did have one question related to one of the commits,
Is this a correct assumption? Otherwise, frames should be reinitialized.
Now, the whole speaker diarization pipeline does not peak past 20GB of RAM for a 9hr recording. this is constrained by both
Inference.aggregateandscipy.cluster.hierarchy.linkagein the AgglomerativeClustering pipeline.