Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(separation): add PixIT task, ToTaToNet model and SpeechSeparation pipeline #1676

Merged
merged 95 commits into from
May 30, 2024
Merged

feat(separation): add PixIT task, ToTaToNet model and SpeechSeparation pipeline #1676

merged 95 commits into from
May 30, 2024
+2,316 −0

Conversation

joonaskalda
Copy link
Contributor

The goal of this pull request is to add the new PixIT task, ToTaToNet model, and a pipeline to perform joint speaker diarization/speech separation inference.

joonaskalda and others added 30 commits March 18, 2024 12:53
@joonaskalda
Merge pull request #5 from pyannote/develop
8365802 
git: rebase to latest develop branch
@joonaskalda
add convnet layers to PyanNet
ff2c705
@joonaskalda
add stft and free encoders/decoder
28c9057
@joonaskalda
multitask learning first attempt
eca0a31
@joonaskalda
properly logging mixit loss in train/valid
1d2d1ab
@joonaskalda
add a weight to mixit_loss
4e175b6
@joonaskalda
reformulate multitask loss
7894bf7
@joonaskalda
add dprnn
9eeb29c
@joonaskalda
pair mixtures from same file with no overlapping speakers
2fb16e9
@joonaskalda
fix mixit loss for odd batch size in validation
1a83fec
@joonaskalda
make the MoM part of the original batch
3ebc46a
@joonaskalda
clean up
055744f
@joonaskalda
check that BS is divisible by 3
4af6be1
@joonaskalda
don't use MoMs with more than 3 speakers
4aa83b3
@joonaskalda
include original mixtures in separation branch training
ef14f8f
@joonaskalda
matching the order of dimensions of branch outputs
b218473
@joonaskalda
make n_sources an argument for model constructor
ebe3471
@joonaskalda
changing LSTM default num_layers to 4
df6794b
@joonaskalda
create separate tasks and models
1fe6106
@joonaskalda
Changing n_sources to 3
b3a7821
@joonaskalda
forcing alignment between separation and diarization
0c790d5
@joonaskalda
fixing edge case of 4 speakers in a second chunk
70adb7e
@joonaskalda
adding a VAD-like forced alignment loss
fcde9b8
@joonaskalda
refactor: remove vad_loss and warm_up, assume powerset everywhere
699a2fb
@joonaskalda
remove double check of num_speakers
065cde0
@joonaskalda
refactor: moved mom constrcution
3ff0ba5
@joonaskalda
remove unused mixit wrapper
8bb63d2
@joonaskalda
format with black
5bd913d
@joonaskalda
fix for last batch in validation having size 1
40c11db
@joonaskalda
adding documentation
3ee1484
@clement-pages clement-pages mentioned this pull request May 22, 2024
Open
@hbredin hbredin changed the title Add the PixIT task, the ToTaToNet model, and a pipeline for joint speaker diarization/speech separation inference feat(separation): add PixIT task, ToTaToNet model and SpeechSeparation pipeline May 30, 2024
@hbredin hbredin merged commit 49d3b8e into pyannote:develop May 30, 2024
4 checks passed
@hbredin
Copy link
Member

hbredin commented May 30, 2024

🎉 Merged! Huge!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@joonaskalda @hbredin