-
Notifications
You must be signed in to change notification settings - Fork 428
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recommendations for Replicability #44
Comments
@dehdari did you reproduce the result using provided model? |
Same here. Training doesn't reproduce exactly, and I get the same results as the OP in #22. Inference on the pretrained checked-in model works on the angry male example in Live Demo. |
@dehdari how did you get the male_angry with the provided test. I am getting an error can you please help me out in this. |
The notebook
final_results_gender_test.ipynb
can benefit from some slight modifications that will allow others to replicate exactly the results:After the label encoder is fitted, print what it looks like (i.e.
lb.classes_
), so that we know the order of the labels if we want to just decode without training our own model. This is a simple suggestion that makes a big difference. (I think it's['female_angry', 'female_calm', 'female_fearful', 'female_happy', 'female_sad', 'male_angry', 'male_calm', 'male_fearful', 'male_happy', 'male_sad']
, but I'm not 100% sure).Use a fixed seed for the shuffle in
shuffle(newdf)
, something likeshuffle(newdf, random_state=1)
.Include a script or function that assembles the two different data sources into the
RawData
directory, from original Zip files. The main issue here is ensuring that copying the different subdirectories in the Savee dataset get the same filename changes (eg. " (1)", " (10)") as the checked-in notebook get. Different operating systems like MacOS and Linux behave differently than Windows in how they copy files when there is already an existing file by the same name. I wouldn't mind writing this, but I actually can't replicate the data assemblage. (cf. Training from scratch doesn't reach the same loss #22)The text was updated successfully, but these errors were encountered: