Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About the number of epochs? #29

Open
mangozy opened this issue Aug 23, 2021 · 2 comments
Open

About the number of epochs? #29

mangozy opened this issue Aug 23, 2021 · 2 comments

Comments

@mangozy
Copy link

mangozy commented Aug 23, 2021

How many epochs did u run in order to obtain the accuracy/error you claimed? I haven't found you have documented that anywhere. If you could explain your training policy a little bit more?

@zwxandy
Copy link

zwxandy commented Jan 1, 2023

I have read other issues in this repo and I find the number of epochs is set to 200 by default.
Other training details are:
batch-size=128, learning-rate=0.1, momentum=0.9, weight-decay=1e-4, optimizer='SGD', criterion='CrossEntropyLoss', lr-scheduler='MultiStepLR' (milestones=[100, 150]), simple data augmentation: RandomHorizontalFlip(p=0.5) and RandomCrop(size32, padding=4), adopt weight initialization, adopt BN, no dropout!!!
As we can see, this implementation strictly follows the "4.2. CIFAR-10 and Analysis" section in the original paper (i.e., ResNet). Great work!

@JonnaMat
Copy link

JonnaMat commented Mar 30, 2023

Regarding the total number of epochs and the LR schedule the paper states that

"We start with a learning
rate of 0.1, divide it by 10 at 32k and 48k iterations, and
terminate training at 64k iterations, which is determined on
a 45k/5k train/val split"

Given, that 64000 iterations roughly translate to 180 epochs (see reasoning below), it would be more accurate to use those epochs with the following LR steps for the scheduler: [90, 135].
Since 45000/128 = 351.56 (i.e. 351 iterations per epoch) and 64000/351.56 = 182.05
Conversion to epochs: iterations / (n_observations/batch_size)

I also often see 160 epochs with [80, 120].
Since 50000/128 = 390.63 and 64000/390.63 = 163.84

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants