Understanding the output during training

[yuezha01]

Hey,

Thanks a lot for implementing ENAS in Pytorch! I am able to run ENAS with Penn Treebank dataset. I am trying to understand the output during training. At beginning, the output shows some information about the gradients, for example:

2018-08-29 23:53:50,910:INFO::abs max grad 0.5459082126617432
2018-08-29 23:53:56,523:INFO::abs max grad 0.569364070892334
2018-08-29 23:54:02,990:INFO::abs max grad 0.6024199724197388
2018-08-29 23:54:11,814:INFO::max hidden 16.342870712280273

Then it shows loss and ppl but the epoch number is 0 all the time.
2018-08-29 23:54:17,591:INFO::| epoch 0 | lr 20.00 | raw loss 8.33 | loss 8.33 | ppl 4140.68
2018-08-29 23:54:58,164:INFO::| epoch 0 | lr 20.00 | raw loss 7.29 | loss 7.29 | ppl 1470.41

But after a few epochs, when ppl goes down to 600~, the output changes to show loss and entropy. No ppl is shown.
2018-08-30 00:03:58,605:INFO::| epoch 0 | lr 0.00035 | R 0.13311 | entropy 1.1463 | loss 0.00677
2018-08-30 00:04:10,912:INFO::| epoch 0 | lr 0.00035 | R 0.13406 | entropy 1.1326 | loss 0.01401

Could anyone help me understand the output a little? Why does the output stop showing ppl and why the epoch number is always 0? How to directly monitor the ppl?

Thanks!
Yue