A series of notebooks implementing key papers and architectures for neural language modeling. I'm still rewriting my other stuff and I'll update this soonest I finish them. Don't hesitate to contact me if you find any bugs in the code!
PyTorch 1.0 and TorchText 0.3
- 01 - Basic Neural Language Models
- 02 - Regularized and Variational RNNs
- 03 - Weight-tied LSTMs
- 04 - Recurrent Highway Networks
- 05 - AWD-LSTMs
- 06 - Transformer Architectures for Language Modeling
This repository is a work in progress and I will make notebooks available as early as I can, so be sure to check back often!
- 04/20/19 -- Added Recurrent Highway Networks
- 04/21/19 -- Added rewritten version of Weight-Tied LSTMs. Switched to WikiText-2 on post-2014 papers. Will upload the other notebooks as soon as I finish rewriting.