pre-trained-language-models

• [bengio-NPLM] A Neural Probabilistic Language Model

  • code
  • paoer

• [Mikolov-word2vec-models] Efficient Estimation of Word Representations in Vector Space

• [Mikolov-word2vec-training for skip-gram] Distributed Representations of Words and Phrases and their Compositionality

• [doc2vec] Distributed representations of sentences and documents

• [ELMO] Deep contextualized word representations

• [GPT] Improving Language Understandingby Generative Pre-Training

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

• [GPT2.0] Language Models are Unsupervised Multitask Learners

• XLNet: Generalized Autoregressive Pretraining for Language Understanding