Skip to content

Documents related to the workshop on Dutch language models for historical research

License

Notifications You must be signed in to change notification settings

Semantics-of-Sustainability/2022-12-09-workshop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Workshop on Dutch language models for historical research

This workshop will be held on Dec 9th, 2022 at the Netherlands eScience Center.

If you're interested in attending the workshop, please contact Pim Huijnen ([email protected]).

Description of workshop

The rise of transformer language models such as BERT has opened up possibilities to use contextualized word embeddings for downstream text processing tasks. This includes applications in humanities research, such as investigating the conceptual history of a certain topic in a collection of texts.

However, the current available language models for Dutch (at time of writing) fall short for historical research, because they are trained only on recent data. The performance of models can decline steadily when applied on data that lies outside of the distribution of the training on corpora. On the other hand, pretraining a new language model for each new research task is too computational intensive and not always desirable. Therefore, the need arises for broad general language models on historical texts. At the same time, the methods to properly use these models for historical research are still under development.

The aim of this workshop is to share knowledge on the state-of-the-art of language models for historical research, and to coordinate and lay out a strategy for training language models for historical Dutch. This strategy includes plans on training corpora, model sizes and architectures and model evaluation. The workshop is intended for researchers and practitioners developing or working with Dutch language models.

The morning session will consist of presentations on the following topics:

  • Methodologies for using language models for historical research
  • Evaluation of language models for historical research

The afternoon consists of discussion tables, providing input for a white paper on a strategy for training language models for historical Dutch. We will touch on the following topics:

  • Suitable training corpora
  • Suitable model size and architectures
  • Pretraining vs. finetuning
  • Evaluation of models
  • Methodologies for using language models for historical research

About

Documents related to the workshop on Dutch language models for historical research

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published