Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0
By F. Toni, Christopher Akiki, Javier de la Rosa, Clémentine Fourrier, Enrique Manjavacas, Stefan Schweter, Daniel Alexander van Strien. 2022
11 citations recorded by the source corpus.
In this work, we explore whether the recently demonstrated zero-shot abilities of the T0 model extend to Named Entity Recognition for out-of-distribution languages and time periods. Using a historical newspaper corpus in 3 languages as test-bed, we use prompts to extract possible named entities. Our results show that a naive approach for prompt-based zero-shot multilingual Named Entity Recognition is error-prone, but highlights the potential of such an approach for historical languages lacking labeled datasets. Moreover, we also find that T0-like models can be probed to predict the publication date and language of a document, which could be very relevant for the study of historical texts.
Subfields: Artificial Intelligence, Large Language Models, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Mohit Bansal
- Yejin Choi
- Noah A. Smith
- Yonatan Belinkov
- Percy Liang
- Graham Neubig
- Jianfeng Gao
In Large Language Models
- Timo Schick
- Colin Raffel
- Pascale Fung
- Luke S. Zettlemoyer
- Douwe Kiela
- Alexander M. Rush
- Niklas Muennighoff
- Yonatan Belinkov
In Natural Language Processing
- Mohit Bansal
- Yonatan Belinkov
- Yejin Choi
- Noah A. Smith
- Sebastian Gehrmann
- Graham Neubig
- Samuel R. Bowman
- Percy Liang
Citations in the corpus
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “In addition, a few groups focused on the evaluation of LLMs and dataset development in specific domains, such as biomedical texts (Fries et al., 2022b) and historical texts (De Toni et al., 2022).”