IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP
By Fajri Koto, Afshin Rahimi, Jey Han Lau, Timothy Baldwin. International Conference on Computational Linguistics, 2020
448 citations recorded by the source corpus.
Although the Indonesian language is spoken by almost 200 million people and the 10th most spoken language in the world, it is under-represented in NLP research. Previous work on Indonesian has been hampered by a lack of annotated datasets, a sparsity of language resources, and a lack of resource standardization. In this work, we release the IndoLEM dataset comprising seven tasks for the Indonesian language, spanning morpho-syntax, semantics, and discourse. We additionally release IndoBERT, a new pre-trained language model for Indonesian, and evaluate it over IndoLEM, in addition to benchmarking it against existing resources. Our experiments show that IndoBERT achieves state-of-the-art performance over most of the tasks in IndoLEM.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/1109d62ebd2b29a7dc148bc30dd6cfc803a63dec
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Daniel D'souza
- Wei-Yin Ko
- Zheng-Xin Yong
- Niklas Muennighoff
- Gbemileke Onilude
- Marzieh Fadaee
- Shayne Longpre
- Julia Kreutzer
Citations in the corpus
- Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model (2024, background): “…pre-training outside of English (Faysse et al., 2024; Gutiérrez-Fandiño et al., 2021; Zeng et al., 2021; Sengupta et al., 2023; Phan et al., 2022; Koto et al., 2020; Ko et al., 2023) or training models on a small set of languages (Nguyen et al., 2023b; Mesham et al., 2021; Ogueji et al., 2021;…”