Large-Scale Multi-Label Text Classification on EU Legislation

By Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Ion Androutsopoulos. 2019

261 citations recorded by the source corpus.

We consider Large-Scale Multi-Label Text Classification (LMTC) in the legal domain. We release a new dataset of 57k legislative documents from EUR-LEX, annotated with ∼4.3k EUROVOC labels, which is suitable for LMTC, few- and zero-shot learning. Experimenting with several neural classifiers, we show that BIGRUs with label-wise attention perform better than other current state of the art methods. Domain-specific WORD2VEC and context-sensitive ELMO embeddings further improve performance. We also find that considering only particular zones of the documents is sufficient. This allows us to bypass BERT’s maximum text length limit and fine-tune BERT, obtaining the best results in all but zero-shot learning cases.

Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Information Extraction

Citations in the corpus

Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, background): “EURLEX57K (Chalkidis et al., 2019).”
A Survey on Text Classification: From Traditional to Deep Learning (2020, unknown): “The SICK contains almost 10,000 English sentence pairs.”
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2021, background): “The ﬁrst one (Aletras et al., 2016; Chalkidis et al., 2019a; Medvedeva et al., 2020, 2021) predicts violations of human rights in cases of the European Court of Human Rights (ECtHR).”
Privacy Risks of General-Purpose Language Models (2020, methodology): “Language Processing (NLP), the last year has witnessed many breakthroughs in building general-purpose language models by industry leaders like Google, OpenAI and Facebook [16], [17], [41], [44], [54], [55], [67], [76], which have been widely used in various downstream NLP tasks such as text classifi”
LEGAL-BERT: “Preparing the Muppets for Court’” (2020, methodology): “Bellow are the details of the legal NLP datasets we used for the evaluation of our models: • EURLEX57K (Chalkidis et al., 2019b) contains 57k legislative documents from EURLEX with an average length of 727 words.”
A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering (2020, unknown): “Some datasets are concerned with very specific tasks, as in tagging in contracts [10], classifying clauses [11], and classification of documents [12] or single paragraphs [6].”
Meta-tuning Language Models to Answer Prompts Better (2021, background): “" EURLEX57K (Chalkidis et al., 2019 (Zampieri et al., 2019).”
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review (2021, background): “Holzenberger et al. (2020) introduce a dataset for tax law entailment and question answering and Chalkidis et al. (2019) introduce a large dataset of text classiﬁcation for EU legislation.”
The Emerging Trends of Multi-Label Learning (2020, background): “XMLC [192] and DL MLC [193] are proposed to”
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (2022, methodology): “0 EUR-Lex [28] Creative Commons Attribution 4.”
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence (2020, background): “Besides, English LJP datasets also exist (Chalkidis et al., 2019a), but the size is limited.”
A Survey on Text Classification: From Shallow to Deep Learning (2020, unknown): “…contextualized representation [66] Accuracy SQuAD, SNLI, SST-5 bidirectional encoder [19] Accuracy SST-2, QQP, QNLI, CoLA multi-label legal text [134] P@K, RP@K, R@K, etc. EUR-LEX Trans fine-tune BERT [135] Error Rate IMDB, TREC, DBPedia, etc. autoregressive pretraining [85] DNCG@K, EM, F1,…”