Neural Legal Judgment Prediction in English

By Ilias Chalkidis, Ion Androutsopoulos, Nikolaos Aletras. 2019

464 citations recorded by the source corpus.

Legal judgment prediction is the task of automatically predicting the outcome of a court case, given a text describing the case’s facts. Previous work on using neural models for this task has focused on Chinese; only feature-based models (e.g., using bags of words and topics) have been considered in English. We release a new English legal judgment prediction dataset, containing cases from the European Court of Human Rights. We evaluate a broad variety of neural models on the new dataset, establishing strong baselines that surpass previous feature-based models in three tasks: (1) binary violation classification; (2) multi-label classification; (3) case importance prediction. We also explore if models are biased towards demographic information via data anonymization. As a side-product, we propose a hierarchical version of BERT, which bypasses BERT’s length limitation.

Subfields: Artificial Intelligence, Law, Political Science and International Relations, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Law

In Political Science and International Relations

In Natural Language Processing

Citations in the corpus

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022, methodology): “ECtHR contains legal cases from the European Court of Human Rights, each of which is mapped to articles of the Convention of Human Rights that were allegedly violaged.”
CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain (2019, background): “With the development of deep learning, more researchers pay much efforts on predicting the judgment result of legal cases (Luo et al., 2017; Hu et al., 2018; Zhong et al., 2018a; Chalkidis et al., 2019; Jiang et al., 2018; Yang et al., 2019).”
When does pretraining help?: assessing self-supervised learning for law and the CaseHOLD dataset of 53,000+ legal holdings (2021, methodology): “8% improvement in F1 on a binary classification task derived from the ECHR-CASES dataset [5], a 2.”
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence (2020, background): “Besides, English LJP datasets also exist (Chalkidis et al., 2019a), but the size is limited.”
Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents (2021, unknown): “…efforts have been devoted to solving various legal tasks, such as legal charge prediction (Luo et al., 2017; Zhong et al., 2018; Xiao et al., 2018; Chalkidis et al., 2019; Yang et al., 2019), relevant law article retrieval (Chen et al., 2013; Raghav et al., 2016), court view generation (Ye et…”
LEGAL-BERT: “Preparing the Muppets for Court’” (2020, methodology): “Bellow are the details of the legal NLP datasets we used for the evaluation of our models: • EURLEX57K (Chalkidis et al., 2019b) contains 57k legislative documents from EURLEX with an average length of 727 words.”
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2021, background): “, 2021), from judgment prediction (Aletras et al., 2016; Sim et al., 2016; Katz et al., 2017; Zhong et al., 2018; Chalkidis et al., 2019a; Malik et al., 2021), information extraction from legal documents (Chalkidis et al.”
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases (2021, methodology): “3 The New ECtHR Dataset The court ( EC t HR ) hears allegations regarding breaches in human rights provisions of the European Convention of Human Rights ( ECHR ) by European states (Fig.”