Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents

By Chaojun Xiao, Xueyu Hu, Zhiyuan Liu, Cunchao Tu, Maosong Sun. AI Open, 2021

351 citations recorded by the source corpus.

Legal artificial intelligence (LegalAI) aims to benefit legal systems with the technology of artificial intelligence, especially natural language processing (NLP). Recently, inspired by the success of pre-trained language models (PLMs) in the generic domain, many LegalAI researchers devote their effort to apply PLMs to legal tasks. However, utilizing PLMs to address legal tasks is still challenging, as the legal documents usually consist of thousands of tokens, which is far longer than the length that mainstream PLMs can process. In this paper, we release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding. We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering. The experimental results demonstrate that our model can achieve promising improvement on tasks with long documents as inputs.

Subfields: Artificial Intelligence, Political Science and International Relations

View Paper: https://www.semanticscholar.org/paper/1e3e65e7773b7869d9bd7f5394b54199e48195e6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Daniel Martin Katz
2. Michael James Bommarito II

Citations in the corpus

• GPT-4 passes the bar exam (2024, background): “NLP models have progressed in the legal domain(19,42,43) with increasing application of neural techniques on specific legal tasks.(44,45,46) Several recent papers have demonstrated meaningful zero-shot progress on a variety of applied tasks,(2,47,48,49,50,51) suggesting further potential for applica”