MAVEN: A Massive General Domain Event Detection Dataset

By Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juan-Zi Li, Peng Li, Yankai Lin, Jie Zhou. 2020

225 citations recorded by the source corpus.

Event detection (ED), which identifies event trigger words and classifies event types according to contexts, is the first and most fundamental step for extracting event knowledge from plain text. Most existing datasets exhibit the following issues that limit further development of ED: (1) Small scale of existing datasets is not sufficient for training and stably benchmarking increasingly sophisticated modern neural methods. (2) Limited event types of existing datasets lead to the trained models cannot be easily adapted to general-domain scenarios. To alleviate these problems, we present a MAssive eVENt detection dataset (MAVEN), which contains 4,480 Wikipedia documents, 117,200 event mention instances, and 207 event types. MAVEN alleviates the lack of data problem and covers much more general event types. Besides the dataset, we reproduce the recent state-of-the-art ED models and conduct a thorough evaluation for these models on MAVEN. The experimental results and empirical analyses show that existing ED methods cannot achieve promising results as on the small datasets, which suggests ED in real world remains a challenging task and requires further research efforts. The dataset and baseline code will be released in the future to promote this field.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Lifu Huang
3. Shiyu Chang
4. Xu Han
5. Mo Yu
6. Ning Ding
7. Heng Ji
8. Haitao Zheng

Citations in the corpus

• The Art of Prompting: Event Detection based on Type Specific Prompts (2022, methodology): “…Detection: Two primary learning strategies in few-shot classification tasks are Meta-Learning (Kang et al., 2019; Li et al., 2021; Xiao and Marlet, 2020; Yan et al., 2019; Chowd-hury et al., 2021), and Metric Learning (Sun et al., 2021; Wang et al., 2020b; Zhang et al., 2021a; Agarwal et al., 2021).”
• Query and Extract: Refining Event Extraction as Type-oriented Binary Decoding (2021, methodology): “We do not discuss more as MAVEN only contains trigger annotations.”
• LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2024, unknown): “(Ding et al., 2021), the relation classification dataset DocRED (Yao et al., 2019), the event detection dataset MAVEN (Wang et al., 2020), and the sentiment classification dataset GoEmotions (Demszky et al., 2020).”
• Lifelong Event Detection with Knowledge Transfer (2021, methodology): “We use the provided negative instances for MAVEN (see original paper (Wang et al., 2020) for more details), and collect negative instances as all unlabeled consecutive text spans of at most 3 tokens for ACE 2005.”
• Few-NERD: A Few-shot Named Entity Recognition Dataset (2021, methodology): “The raw corpus we use is the entire Wikipedia dump in English, which has been widely used in constructions of NLP datasets (Han et al., 2018; Yang et al., 2018; Wang et al., 2020).”
• Pretrained Language Model in Continual Learning: A Comparative Study (2022, background): “Maven (Wang et al., 2020) is a long-tailed event detection dataset with 163 classes, each of which is labelled at least 15 instances, reaching a total of 47,921 instances.”