YEDDA: A Lightweight Collaborative Text Span Annotation Tool

By Jie Yang, Yue Zhang, Xingxuan Li. 2017

117 citations recorded by the source corpus.

In this paper, we introduce Yedda, a lightweight but efficient and comprehensive open-source tool for text span annotation. Yedda provides a systematic solution for text span annotation, ranging from collaborative user annotation to administrator evaluation and analysis. It overcomes the low efficiency of traditional text annotation tools by annotating entities through both command line and shortcut keys, which are configurable with custom labels. Yedda also gives intelligent recommendations by learning the up-to-date annotated text. An administrator client is developed to evaluate annotation quality of multiple annotators and generate detailed comparison report for each annotator pair. Experiments show that the proposed system can reduce the annotation time by half compared with existing annotation tools. And the annotation time can be further compressed by 16.47% through intelligent recommendation.

Subfields: Information Extraction, Software Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Extraction

1. Jie Yang
2. Colin Raffel
3. Alexander M. Rush
4. Frank F. Xu

In Software Engineering

1. Jie Yang
2. Tingwen Liu
3. Quangang Li
4. Yanzeng Li
5. Yu Bowen

Citations in the corpus

• PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts (2022, background): “Other systems support collaboration among multiple annotators (Yang et al., 2018; Stew-art et al., 2019).”
• Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021, methodology): “Many of these tools (WordFreak, GATE, BRAT, WebAnno, Knowtator and YEDDA) do include system suggestions and pre-annotations [22, 29, 35, 46, 51, 52].”
• Chinese NER Using Lattice LSTM (2018, methodology): “We randomly selected 1027 resume summaries and manually annotated 8 types of named entities with YEDDA system (Yang et al., 2018).”
• FITAnnotator: A Flexible and Intelligent Text Annotation System (2021, methodology): “As Figure 5 shown, by creating a prototype for each category, the clas-sification problem is converted into a problem of matching the samples to the prototypes (Yang et al., 2018a).”
• Redcoat: A Collaborative Annotation Tool for Hierarchical Entity Typing (2019, background): “Yedda (Yang et al., 2018) offers label recommendations via machine learning and provides both command line and web-based interfaces.”
• LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020, background): “…performance on a wide range of sequence labeling and classification tasks such as named entity recognition (NER) (Lample et al., 2016; Ma and Hovy, 2016), relation extraction (RE) (Zeng et al., 2015; Zhang et al., 2017; Ye et al., 2019), and sentiment analysis (SA) (Wang et al., 2016).”
• AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (2019, background): “Existing open-source sequence annotation tools (Stenetorp et al., 2012; Druskat et al., 2014a; de Castilho et al., 2016; Yang et al., 2018a) mainly focus on enhancing the friendliness of user interfaces (UI) such as data management, fast tagging with shortcut keys, supporting more platforms, and…”