Partial Or Complete, That’s The Question
By Qiang Ning, Hangfeng He, Chuchu Fan, Dan Roth. 2019
15 citations recorded by the source corpus.
For many structured learning tasks, the data annotation process is complex and costly. Existing annotation schemes usually aim at acquiring completely annotated structures, under the common perception that partial structures are of low quality and could hurt the learning process. This paper questions this common perception, motivated by the fact that structures consist of interdependent sets of variables. Thus, given a fixed budget, partly annotating each structure may provide the same level of supervision, while allowing for more structures to be annotated. We provide an information theoretic formulation for this perspective and use it, in the context of three diverse structured learning tasks, to show that learning from partial structures can sometimes outperform learning from complete ones. Our findings may provide important insights into structured data annotation schemes and could support progress in learning protocols for structured tasks.
Subfields: Artificial Intelligence, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Matt Gardner
- Sameer Singh
- Dheeru Dua
- Jonathan Berant
- Hannaneh Hajishirzi
- Daniel Khashabi
- Ashish Sabharwal
- Mohit Bansal
In Statistics and Probability
Citations in the corpus
- Benefits of Intermediate Annotations in Reading Comprehension (2020, background): “Another line of work (Ning et al., 2019) explores the cost vs. benefit of collecting full vs. partial annotations for various structured predictions tasks.”