Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches

By Nelson F. Liu, Tony Lee, Robin Jia, Percy Liang. 2021

21 citations recorded by the source corpus.

Subfields: Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Mohit Bansal
2. Luke S. Zettlemoyer
3. Peter Hase
4. Alon Y. Halevy
5. Madian Khabsa
6. Jacob Andreas
7. Jane A. Yu
8. Belinda Z. Li

Citations in the corpus

• When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, background): “For further discussion of the pros and cons of synthetic datasets, see Liu et al. (2021).”
• Quantifying Adaptability in Pre-trained Language Models with 500 Tasks (2021, background): “…to past work that uses synthetic data to characterize LM performance on single examples (Weston et al., 2016; Lake and Baroni, 2018; Saxton et al., 2019; Kim and Linzen, 2020; Keysers et al., 2020; Liu et al., 2021a), T ASK B ENCH 500 enables systematic study of LM adaptability at the task level.”