Interscript: A dataset for interactive learning of scripts through error feedback

By Niket Tandon, Aman Madaan, Peter Clark, Keisuke Sakaguchi, Yiming Yang. 2021

12 citations recorded by the source corpus.

How can an end-user provide feedback if a deployed structured prediction model generates inconsistent output, ignoring the structural complexity of human language? This is an emerging topic with recent progress in synthetic or constrained settings, and the next big leap would require testing and tuning models in real-world settings. We present a new dataset, Interscript, containing user feedback on a deployed model that generates complex everyday tasks. Interscript contains 8,466 data points -- the input is a possibly erroneous script and a user feedback, and the output is a modified script. We posit two use-cases of \ours that might significantly advance the state-of-the-art in interactive learning. The dataset is available at: https://github.com/allenai/interscript.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yiming Yang
2. Sarah Wiegreffe
3. Luyu Gao
4. Aman Madaan
5. Uri Alon
6. Sean Welleck
7. Peter Clark
8. Niket Tandon

In Natural Language Processing

1. Sarah Wiegreffe
2. Sean Welleck
3. Aman Madaan
4. Yiming Yang
5. Luyu Gao
6. Peter Clark
7. Uri Alon
8. Niket Tandon

In Evaluation & Benchmarks

1. Peter Clark
2. Sean Welleck
3. Prakhar Gupta
4. Niket Tandon
5. Aman Madaan
6. Bodhisattwa Prasad Majumder
7. Yiming Yang
8. Sarah Wiegreffe

Citations in the corpus

• Self-Refine: Iterative Refinement with Self-Feedback (2023, background): “…(NL) feedback for refining outputs has been effective for a variety of tasks, including summarization (Scheurer et al., 2022), script generation (Tandon et al., 2021), program synthesis (Le et al., 2022a; Yasunaga and Liang, 2020), and other tasks (Bai et al., 2022a; Schick et al., 2022b;…”
• OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement (2024, unknown): “For many sequence generation tasks, iterative approaches are often taken to improve the generation quality, e.g., script generation (Tandon et al., 2021), summarization (Scheurer et al., 2022), and other tasks as shown in (Madaan et al., 2022; Saunders et al., 2022).”