Syntactic Scaffolds for Semantic Structures

By Swabha Swayamdipta, Sam Thomson, Kenton C. T. Lee, Luke S. Zettlemoyer, Chris Dyer, Noah A. Smith. 2018

103 citations recorded by the source corpus.

We introduce the syntactic scaffold, an approach to incorporating syntactic information into semantic tasks. Syntactic scaffolds avoid expensive syntactic processing at runtime, only making use of a treebank during training, through a multitask objective. We improve over strong baselines on PropBank semantics, frame semantics, and coreference resolution, achieving competitive performance on all three tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Percy Liang
2. Mohit Bansal
3. Jonathan Berant
4. Luke S. Zettlemoyer
5. Marek Rei
6. Iryna Gurevych
7. Noah A. Smith
8. Arman Cohan

In Computer Vision and Pattern Recognition

1. Jonathan Berant
2. Iryna Gurevych
3. Arman Cohan
4. Mor Geva
5. Kyle Lo
6. Marek Rei
7. Junichi Tsujii
8. Daniel S. Weld

Citations in the corpus

• Transfer Learning in Natural Language Processing (2019, neutral): “We will then focus on a selection of downstream applications such as classification (Howard and Ruder, 2018), natural language generation, structured prediction (Swayamdipta et al., 2018) or other classification tasks (Peters et al., 2018a; De-vlin et al., 2018).”
• Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language (2018, background): “Swayamdipta et al. (2018) use similar ideas and show that syntactic information can be incorporated in a semantic task with MTL, using auxiliary syntactic tasks without building full-fledged syntactic structure at prediction time.”
• Transfer Fine-Tuning: A BERT Case Study (2019, background): “Swayamdipta et al. (2018) showed that supervision of phrasebased syntax improves the performance of a task relevant to semantics, e.g., semantic role labeling.”
• Control Prefixes for Text Generation (2021, methodology): “Auxiliary scaffold tasks Incorporating auxiliary scaffold tasks via multitask learning has been previously used for improving span-labeling and text classification (Swayamdipta et al., 2018; Cohan et al., 2019).”
• TLDR: Extreme Summarization of Scientific Documents (2020, methodology): “Incorporating auxiliary scaffold tasks via multitask learning has been studied before for improving span-labeling and text classification (Swayamdipta et al., 2018b; Cohan et al., 2019).”
• Control Prefixes for Parameter-Efficient Text Generation (2021, methodology): “This strategy was inspired by previous work which incorporates auxiliary scaffold tasks in multitask learning to improve span-labeling and text classification (Swayamdipta et al., 2018; Cohan et al., 2019; Cachola et al., 2020).”
• Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures (2020, background): “They use syntactic information for the tasks of SRL and coreference resolution.”
• What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models (2021, background): “Hierarchies between NLP tasks have also been explored in multi-task models not based on transformers (Søgaard and Goldberg, 2016; Hashimoto et al., 2017; Swayamdipta et al., 2018).”