Structured Neural Summarization
By Patrick Fernandes, Miltiadis Allamanis, Marc Brockschmidt. 2018
230 citations recorded by the source corpus.
Summarization of long sequences into a concise statement is a core problem in natural language processing, requiring non-trivial understanding of the input. Based on the promising results of graph neural networks on highly structured data, we develop a framework to extend existing sequence encoders with a graph component that can reason about long-distance relationships in weakly structured data such as text. In an extensive evaluation, we show that the resulting hybrid sequence-graph models outperform both pure sequence models as well as pure graph models on a range of summarization tasks.
Subfields: Artificial Intelligence, Summarization
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Percy Liang
- Graham Neubig
- Luke S. Zettlemoyer
- Yoshua Bengio
- Zhe Gan
- Kyunghyun Cho
In Summarization
Citations in the corpus
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, background): “• code-code (clone detection [65, 84, 46, 80, 9, 89, 86], defect detection [91, 55, 51, 42, 78, 79], cloze test [15], code completion [57, 1, 56, 66, 67, 8, 27, 7], code repair [69, 4, 23, 26, 71, 73], and code-to-code translation [48, 37, 10, 41]) • text-code (natural language code search [32, 20, ”
- CodeSearchNet Challenge: Evaluating the State of Semantic Code Search (2019, methodology): “[9] consider the task of predicting the documentation text from source code.”
- Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking (2020, background): “…et al., 2018; De Cao et al., 2019), syntactic and semantic parsing tasks (Marcheggiani & Titov, 2017; Bogin et al., 2019; Ji et al., 2019), summarisation (Fernandes et al., 2019), machine translation (Bastings et al., 2017) and abusive language detection in social networks (Mishra et al., 2019).”
- Improved Code Summarization via a Graph Neural Network (2020, methodology): “[16] x x”
- Neural Edit Completion (2020, unknown): “…line of work represent programs as graphs – these usually augment the AST of a program with additional semantic edges and use a graph neural network to learn from the resulting graph [Allamanis et al. 2018; Brockschmidt et al. 2019; Fernandes et al. 2019; Hellendoorn et al. 2020; Yin et al. 2019].”
- Adversarial examples for models of code (2019, background): “2015], code summarization [Allamanis et al. 2016; Alon et al. 2019a; Fernandes et al. 2019], code generation [Alon et al.”
- Hierarchical Transformers for Multi-Document Summarization (2019, background): “…latent representations are beneficial for downstream NLP tasks (Liu and Lapata, 2018; Kim et al., 2017; Williams et al., 2018; Niculae et al., 2018; Fernandes et al., 2019), much work in multi-document summarization has taken advantage of explicit graph representations, each focusing on different…”
- Integrating Tree Path in Transformer for Code Representation (2021, background): “…absolute path for token result starts at the leaf node and ends at the tree root, which reveals the behaviour of assignment for this token. typically rely on synchronous message passing, requiring message passing iterations to aggregate information [Allamanis et al., 2017, Fernandes et al., 2018].”
- PLUR: A Unifying, Graph-Based View of Program Learning, Understanding, and Repair (2021, background): “…by the task to produce the intended raw output, or a task-specific performance metric. pointers required by the task output, such mechanisms use pointer decoders to identify content in the input from which to copy into the output, e.g., for summarization [See et al., 2017, Fernandes et al., 2018].”
- Deep Just-In-Time Inconsistency Detection Between Comments and Source Code (2020, methodology): “Ad-ditionally, we remove duplicate examples, as they have been found to negatively affect training machine learning models for source code (Allamanis 2019).”
- Teaching Temporal Logics to Neural Networks (2020, methodology): “Transformers have also been applied to programs for tasks such as summarizing code (Fernandes et al., 2018) or variable naming and misuse (Hellendoorn et al.”
- Typilus: neural type hints (2020, unknown): “com/allenai/allennlp/pull/3376 versa [8, 11, 24, 35].”
- Discourse-Aware Neural Extractive Text Summarization (2019, background): “Fernandes et al. (2018) developed a framework to reason long-distance relationships for text summarization.”
- Structural Language Models for Any-Code Generation (2019, background): “…be necessary in semantic parsing (Rabinovich et al., 2017; Dong and Lapata, 2018), NL → code (Yin and Neubig, 2017) and code → NL (Alon et al., 2019a; Fernandes et al., 2019) tasks because of the different modalities of the input and the output, this separation may hurt performance when the…”
- Code Prediction by Feeding Trees to Transformers (2020, background): “Possible mitigation includes copying mechanism [48], [51], [65] and open-vocabulary models [7], [66].”
- HELoC: Hierarchical Contrastive Learning of Source Code Representation (2022, background): “Previous source code representations [31, 32] have demonstrated that GNNs can better encode the local structure of ASTs [33].”
- Language-Agnostic Representation Learning of Source Code from Structure and Context (2021, methodology): “We compare with code2seq (Alon et al., 2019a), the Graph Relational Embedding Attention Transformer (GREAT) (Hellendoorn et al., 2020), and the BiLSTM+GNN → LSTM+Pointer model presented in Fernandes et al. (2019).”