KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations

By Fabio Massimo Zanzotto, Andrea Santilli, Leonardo Ranaldi, Dario Onorati, Pierfrancesco Tommasino, Francesca Fallucchi. 2020

60 citations recorded by the source corpus.

Syntactic parsers have dominated natural language understanding for decades. Yet, their syntactic interpretations are losing centrality in downstream tasks due to the success of large-scale textual representation learners. In this paper, we propose KERMIT (Kernel-inspired Encoder with Recursive Mechanism for Interpretable Trees) to embed symbolic syntactic parse trees into artificial neural networks and to visualize how syntax is used in inference. We experimented with KERMIT paired with two state-of-the-art transformer-based universal sentence encoders (BERT and XLNet) and we showed that KERMIT can indeed boost their performance by effectively embedding human-coded universal syntactic representations in neural networks

Subfields: Artificial Intelligence, Model Architectures, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Trung Bui
2. Anh Totti Nguyen
3. Thang M. Pham
4. Long Mai
5. Jacob Andreas
6. Douwe Kiela
7. Ellie Pavlick
8. Albert Webson

In Model Architectures

1. Trung Bui
2. Anh Totti Nguyen
3. Long Mai

Citations in the corpus

• Out of Order: How important is the sequential order of words in a sentence in Natural Language Understanding tasks? (2020, methodology): “Zanzotto et al. 2020 found that for several text classification tasks, syntactic information was not always required.”