A Deep Network with Visual Text Composition Behavior

By Hongyu Guo. 2017

3 citations recorded by the source corpus.

While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tommi S. Jaakkola
2. Regina Barzilay
3. Tal Schuster
4. Adam Fisch
5. Percy Liang
6. Luke S. Zettlemoyer
7. Mohit Bansal
8. Caiming Xiong

In Computer Vision and Pattern Recognition

1. Regina Barzilay
2. Adam Fisch
3. Tal Schuster
4. Tommi S. Jaakkola
5. Jason Aaron Edward Weston
6. Percy Liang
7. Hannaneh Hajishirzi
8. Mohit Bansal

In Multimodal Learning

1. Regina Barzilay
2. Adam Fisch
3. Tal Schuster
4. Tommi S. Jaakkola

Citations in the corpus

• Consistent Accelerated Inference via Confident Adaptive Transformers (2021, methodology): “Softmax values are calibrated using temperature scaling (Guo, 2017) on another held-out (labeled) data split, Dscale.”