Which transformer architecture fits my data? A vocabulary bottleneck in self-attention

By Noam Wies, Yoav Levine, Daniel Jannai, Amnon Shashua. 2021

Subfields: Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

1. Ankur Bapna
2. Xavier García
3. Colin Cherry
4. Ciprian Chelba
5. Maxim Krikun
6. Orhan Firat
7. Markus Freitag
8. Behrooz Ghorbani

Citations in the corpus

• Scaling Laws for Neural Machine Translation (2021, methodology): “Considering the combinatorial expansion of the search space for scaling each one [29, 26, 37], in this study we choose to vary only the total number of Transformer Blocks, while keeping the internal hyper-parameters intact across different scales.”