Investigating the Limitations of the Transformers with Simple Arithmetic Tasks

By Rodrigo Nogueira, Zhiying Jiang, Jimmy Lin. 2021

3 citations recorded by the source corpus.

Citations in the corpus

• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2021, neutral): “…models trained on an artificial dataset (Hupkes et al., 2020), pretrained sequence-to-sequence models tested on arithmetic tasks (Nogueira et al., 2021, Appendix C), models trained with reinforcement learning (Lampinen et al., 2021), image, speech recognition, and machine…”