Not All Images are Worth 16x16 Words: Dynamic Vision Transformers with Adaptive Sequence Length

By Yulin Wang, Rui Huang, Shiji Song, Zeyi Huang, Gao Huang. 2021

48 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Yizeng Han
5. Le Yang
6. Yibing Song
7. Zhan Tong
8. Pengtao Xie

In Efficiency & Compression

1. Shiji Song
2. Gao Huang
3. Yulin Wang
4. Le Yang
5. Yizeng Han
6. Honghui Wang

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, methodology): “Although inference with batches has been enabled for early-exiting networks [271], the conflict between adaptive computational graph and parallel computation still exists for other types of dynamic”
• Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations (2022, methodology): “Notably, Wang et al. (2021b) proposed a method to dynamically determine the number of patches to divide on an image.”
• AdaViT: Adaptive Vision Transformers for Efficient Image Recognition (2021, unknown): “3.3.”
• A Survey of Visual Transformers (2021, methodology): “In the encoder-decoder structure, the learned positional encoding of the decoder (object query [30], [81] or mask embedding [202]) is attached to the decoder inputs explicitly [30], [202] or implicitly [80], [81].”