Sparse-MLP: A Fully-MLP Architecture with Conditional Computation

By Yuxuan Lou, Fuzhao Xue, Zangwei Zheng, Yang You. 2021

25 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Barret Zoph
2. Irwan Bello
3. Yi Tay
4. Quoc V. Le
5. William Fedus
6. Yoshua Bengio
7. Mostafa Dehghani
8. Mohit Bansal

Citations in the corpus

• Focal Modulation Networks (2022, unknown): “(3).”
• Designing Effective Sparse Expert Models (2022, methodology): “Lou et al. (2021) similarly showed a benefit over dense vision models by using MoE layers across both image patch and channel dimensions.”
• ST-MoE: Designing Stable and Transferable Sparse Expert Models (2022, methodology): “Lou et al. (2021) similarly showed a benefit over dense vision models by using MoE layers across both image patch and channel dimensions.”