Model complexity of deep learning: a survey

By Xia Hu, Lingyang Chu, Jian Pei, Weiqing Liu, Jiang Bian. Knowledge and Information Systems, 2021

399 citations recorded by the source corpus.

Model complexity is a fundamental problem in deep learning. In this paper, we conduct a systematic overview of the latest studies on model complexity in deep learning. Model complexity of deep learning can be categorized into expressive capacity and effective model complexity. We review the existing studies on those two categories along four important factors, including model framework, model size, optimization process, and data complexity. We also discuss the applications of deep learning model complexity including understanding model generalization, model optimization, and model selection and design. We conclude by proposing several interesting future directions.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/932b41cd3d904f3158daa352ddfc0ee454d4eb15

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bo Li
2. Bowen Zhou
3. Jinfeng Yi
4. Ahad Nasab
5. Shuai Di
6. Jingen Liu
7. Jiquan Pei
8. Bo Liu

Citations in the corpus

• Trustworthy AI: From Principles to Practices (2021, background): “Recent research on the complexity of ML models [162] and their cognitive functional complexity [347] may inspire future research on a unified quantitative evaluation metric.”
• Evaluation of artificial intelligence techniques in disease diagnosis and prediction (2023, background): “Even though various researchers have recently addressed the issue of model complexity in DL [127–130], there is still a need for further investigation and effort in this area [131].”