AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction

By Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, Yong Yu. 2020

206 citations recorded by the source corpus.

Learning feature interactions is crucial for click-through rate (CTR) prediction in recommender systems. In most existing deep learning models, feature interactions are either manually designed or simply enumerated. However, enumerating all feature interactions brings large memory and computation cost. Even worse, useless interactions may introduce noise and complicate the training process. In this work, we propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS). AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. In the search stage, instead of searching over a discrete set of candidate feature interactions, we relax the choices to be continuous by introducing the architecture parameters. By implementing a regularized optimizer over the architecture parameters, the model can automatically identify and remove the redundant feature interactions during the training process of the model. In the re-train stage, we keep the architecture parameters serving as an attention unit to further boost the performance. Offline experiments on three large-scale datasets (two public benchmarks, one private) demonstrate that AutoFIS can significantly improve various FM based models. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service, where a 10-day online A/B test demonstrated that AutoFIS improved the DeepFM model by 20.3% and 20.1% in terms of CTR and CVR respectively.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhenguo Li
2. Hang Xu
3. Wei Zhang
4. Lanqing Hong
5. Shifeng Zhang
6. Ning Kang
7. Xinyue Cai
8. Kaigui Bian

In Information Systems

1. Kaigui Bian
2. Jongsoo Park
3. Yuanxing Zhang
4. Wei Yan
5. Wei Zhang
6. Dheevatsa Mudigere
7. Jiyan Yang
8. Tong Zhang

In Management Science and Operations Research

1. Shifeng Zhang
2. Kaigui Bian
3. Yuanxing Zhang
4. Ning Kang
5. Wei Yan
6. Wei Zhang
7. Dheevatsa Mudigere
8. Hang Xu

Citations in the corpus

• VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, background): “…intelligence in various tasks, such as computer vision (He et al., 2016; Ren et al., 2015), natural language processing (NLP) (Yin et al., 2017; Devlin et al., 2018), autonomous driving (Al-Qizwini et al., 2017; Kuutti et al., 2020) and recommender systems (Guo et al., 2017; Liu et al., 2020a).”
• DARTS+: Improved Differentiable Architecture Search with Early Stopping (2019, background): “1 Introduction Neural Architecture Search (NAS) plays an important role in Automated Machine Learning (AutoML), which has attracted a lot of attention recently [3, 8, 20–22, 26, 35, 42].”
• AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System (2020, methodology): “• AutoFIS [41] identifies important feature interactions in the FM model with differentiable architecture search [44], and keeps the architecture parameters as attention weights for the corresponding interactions.”
• DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction (2022, background): “are essential, AutoFIS [22] introduces a gate (in open or closed status) for each feature interaction to control whether its output should be passed to the next layer.”