Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction

By Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, Xia Hu. 2020

94 citations recorded by the source corpus.

Click-Through Rate (CTR) prediction is one of the most important machine learning tasks in recommender systems, driving personalized experience for billions of consumers. Neural architecture search (NAS), as an emerging field, has demonstrated its capabilities in discovering powerful neural network architectures, which motivates us to explore its potential for CTR predictions. Due to 1) diverse unstructured feature interactions, 2) heterogeneous feature space, and 3) high data volume and intrinsic data randomness, it is challenging to construct, search, and compare different architectures effectively for recommendation models. To address these challenges, we propose an automated interaction architecture discovering framework for CTR prediction named AutoCTR. Via modularizing simple yet representative interactions as virtual building blocks and wiring them into a space of direct acyclic graphs, AutoCTR performs evolutionary architecture exploration with learning-to-rank guidance at the architecture level and achieves acceleration using low-fidelity model. Empirical analysis demonstrates the effectiveness of AutoCTR on different datasets comparing to human-crafted architectures. The discovered architecture also enjoys generalizability and transferability among different datasets.

Subfields: Computer Vision and Pattern Recognition, Information Systems, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Information Systems

In Optimization & Training

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “Beyond model accuracy and timing performance (Joglekar et al., 2020; Song et al., 2020; Tan & Le, 2020; Eriksson et al., 2021), energy and carbon footprint can be directly incorporated into the cost function as optimization objectives (Yang et al.”
When large language models meet personalization: perspectives of challenges and opportunities (2023, unknown): “Model architecture search, like [170], [171], [172], [173], expands the search space to the integral architectures.”
Understanding Training Efficiency of Deep Learning Recommendation Models at Scale (2020, background): “At Facebook, personalization and ranking use cases are powered by deep learning recommendation models: Instagram stories are ranked with multi-stage deep neural networks (DNNs) [40]; Newsfeed Ranking and Search tasks are also built upon DNNs [20]–[22], [42], [50].”
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery (2020, background): “…2000; Sarwar et al., 2001; Koren et al., 2009) to machine learning (He et al., 2017; Wang et al., 2016; Rendle & Schmidt-Thieme, 2010) and deep learning (Cheng et al., 2016; Naumov et al., 2019; Zhou et al., 2018; Zhang et al., 2019; Guo et al., 2017; Song et al., 2020; Weinberger et al., 2009).”