Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction

By Kun Gai, Xiaoqiang Zhu, Han Li, Kai Liu, Zhe Wang. 2017

91 citations recorded by the source corpus.

CTR prediction in real-world business is a difficult machine learning problem with large scale nonlinear sparse data. In this paper, we introduce an industrial strength solution with model named Large Scale Piece-wise Linear Model (LS-PLM). We formulate the learning problem with $L_1$ and $L_{2,1}$ regularizers, leading to a non-convex and non-smooth optimization problem. Then, we propose a novel algorithm to solve it efficiently, based on directional derivatives and quasi-Newton method. In addition, we design a distributed system which can run on hundreds of machines parallel and provides us with the industrial scalability. LS-PLM model can capture nonlinear patterns from massive sparse data, saving us from heavy feature engineering jobs. Since 2012, LS-PLM has become the main CTR prediction model in Alibaba's online display advertising system, serving hundreds of millions users every day.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kun Gai
2. Xiaoqiang Zhu
3. Jie He
4. Han Zhu
5. Xiao Ma
6. Han Li
7. Junqi Jin
8. Pengye Zhang

In Computer Vision and Pattern Recognition

1. Kun Gai
2. Xiaoqiang Zhu
3. Jie He
4. Han Zhu
5. Han Li
6. Junqi Jin
7. Pengye Zhang
8. Guorui Zhou

Citations in the corpus

• Deep Interest Network for Click-Through Rate Prediction (2017, methodology): “urse of dimensionality in language modeling. This idea, we name it embedding, has inspired many natural language models and CTR prediction models that need to handle large-scale sparse inputs. LS-PLM [8] and FM [10] models can be viewed as a class of networks with one hidden layer, which rst employs”
• Learning Tree-based Deep Model for Recommender Systems (2018, methodology): “The comparison method is mixture of logistic regression [8] that used to pick out superior results only from those interacted clusters, which is a strong baseline.”
• Optimized Cost per Click in Taobao Display Advertising (2017, unknown): “The time complexity of rank function isO (N ∗ ∥A∥ ∗ log ∥A∥).”