Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations
By Hongyan Tang, Junning Liu, Ming Zhao, Xudong Gong. 2020
780 citations recorded by the source corpus.
Multi-task learning (MTL) has been successfully applied to many recommendation applications. However, MTL models often suffer from performance degeneration with negative transfer due to the complex and competing task correlation in real-world recommender systems. Moreover, through extensive experiments across SOTA MTL models, we have observed an interesting seesaw phenomenon that performance of one task is often improved by hurting the performance of some other tasks. To address these issues, we propose a Progressive Layered Extraction (PLE) model with a novel sharing structure design. PLE separates shared components and task-specific components explicitly and adopts a progressive routing mechanism to extract and separate deeper semantic knowledge gradually, improving efficiency of joint representation learning and information routing across tasks in a general setup. We apply PLE to both complicatedly correlated and normally correlated tasks, ranging from two-task cases to multi-task cases on a real-world Tencent video recommendation dataset with 1 billion samples, and results show that PLE outperforms state-of-the-art MTL models significantly under different task correlations and task-group size. Furthermore, online evaluation of PLE on a large-scale content recommendation platform at Tencent manifests 2.23% increase in view-count and 1.84% increase in watch time compared to SOTA MTL models, which is a significant improvement and demonstrates the effectiveness of PLE. Finally, extensive offline experiments on public benchmark datasets demonstrate that PLE can be applied to a variety of scenarios besides recommendations to eliminate the seesaw phenomenon. PLE now has been deployed to the online video recommender system in Tencent successfully.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Model Architectures, Recommender Systems
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Information Systems
In Model Architectures
In Recommender Systems
Citations in the corpus
- DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning (2021, neutral): “Architecture and Dataset: The system consists of a candidate generator followed by a multi-task ranking model, and it adopts a framework similar to [41, 35].”
- Towards Universal Sequence Representation Learning for Recommender Systems (2022, background): “parameters to be tuned are those in 𝒃 and 𝑾 𝑟 Transductive setting.”
- A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, background): “Recently, some works of deep learning-based recommendation, such as MMOE [120], PLE [165], etc., studied how to deploy multiple separate models, each of which is designed towards one objective, and then optimize them together.”
- A Survey on the Fairness of Recommender Systems (2022, background): “References [14, 63] concentrate on fairness in machine learning, but fairness in recommendation is not covered, especially its unique characteristics.”
- A Survey on Mixture of Experts in Large Language Models (2024, neutral): “[124]proposetheProgressiveLayered Extraction(PLE)modelforpersonalizedrecommendations.”
- Mixture-of-Experts in Remote Sensing: A Survey (2026, unknown): “…settings, multi-gate MoE (MMoE) and progressive layered extraction (PLE) architectures use separate gating networks per task, together with shared and task-specific experts, to encourage both parameter sharing and task specialization in recommendation and advertising systems [93, 136].”