Feature Engineering for Predictive Modeling using Reinforcement Learning
By Udayan Khurana, Horst Samulowitz, Deepak S. Turaga. 2017
237 citations recorded by the source corpus.
Feature engineering is a crucial step in the process of predictive modeling. It involves the transformation of given feature space, typically using mathematical functions, with the objective of reducing the modeling error for a given target. However, there is no well-defined basis for performing effective feature engineering. It involves domain knowledge, intuition, and most of all, a lengthy process of trial and error. The human attention involved in overseeing this process significantly influences the cost of model generation. We present a new framework to automate feature engineering. It is based on performance driven exploration of a transformation graph, which systematically and compactly captures the space of given options. A highly efficient exploration strategy is derived through reinforcement learning on past examples.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Marco F. Huber
- Marc-André Zöller
- Xia Hu
- Qingquan Song
- Yi-Wei Chen
- Micah J. Smith
- Chengxiang Zhai
- Kalyan Veeramachaneni
Citations in the corpus
- Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Finally, Khurana et al. (2018a) proposed to create an ensemble of sub-optimal feature sets (see Section 7.4).”
- Survey on Automated Machine Learning (2019, methodology): “By combining the parent’s performance with features of T like node depth or number of different operators leading to the selected node, a policy can be constructed (Khurana et al., 2018a).”
- AutoML to Date and Beyond: Challenges and Opportunities (2020, methodology): “Khurana et al. [34] presented such an approach using reinforcement learning (RL) to automate FE).”
- Data-centric Artificial Intelligence: A Survey (2023, unknown): “Although learning-based methods can search for the optimal preparation strategy automatically [122, 126], it remains a challenge to design the appropriate search space, and the search often requires a non-trivial amount of time.”
- Techniques for Automated Machine Learning (2019, methodology): “Automated Feature Engineering (AutoFE) FeatureRL [18] GP for Feature Engineering [30] Data Science Machine [14] ExploreKit [15] Cognito [19] AutoLearn [16]”