Multi-target regression via input space expansion: treating targets as inputs

By Eleftherios Spyromitros Xioufis, Grigorios Tsoumakas, William Groves, Ioannis P. Vlahavas. Machine-mediated learning, 2012

367 citations recorded by the source corpus.

In many practical applications of supervised learning the task involves the prediction of multiple target variables from a common set of input variables. When the prediction targets are binary the task is called multi-label classification, while when the targets are continuous the task is called multi-target regression. In both tasks, target variables often exhibit statistical dependencies and exploiting them in order to improve predictive accuracy is a core challenge. A family of multi-label classification methods address this challenge by building a separate model for each target on an expanded input space where other targets are treated as additional input variables. Despite the success of these methods in the multi-label classification domain, their applicability and effectiveness in multi-target regression has not been studied until now. In this paper, we introduce two new methods for multi-target regression, called stacked single-target and ensemble of regressor chains, by adapting two popular multi-label classification methods of this family. Furthermore, we highlight an inherent problem of these methods—a discrepancy of the values of the additional input variables between training and prediction—and develop extensions that use out-of-sample estimates of the target variables during training in order to tackle this problem. The results of an extensive experimental evaluation carried out on a large and diverse collection of datasets show that, when the discrepancy is appropriately mitigated, the proposed methods attain consistent improvements over the independent regressions baseline. Moreover, two versions of Ensemble of Regression Chains perform significantly better than four state-of-the-art methods including regularization-based multi-task learning methods and a multi-objective random forest approach.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7c5ccd935b1bf59ff960edd905ffc5cc15d0099b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design (2017, methodology): “2012), and the general stacked single-target framework and ensemble of regressor chains (Spyromitros-Xioufis et al. 2016).”
Better & Faster Large Language Models via Multi-token Prediction (2024, background): “While more specifically tailored architectures for multi-target prediction are conceivable (Spyromitros-Xioufis et al., 2016; Read et al., 2021), modern deep learning approaches usually rely on large shared model trunks with separate prediction heads for the respective tasks (Caru-ana, 1997; Silver…”
Remarks on multi-output Gaussian process regression (2018, methodology): “Model K−1 M nc nSE nnoise MTGP [34] O(N(3)) T (T + 1)/2 d+ 1 T SLFM [47] O(N(3)) QT (T + 1)/2 Q(d+ 1) T CoMOGP [42] O(N(3)) QT (T + 1)/2 (T +Q)(d+ 1) T CONV [58] O(N(3)) T (T + 1)/2 T (d+ 1) T SST [64] O((n+ T )(3)) N/A T (d+ 1) + T (d+ T + 1) 2T ERC [64] O((n+ T )(3)) N/A s[T (d+ 1) + T (T − 1)/2] ”