The Elements of Statistical Learning
By Eric R. Ziegel. 2003
18,764 citations recorded by the source corpus.
Chapter 11 includes more case studies in other areas, ranging from manufacturing to marketing research. Chapter 12 concludes the book with some commentary about the scienti c contributions of MTS. The Taguchi method for design of experiment has generated considerable controversy in the statistical community over the past few decades. The MTS/MTGS method seems to lead another source of discussions on the methodology it advocates (Montgomery 2003). As pointed out by Woodall et al. (2003), the MTS/MTGS methods are considered ad hoc in the sense that they have not been developed using any underlying statistical theory. Because the “normal” and “abnormal” groups form the basis of the theory, some sampling restrictions are fundamental to the applications. First, it is essential that the “normal” sample be uniform, unbiased, and/or complete so that a reliable measurement scale is obtained. Second, the selection of “abnormal” samples is crucial to the success of dimensionality reduction when OAs are used. For example, if each abnormal item is really unique in the medical example, then it is unclear how the statistical distance MD can be guaranteed to give a consistent diagnosis measure of severity on a continuous scale when the larger-the-better type S/N ratio is used. Multivariate diagnosis is not new to Technometrics readers and is now becoming increasingly more popular in statistical analysis and data mining for knowledge discovery. As a promising alternative that assumes no underlying data model, The Mahalanobis–Taguchi Strategy does not provide suf cient evidence of gains achieved by using the proposed method over existing tools. Readers may be very interested in a detailed comparison with other diagnostic tools, such as logistic regression and tree-based methods. Overall, although the idea of MTS/MTGS is intriguing, this book would be more valuable had it been written in a rigorous fashion as a technical reference. There is some lack of precision even in several mathematical notations. Perhaps a follow-up with additional theoretical justi cation and careful case studies would answer some of the lingering questions.
Subfields: Artificial Intelligence, Statistics and Probability, Statistical Learning Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Sergey Levine
- Percy Liang
- Quoc V. Le
- Jianfeng Gao
- Ruslan Salakhutdinov
- Dacheng Tao
In Statistics and Probability
- Francis R. Bach
- Stefan Wager
- Bernhard Schölkopf
- Peter Buhlmann
- Arthur Gretton
- Susan Athey
- Martin J. Wainwright
- Robert Tibshirani
In Statistical Learning Theory
- Francis R. Bach
- Arthur Gretton
- Bernhard Schölkopf
- Kenji Fukumizu
- Bharath K. Sriperumbudur
- Le Song
- Alex Smola
- Martin J. Wainwright
Citations in the corpus
- Implicit Regularization of Discrete Gradient Dynamics in Deep Linear Neural Networks (2019, background): “One of the first results regarding implicit regularization for linear models is the fact that the stochastic gradient descent algorithm finds the minimum norm solution for least-squares [Hastie et al., 2001] which is actually an over-parametrized problem when the dimension d of the input space is la”
- Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (2017, background): “However, it is known that logistic regression works best for likelihood-ratio estimation when comparing two very similar densities (Friedman et al., 2001).”
- True Few-Shot Learning with Language Models (2021, methodology): “Cross-Validation (CV) [22–24] is one of the most widely used methods for estimating generalization loss [25].”
- Deep double descent: where bigger models and more data hurt (2019, neutral): “The bias-variance trade-off is a fundamental concept in classical statistical learning theory (e.g., Hastie et al. (2005)).”
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “Combining the outputs of many models is a foundational technique for improving the accuracy and robustness of machine learning models (Dietterich, 2000; Bauer and Kohavi, 1999; Breiman, 1996; Friedman et al., 2001; Lakshminarayanan et al., 2017; Freund and Schapire, 1997).”
- Focal Loss for Dense Object Detection (2017, methodology): “, Huber loss [12]) that reduce the contribution of outliers by down-weighting the loss of examples with large errors (hard examples).”
- Deep Clustering for Unsupervised Learning of Visual Features (2018, methodology): “Unsupervised learning has been widely studied in the Machine Learning community [12], and algorithms for clustering, dimensionality reduction or density”
- On Calibration of Modern Neural Networks (2017, background): “|X) and n samples, NLL is defined as: L = − n∑ i=1 log(π̂(yi|xi)) (6) It is a standard result (Friedman et al., 2001) that, in expectation, NLL is minimized if and only if π̂(Y”
- Distributional Smoothing with Virtual Adversarial Training (2015, background): “Therefore, regardless of the parametrization, the optimal model distribution trained with LDS regularization is unique.”
- Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models (2015, methodology): “This can be accomplished using dimensionality reduction, clustering, or graph-based techniques [4, 5].”
- Machine Learning Methods That Economists Should Know About (2019, neutral): “” (Hastie et al. [2015], page 24).”
- Protection Against Reconstruction and Its Applications in Private Federated Learning (2018, background): “( The standard approach [40] to such problems is to construct the empirical risk minimizer (cid:98) θ n = argmin θ 1 n =1 ( θ, X i ).”
- Deep learning in neural networks: An overview (2014, methodology): “…to treat SL and UL in the same section: often gradient-based methods, such as BP (Section 5.5.1), are used to optimize objective functions of both UL and SL, and the boundary between SL andULmay blur, for example, when it comes to time series prediction and sequence classification, e.g., Sections”
- Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics (2016, background): “extrapolation in a 4, 000dimensional or 40, 000-dimensional space? It looks like almost everything is extrapolation since the data is inherently sparse in such a high-dimensional space [9, 6], a phenomenon well-known as the curse of dimensionality.”
- On the Art and Science of Machine Learning Explanations (2018, methodology): “Partial dependence (PD) plots are a widely-used method for describing the average predictions of a complex model g across some partition of data X for some interesting input feature X j [10].”
- Distilling a Neural Network Into a Soft Decision Tree (2017, methodology): “Also, they are trained by first picking the size of the tree and then using mini-batch gradient descent to update all of their parameters simultaneously, rather than the more standard greedy approach that decides the splits one node at a time [Friedman et al., 2001].”