Large-Scale Machine Learning

By Sanjiv Kumar. 2020

Course Outline: This course will explore the mathematical foundations of a rapidly evolving new field: large-scale machine learning and optimization. We will focus on recent texts in machine learning, optimization, and randomized algorithms, with the goal to understand the tradeoffs that are driving algorithmic design in this new discipline. These trade-offs will revolve around statistical accuracy, scalability, algorithmic complexity, and implementation.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Ji Lin
2. Gong Cheng
3. Song Han
4. Mingxing Tan
5. Tien-Ju Yang
6. Quoc V. Le
7. Zhijian Liu
8. Junwei Han

Citations in the corpus

• MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks (2017, methodology): “We trained with the RMSProp optimizer [29] implemented in Tensorflow [1] with a batch-size of 96 .”
• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (2017, methodology): “We used the RMSprop optimizer from TensorFlow [1] with a batch size of 32 .”
• Deep Leakage from Gradients (2019, background): “There are many works that aim to improve the scalability of distributed training, both at the algorithm level [7,11,16,23,32] and at the framework level [1,2,6,29,33].”
• Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark (2019, background): “have achieved great success in natural scene images (Agarwal et al., 2018; Dai et al., 2016; Girshick, 2015; Girshick et al., 2014; Han et al., 2018; Liu et al., 2018a; Liu et al., 2016a; Redmon et al., 2016; Redmon and Farhadi, 2017; Ren et al., 2017).”
• Deep learning model to predict complex stress and strain fields in hierarchical composites (2021, methodology): “The ML calculations are performed using TensorFlow ( 51 ), a general purpose ML framework.”