Noname manuscript No. (will be inserted by the editor) Adaptive Regularization of Weight Vectors

Citations in the corpus

• Dropout Training as Adaptive Regularization (2013, methodology): “In contrast, L 2 -regularization makes no attempt to sphere the level curves, and AROW [18]—another popular adaptive method for online learning—only attempts to normalize the effective feature matrix but does not consider the sensitivity of the loss to changes in the model weights.”
• Measuring and regularizing networks in function space (2018, background): “…that many approaches to normalize and whiten activations or gradients have been forwarded in the literature (Raiko et al. (2012);Simard et al. (1998); Schraudolph & Sejnowski (1996); Crammer et al. (2009); Wang et al. (2013); LeCun et al. (1991); Schraudolph (1998); Salimans & Kingma (2016)).”
• Datasheet for the Pile (2022, methodology): “Enron Emails: The Enron Emails have been used extensively, including by [Diesner et al., 2005, Kossinets et al., 2008, Tang et al., 2008, Crammer et al., 2009, Madjarov et al., 2012].”
• Food image recognition using deep convolutional network with pre-training and fine-tuning (2015, methodology): “For lower-dimensional DCNN features, we use a standard linear SVM, while for higherdimensional FV features, we use an online learning method, AROW [22].”
• Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge (2019, background): “It is based on imitation learning using linear classifiers (Crammer et al., 2009) in a two-level generation approach, where the classifiers first select the next slot to be realised and then the corresponding word-by-word realisation of that”
• Online Learning of Multiple Tasks and Their Relationships (2011, background): “Spam [Crammer et al., 2009] consists of 3000 test and 4000 training examples constructed from email messages of 3 different users (each user is a task).”
• Hope and Fear for Discriminative Training of Statistical Translation Models (2012, methodology): “Keywords: machine translation, structured prediction, large-margin methods, online learning, distributed computing”
• Malicious URL Detection using Machine Learning: A Survey (2017, methodology): “Lastly, we discuss other learning algorithms, in challenges specific to Malicious URL Detection are addressed, including cost-sensitive learning, active learning, similarity learning, unsupervised learning and string pattern matching.”
• Online Learning: A Comprehensive Survey (2018, background): “Other approaches include New variant of Adaptive Regularization (NAROW) (Orabona and Crammer, 2010) and the Normal Herding method via Gaussian Herding (NHERD) (Crammer and Lee, 2010).”
• Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous or Discrete Weights (2014, methodology): “The test results are calculated after each epoch using 8-fold cross-validation, similarly to [7].”