Orthogonal Estimation of Wasserstein Distances

By Mark Rowland, Jiri Hron, Yunhao Tang, Krzysztof Marcin Choromanski, Tamás Sarlós, Adrian Weller. 2019

53 citations recorded by the source corpus.

Wasserstein distances are increasingly used in a wide variety of applications in machine learning. Sliced Wasserstein distances form an important subclass which may be estimated efficiently through one-dimensional sorting operations. In this paper, we propose a new variant of sliced Wasserstein distance, study the use of orthogonal coupling in Monte Carlo estimation of Wasserstein distances and draw connections with stratified sampling, and evaluate our approaches experimentally in a range of large-scale experiments in generative modelling and reinforcement learning.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Krzysztof Marcin Choromanski
2. Adrian Weller
3. Tamás Sarlós
4. Valerii Likhosherstov
5. David Belanger
6. Haoxian Chen
7. Xingyou Song
8. Jared Davis

In Computational Mechanics

1. Krzysztof Marcin Choromanski
2. Adrian Weller
3. Tamás Sarlós
4. David Belanger
5. David P. Woodruff
6. Michael W. Mahoney

In Statistics and Probability

1. Adrian Weller
2. Krzysztof Marcin Choromanski
3. Tamás Sarlós
4. David Belanger
5. Lucy J. Colwell
6. Lukasz Kaiser

In Mathematical Optimization

1. Adrian Weller
2. Krzysztof Marcin Choromanski
3. Tamás Sarlós
4. David Belanger

Citations in the corpus

• Rethinking Attention with Performers (2020, background): “ORFs were introduced to reduce the variance of Monte Carlo estimators (Yu et al., 2016; Choromanski et al., 2017; 2018b; 2019a; Rowland et al., 2019; Choromanski et al., 2018a; 2019b) and we showed in the theoretical and experimental sections from the main body that they do indeed lead to more accur”
• Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers (2020, methodology): “ORFs were introduced to reduce the variance of Monte Carlo estimators [56, 13, 12, 10, 41, 9, 11] and we show in Secs.”
• Demystifying Orthogonal Monte Carlo and Beyond (2020, methodology): “Due to its simplicity and superior performance as compared to its Quasi Monte Carlo counterparts, OMC is used in a wide spectrum of challenging machine learning applications ranging from scalable kernel methods [18] to predictive recurrent neural networks [11], generative models [36] and reinforceme”