Determinantal Point Processes for Machine Learning

By Alex Kulesza, B. Taskar. 2012

1,364 citations recorded by the source corpus.

Determinantal point processes (DPPs) are elegant probabilistic models of repulsion that arise in quantum physics and random matrix theory. In contrast to traditional structured models like Markov random fields, which become intractable and hard to approximate in the presence of negative correlations, DPPs offer efficient and exact algorithms for sampling, marginalization, conditioning, and other inference tasks. While they have been studied extensively by mathematicians, giving rise to a deep and beautiful theory, DPPs are relatively new in machine learning. Determinantal Point Processes for Machine Learning provides a comprehensible introduction to DPPs, focusing on the intuitions, algorithms, and extensions that are most relevant to the machine learning community, and shows how DPPs can be applied to real-world applications like finding diverse sets of high-quality search results, building informative summaries by selecting diverse sentences from documents, modeling non-overlapping human poses in images or video, and automatically building timelines of important news stories. It presents the general mathematical background to DPPs along with a range of modeling extensions, efficient algorithms, and theoretical results that aim to enable practical modeling and learning.

Subfields: Applied Mathematics, Mathematical Physics, Molecular Biology, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Mathematical Physics

In Molecular Biology

In Probabilistic & Bayesian Methods

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “As an aside, it is instructive to note that the herding sample exhibits similar behaviour to those defined under Determinantal Point Process (DPP) prior, i.e., a DPP assigns a high probability to sets of items that are repulsive (Kulesza and Taskar 2012).”
Memory Replay with Data Compression for Continual Learning (2022, neutral): “Just as Kulesza & Taskar (2012) claimed, sparse storage of larger matrices is possible for DPPs, but computing determinants remains prohibitively expensive unless the level of sparsity is extreme.”
Scalable Recollections for Continual Lifelong Learning (2017, background): “We use the dot product of the inputs as a measure of similarity between recollections and found the DPP to achieve effective performance as a diverse sampling step.”
Explaining machine learning classifiers through diverse counterfactual explanations (2019, methodology): “Alternatively, we can build on determinantal point processes (DPP), a probabilistic model that has been adopted for solving subset selection problems with diversity constraints [17].”
Individualness and Determinantal Point Processes for Pedestrian Detection (2016, methodology): “A DPP is a random process used to model particles with repulsive interactions in theoretical quantum physics [8].”
Implicit Diversity in Image Summarization (2019, methodology): “Other efficient methods to ensure non-redundancy in the summary include the use of determinantal point processes [50] and submodular maximization models [86].”
Stochastic Learning on Imbalanced Data: Determinantal Point Processes for Mini-batch Diversification (2017, methodology): “The DPP [19, 25] has been proposed [20, 22, 44] and advanced [4, 23, 24] in the machine learning community in the recent years.”
Adapt to Environment Sudden Changes by Learning a Context Sensitive Policy (2022, background): “{ · In Determinant Point Process (DPP) (Kulesza and Taskar 2012), the diversity of a vector set can be measured by the determinant of a relational matrix constructed from the set: DIV ( { u i } ) = det ( R { u i } ) , where det ( ) denotes the matrix determinant, R { u i } is the relational matrix o”
Improving the Estimation of Word Importance for News Multi-Document Summarization (2014, methodology): “Apart from these, we compare our model with two state-of-the-art systems, including the submodular approach (SUBMOD) (Lin and Bilmes, 2012) and the determinantal point process (DPP) summarizer (Kulesza and Taskar, 2012).”
The Vendi Score: A Diversity Evaluation Metric for Machine Learning (2022, methodology): “The Vendi Score bears a relationship to Deter-minantal Point Processs (dpps), which have been used in machine learning for diverse subset selection (Kulesza et al., 2012).”
Demystifying Orthogonal Monte Carlo and Beyond (2020, methodology): “OMCs are also much simpler than the class of MC methods based on determinantal point processes (DPPs) [28].”
Dialect Diversity in Text Summarization on Twitter (2020, methodology): “[38] Alex Kulesza and Ben Taskar.”
Beyond Pairwise: Provably Fast Algorithms for Approximate k-Way Similarity Search (2013, background): “For usual (real-valued) random projections, it is known that the volume (which is related to the determinant) is approximately preserved [24, 17].”
Generative Knowledge Distillation for General Purpose Function Compression (2017, background): “One approach to achieving this is a Determinantal Point Process (DPP) [14] as recently proposed for selecting diverse neural network mini-batches [32].”
Improving the Estimation of Word Importance for News Multi-Document Summarization - Extended Technical Report (2016, methodology): “Apart from these, we compare our model with two state-of-the-art systems, including the submodular approach (SUBMOD) (Lin and Bilmes, 2012) and the determinantal point process (DPP) summarizer (Kulesza and Taskar, 2012).”