Normalized (pointwise) mutual information in collocation extraction

By Gerlof J. Bouma. 2009

1,094 citations recorded by the source corpus.

Subfields: Information Extraction, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Extraction

In Information Theory

Citations in the corpus

A Step Toward More Inclusive People Annotations for Fairness (2021, methodology): “In order to measure the association of attribute labels with missing boxes we use normalized pointwise mutual information (nPMI ) [4].”
Datasheet for the Pile (2022, methodology): “EuroParl: EuroParl has been extensively used, including by [Bouma, 2009, Koehn, 2009, Heafield, 2011, Navigli and Ponzetto, 2012, Cho et al., 2014].”
Measuring Model Biases in the Absence of Ground Truth (2021, methodology): “In order to compensate for this problem, several different normalization techniques have been applied to PMI [4, 21].”
N-gram IDF: A Global Term Weighting Scheme Based on Information Distance (2015, background): “Pointwise Mutual Information (PMI) [9] and their variations are popular and effective for measuring the word compositionality [6, 12, 13, 43, 48].”
Cross-lingual Name Tagging and Linking for 282 Languages (2017, methodology): “We adopt an idea from (Guo et al., 2014) which computes Normalized Pointwise Mutual Information (NPMI) (Bouma, 2009) between a tag and a token: − ln p ( tag, token ) (1) Then we select the sentences in which all annotations satisfy NP MI ( tag, token ) > τ as seeds 5 .”
BERTopic: Neural topic modeling with a class-based TF-IDF procedure (2022, methodology): “For each topic model, its topic coherence was evaluated using normalized pointwise mutual information (NPMI, (Bouma, 2009)).”
Dynamic Patterns of Industry Convergence: Evidence from a Large Amount of Unstructured Data (2015, background): “9 Normalized PMI measures how much two random variables mutually share information or overlapping mutual information between two variables (Bouma, 2009; Church and Hanks, 1990).”
Machine Reading Tea Leaves: Automatically Evaluating Topic Coherence and Topic Model Quality (2014, methodology): “Normalised PMI (NPMI) was ﬁrst introduced by Bouma (2009) as a means of reducing the bias for PMI towards words of lower frequency, in addition to providing a standardised range of [ − 1 , 1] for the calculated values.”
DataLab: A Platform for Data Analysis and Intervention (2022, methodology): “Specifically, we use PMI (Pointwise mutual information) (Bouma, 2009) to detect whether there is an association between two features (e.”
Modeling Islamist Extremist Communications on Social Media using Contextual Dimensions (2019, methodology): “t to different contextual dimensions. To determine the language characteristics, we extract n-grams (n=1 to 3) from tweets of users in extremist and non-extremist datasets using the skip n-gram model [16, 42]. In our experiments with n-grams, we have observed that 2- and 3-grams were particularly in”
Diversity, Serendipity, Novelty, and Coverage (2016, methodology): “To measure the pairwise co-occurrence of items, we employed normalized point-wise mutual information (PMI) [Bouma 2009] , which measures the probability of observing specific outcomes of two independent random variables together.”
Information-theoretical label embeddings for large-scale image classification (2016, methodology): “We consider the matrix of pointwise mutual information PMI [4] between pairs of Lk variables as measured in the ground-truth observations: PMI = log p(Li, Lj) p(Li)p(Lj) The PMI matrix is symmetric definite positive and can be expressed via singular value decomposition as: PMI = U · Σ · U t where Σ ”
An analysis of the coherence of descriptors in topic modeling (2015, result): “As before, these were correlated with human judgements, in addition to the PMI measure, a normalized variant of PMI (Bouma, 2009) (NPMI, range = ½(cid”
Unsupervised Cross-Domain Word Representation Learning (2015, methodology): “We measure the appropriateness of a feature as a pivot according to the score given by: score( x ) = min (NPMI( x, S ) , NPMI( x, T )) .”
A Frame of Mind: Using Statistical Models for Detection of Framing and Agenda Setting Campaigns (2015, positive): “A heatmap of within statement topic cooccurrence based on normalized Pointwise Mutual Information (nPMI) (Bouma and Gerlof, 2009) is presented in Figure 4.”
Full-Text or Abstract? Examining Topic Coherence Scores Using Latent Dirichlet Allocation (2017, background): “NPMI, in contrast to pointwise mutual information (PMI), shows a higher correlation with human topic ranking data [35].”
Exploring the Space of Topic Coherence Measures (2015, background): “…)) + log( P ( ball | sport )) + log( P ( team | game )) + log( P ( team | sport )) + log( P ( team | ball )) Stevens et al. [18] found that both—UCI and UMass cohe-rence—perform better if parameter (cid:15) is chosen to be rather small instead of (cid:15) = 1 as in respective original publications.”
Domain-specific Knowledge Extraction from the Web of Data (2018, methodology): “Therefore, we normalize the PMI(p, dspn−1) to NPMI(p, dspn−1) as proposed in [Bouma 2009]. propscore(p, n) = NPMI(p, dspn−1) (4.8) NPMI(p, dspn−1) = log Prob(p,dspn−1)Prob(p)×Prob(dspn−1) −logProb(p, dspn−1) (4.9) Similar to the type-based ranking, if the same relationship appears in multiple hops…”
Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization (2021, background): “normalized pointwise mutual information [4] and cosine similarity to remove noisy pairs with low C+R score.”
Predicting and Characterising User Impact on Twitter (2014, methodology): “In our case, tweet-term similarity is reﬂected by the Normalised Pointwise Mutual Information (NPMI), an information theoretic measure indicating which words co-occur in the same context (Bouma, 2009).”