Computer Intensive Methods for Testing Hypotheses: An Introduction

By Scott M. Smith, E. Noreen. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback (2018, methodology): “Statistical significance between models is measured using an approximate randomization test (Noreen, 1989).”
End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures (2016, methodology): “Our statistical significance results are based on the Approximate Randomization (AR) test (Noreen, 1989).”
The epiphyte vegetation of the palm Socratea exorrhiza - correlations with tree size, tree age and bryophyte cover (2003, background): “By discarding the six lowest and the six highest values in each category we produced 95% confidence intervals (Noreen 1989).”
A Simple and Effective Hierarchical Phrase Reordering Model (2008, methodology): “7 For testing, Statistical signiﬁcance is computed using the approximate randomization test (Noreen, 1989), whose application to MT evaluation (Riezler and Maxwell, 2005) was shown to be less sensitive to type-I errors (i.e., incorrectly concluding that improvement is signiﬁcant) than the perhaps…”
Controlling the risk of spurious findings from meta‐regression (2004, methodology): “A permutation test may be constructed in the same way as in standard linear regression [26], that is by randomly shu ing the rows of the design matrix and re-assigning them to the response vector.”
Statistical Post-Editing for a Statistical MT System (2011, methodology): “We use approximate randomisation methods (Noreen, 1989) as implemented in FASTMTEVAL (Stroppa and Owczarzak, 2007) to test for statistical significance.”
Learning Structured Predictors from Bandit Feedback for Interactive NLP (2016, methodology): “For statistical significance testing of results against baselines we use Approximate Randomization testing (Noreen, 1989).”
Enhancing clinical concept extraction with distributional semantics (2011, methodology): “Bootstrap Resampling [60] with 1000 repetitions on the test corpus showed the improvement after adding the distributional semantic features derived from clinical trial abstracts is highly signiﬁcant (conﬁ-dence = 100%).”
Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation (2017, methodology): “Statistical signiﬁcance of the out-of-domain system compared to all other systems is measured using Approximate Randomization testing (Noreen, 1989).”
Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition (2011, background): “See Cohen (1995), Noreen (1989) for a discussion of how this affects the validity of the t-test.”
Modelling input texts: from Tree Kernels to Deep Learning (2015, methodology): “To measure if the difference in the observed values of F1 scores of the compared models is statistically significant we employed the implementation [105] of the assumption-free randomization framework [102].”
Semi-Supervised Semantic Role Labeling (2009, methodology): “We performed significance testing on F1 using stratified shuffling (Noreen, 1989), an instance of assumption-free approximative randomization testing.”
Ambiguity-aware Ensemble Training for Semi-supervised Dependency Parsing (2014, methodology): “For signiﬁcance test, we adopt Dan Bikel’s randomized parsing evaluation comparator (Noreen, 1989).”
A Relational View of Information Seeking and Learning in Social Networks (2003, methodology): “QAP and MRQAP are identical to their nonnetwork counterparts with respect to parameter estimates, but use a randomization/permutation technique (Edgington 1969, Noreen 1989) to construct significance tests.”
Commonsense Causal Reasoning Using Millions of Personal Stories (2011, methodology): “For all experiments in this paper, statistical significance is calculated by stratified shuffling (Noreen, 1989).”
Automatic Caption Generation for News Images (2013, methodology): “4 percent and the difference is statistically significant (p < 0:01) using a stratified shuffling-based randomization test [60].”
Modeling Joint Entity and Relation Extraction with Table Representation (2014, methodology): “We performed a statistical test using the approximate randomization method (Noreen, 1989) on our primary measure (“Entity+Relation”).”
More accurate tests for the statistical significance of result differences (2000, methodology): “Then one performs an approximate randomization where each shuffle is performed with random assignments.”