Bias in parametric estimation: reduction and useful side‐effects

By Ioannis Kosmidis. 2013

84 citations recorded by the source corpus.

The bias of an estimator is defined as the difference of its expected value from the parameter to be estimated, where the expectation is with respect to the model. Loosely speaking, small bias reflects the desire that if an experiment is repeated indefinitely then the average of all the resultant estimates will be close to the parameter value that is estimated. The current article is a review of the still‐expanding repository of methods that have been developed to reduce bias in the estimation of parametric models. The review provides a unifying framework where all those methods are seen as attempts to approximate the solution of a simple estimating equation. Of particular focus is the maximum likelihood estimator, which despite being asymptotically unbiased under the usual regularity conditions, has finite‐sample bias that can result in significant loss of performance of standard inferential procedures. An informal comparison of the methods is made revealing some useful practical side‐effects in the estimation of popular models in practice including: (1) shrinkage of the estimators in binomial and multinomial regression models that guarantees finiteness even in cases of data separation where the maximum likelihood estimator is infinite and (2) inferential benefits for models that require the estimation of dispersion or precision parameters.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/01f82adff8ac9c7a87396ffdf96f33753ed82c98

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Estimating logit models with small samples (2021, unknown): “4 For details on numerical optimization of the log-likelihood and penalized log-likelihood functions, we refer readers to an especially clear discussion in Gill (2000, Ch. 5) and the references therein and Kosmidis and Firth (2009: 799).”
Analysing continuous proportions in ecology and evolution: A practical introduction to beta and Dirichlet regression (2019, methodology): “See Appendix S3 accompanying Case study 1 for a demonstration of bias correction and bias reduction and a bootstrap‐based technique for assessing the degree of bias for any given data‐model combination (Kosmidis, 2014).”
Median bias reduction of maximum likelihood estimates (2016, methodology): “This lack of coherence is highlighted e.g. in Kosmidis (2014), but somehow overwhelmed by advantages in applications, possibly with a careful choice of the working parameterization (Kosmidis & Firth, 2010, § 4.2, Remark 3).”