Variational Item Response Theory: Fast, Accurate, and Expressive
By Mike Wu, Richard Lee Davis, B. Domingue, C. Piech, Noah D. Goodman. 2020
62 citations recorded by the source corpus.
Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger datasets pose a difficult speed / accuracy challenge to contemporary algorithms for fitting IRT models. We introduce a variational Bayesian inference algorithm for IRT, and show that it is fast and scaleable without sacrificing accuracy. Using this inference approach we then extend classic IRT with expressive Bayesian models of responses. Applying this method to five large-scale item response datasets from cognitive science and education yields higher log likelihoods and improvements in imputing missing data. The algorithm implementation is open-source, and easily usable.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jordan L. Boyd-Graber
- Pedro Rodriguez
- Robin Jia
- Joe Barrow
- John P. Lalor
- Alexander Miserlis Hoyle
- Mohit Bansal
- Yoav Goldberg
Citations in the corpus
- Foundation Posteriors for Approximate Probabilistic Inference (2022, neutral): “In the Appendix (Section A.8), we demonstrate plating on an item response theory (IRT) [21, 31, 51, 66, 67] model, a popular probabilistic program.”
- WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…et al., 2013; Shermis, 2014; Kulkarni et al., 2014; Taghipour and Ng, 2016), estimating student knowledge and ability (Desmarais and Baker, 2012; Wu et al., 2020), identifying students who need assistance (Ahadi et al., 2015), or automatically generating explanations for student submissions…”
- Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards? (2021, background): “The first ameliorates the unavoidable randomness of finite evaluations while the second enables error analysis (Wu et al., 2019) and model probing (Belinkov and Glass, 2019; Zhang et al., 2019).”