Many-facet Rasch measurement

By T. Eckes. Quantitative Data Analysis for Language Assessment Volume I, 2019

174 citations recorded by the source corpus.

This chapter provides an introductory overview of many-facet Rasch measurement (MFRM). Broadly speaking, MFRM refers to a class of measurement models that extend the basic Rasch model by incorporating more variables (or facets) than the two that are typically included in a test (i.e., examinees and items), such as raters, scoring criteria, and tasks. Throughout the chapter, a sample of rating data taken from a writing performance assessment is used to illustrate the rationale of the MFRM approach and to describe the general methodological steps typically involved. These steps refer to identifying facets that are likely to be relevant in a particular assessment context, specifying a measurement model that is suited to incorporate each of these facets, and applying the model in order to account for each facet in the best possible way. The chapter focuses on the rater facet and on ways to deal with the perennial problem of rater variability. More specifically, the MFRM analysis of the sample data shows how to measure the severity (or leniency) of raters, to assess the degree of rater consistency, to correct examinee scores for rater severity differences, to examine the functioning of the rating scale, and to detect potential interactions between facets. Relevant statistical indicators are successively introduced as the sample data analysis proceeds. The final section deals with issues concerning the choice of an appropriate rating design to achieve the necessary connectedness in the data, the provision of feedback to raters, and applications of the MFRM approach to standard-setting procedures.

Subfields: Applied Psychology, Management Science and Operations Research, Safety Research

View Paper: https://www.semanticscholar.org/paper/0a031924053e2925581bf43c48d18140a0633721

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

1. Shanna N. Ricketts
2. Mei-Lin Chang
3. G. Engelhard

In Management Science and Operations Research

1. Shanna N. Ricketts
2. Hiroki Sayama
3. Mei-Lin Chang
4. G. Engelhard
5. Vahid Aryadoust
6. Li Ying Ng

In Safety Research

1. Shanna N. Ricketts
2. Mei-Lin Chang
3. G. Engelhard

Citations in the corpus

• A comprehensive review of Rasch measurement in language assessment: Recommendations and guidelines for research (2020, background): “Facets is also suitable for conducting differential facet/item functioning (DIF/DFF) to investigate bias in measurement and rater effects (Eckes, 2019).”
• Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application (2020, unknown): “We applied faceted Rasch item response theory (Linacre, 1989), a form of psychometric scaling designed for annotator evaluations, to combine the 10 ordinal components of hate speech into an interval-valued continuous hate speech score - our primary outcome.”
• Development and Validation of a Scale to Measure Academic Resilience in Mathematics (2017, methodology): “The MF model can be used in an exploratory way and can be thought of as an additive linear model that is based on a logistic transformation of observed ratings to a logit scale (Eckes, 2009; Engelhard, 2013).”