Unintended Consequences of using Tests to Improve Learning: How Improvement-Oriented Resources Heighten Conceptions of Assessment as School Accountability

By Gavin T. L. Brown, Lois R. Harris. Journal of MultiDisciplinary Evaluation, 2009

91 citations recorded by the source corpus.

Over the past decade, the New Zealand government has created a set of resources to support teachers’ use of assessment for learning. These include Assessment Tools for Teaching and Learning (asTTle), a software program enabling teachers to create personalized but standardized tests for diagnostic purposes, and Assess to Learn (AtoL), an intensive professional development program. These resources were expected to increase teacher agreement that improvement is the major purpose of assessment. Instead, a 2008 sample of teachers completing the Teacher Conceptions of Assessment questionnaire showed significantly higher agreement that assessment is about school accountability than participants in previous national surveys. Unlike previous surveys, the correlation between school accountability and improvement conceptions was not statistically significant. However, as only the improvement conception predicted the practices teachers used to define assessment (β = .32), it appears that these teachers still saw many of the practices they used in the classroom (e.g. oral and interactive assessments) as improvement-oriented. Interviews with twenty-six of the surveyed teachers identified that while a few saw the new resources as contributing to improvement and accountability purposes, a larger group failed to make that connection. This second group seemed to be unable to accept that tests, an assessment genre traditionally associated with school and student accountability, could be meaningfully used for improvement at the classroom level. These data show that schools and individuals mediate the implementation of any policy initiative and can therefore cause it to have a range of often unintended consequences. In light of this, the thinking of teachers and other educational stakeholders should be taken into account when enacting policy changes.

View Paper: https://www.semanticscholar.org/paper/f4df8edab2224c3043e0296f903475addd406832

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Gavin W. Fulmer
2. I. Lee
3. C. Coombe
4. K. Tan
5. N. Barnes
6. H. Vafadar
7. H. Mohebbi
8. Lingbiao Gao

Citations in the corpus

• The complexity of teachers’ conceptions of assessment: tensions between the needs of schools and students (2009, methodology): “After the second author analysed the questionnaire results, he selected 26 teachers with noticeably different conception profiles for interview (Brown and Harris 2009 provides details of questionnaire results).”
• Multi-level model of contextual factors and teachers’ assessment practices: an integrative review of research (2015, methodology): “In their study of New Zealand teachers’ COAs, Brown and Harris (2009) found that school-level differences in the use of assessment data explained significant variation in the teachers’ COAs.”
• Chinese teachers’ conceptions of assessment for and of learning: Six competing and complementary purposes (2015, background): “…and probably universal tensions between accountability and improvement functions, it is likely that teachers’ conceptions of assessment will reflect the relative emphasis put on either accountability or improvement-oriented purposes of assessment within each system (Brown & Harris, 2009).”
• Mixing interview and questionnaire methods: Practical problems in aligning data (2010, unknown): “Responses to the four conceptions of the TCoA-IIIA inventory were aggregated into profiles relative to national sample norms and compared to previous New Zealand sample populations (see Brown & Harris, 2009).”
• Approaches to Classroom Assessment Inventory: A New Instrument to Support Teacher Assessment Literacy (2016, background): “Brown (2004) and his later work with colleagues (e.g., Brown & Harris, 2009; Brown & Hirschfeld, 2008; Brown, Hui, Flora, & Kennedy, 2011) used the Teachers’ Conceptions of Assessment (COA) questionnaire to determine teachers’ priorities related to four purposes of assessment: (a) improvement of…”
• Assessment in mathematics education: responding to issues regarding methodology, policy, and equity (2018, background): “In this situation, test outcomes might be used primarily for school self-presentation rather than to initiate change processes (Brown and Harris 2009).”
• U.S. teachers' conceptions of the purposes of assessment (2017, methodology): “Research findings using COA Following Brown’s (2004, 2006) initial development of the COAIII, Brown and colleagues (e.g., Brown & Harris, 2009; Brown & Remesal, 2012) and other research teams (e.g., Segers & Tillema, 2011) have used the COA-III to examine teachers' conceptions of assessment in…”
• Language assessment literacy: what do we need to learn, unlearn, and relearn? (2020, background): “Brown (2004) and his later co-authors (e.g., Brown & Harris, 2009; Brown & Hirschfeld, 2008; Brown, Hui, Flora, & Kennedy, 2011) employed the Teachers’ Conceptions of Assessment (COA) questionnaire to specify New Zealand primary school teachers’ and managers’ priorities based on four purposes of…”