teaching reliability and validity fun with classroom applications

The average effect size was only 0.20, with formative assessment being more effective in English language arts (effect size = 0.32) than in mathematics (effect size = 0.17) or science (effect size = 0.09). They found only 13 studies (42 independent effect sizes) that reported enough information to calculate effect sizes. Kingston and Nash ( Kingston and Nash, 2011) did a new meta-analysis of more than 300 studies on the efficacy of formative assessment. Bennet (2011) noted that most of the original claims of effectiveness in Black and Wiliam’s (1998) review were exaggerated or misplaced. They rightly pointed out that “without a clear understanding of what is being studied, empirical evidence supporting formative evidence will more than likely remain in short supply” (p. Dunn and Mulvenon (2009) focused on the lack of consensus on definition. Recent years have seen repeated challenges to the effectiveness promise of formative assessment. Part of the surge of interest comes from its intuitive appeal part of it comes from claims of its effectiveness in “doubling the speed of student learning” ( Wiliam, 2007, 36–37).

Since Black and Wiliam’s (1998) review article, formative assessment has gained increasing currency in educational systems as different as Australia ( Klenowski, 2011), China ( Xu and Harfitt, 2019), New Zealand ( Bell and Cowie, 2001), Norway ( Hopfenbeck et al., 2015), the United Kingdom ( Torrance and Pryor, 1998) and the United States ( Ruiz-Primo and Furtak, 2007). Teachers may find the framework useful in validating their own CBFA as well. It is posited that a coherent and practical framework, together with its suggested list of inferences, warrants and backings, will help researchers evaluate the usefulness of CBFA. Meas., 2013, 50 (1), 1–73) as a starting point to map out the types of inferences made in CBFA (interpretation and use argument) and the structure of arguments for the validity of the inferences (validity argument).

This article outlines a framework for evaluating the argument-based validity of CBFA.

Many others have conceptualized the validity of formative assessment in different ways (e.g., Nichols et al., 2009, 28 (3), 14–23 Stobart, Validity in formative assessment, 2012, SAGE Publications Ltd, London, UK Pellegrino et al., Educ. In fact, some scholars (e.g., Gipps, Beyond testing: towards a theory of educational assessment, 1994, Falmer Press, London, UK) have even contended for an entirely different paradigm with an independent set of criteria for its evaluation. The embedded and contingent nature of classroom-based formative assessment means that validity in the norm-referenced, summative tradition cannot be understood in exactly the same way for formative assessment. School of Linguistics and Applied Language Studies, Victoria University of Wellington, Wellington, New Zealand.

Teaching reliability and validity fun with classroom applications