Modeling Growth Using Many-faceted Rasch Measurement
Format: 30-minute research presentation
Summary
This presentation will demonstrate the use of Rasch analysis to model growth across repeated classroom assessments. Textbook-based target vocabulary was tested for several consecutive weeks. Substantively large learning growth was measured across the semester, with high reliability indicating extremely precise measurement and well-ordered gains observed across each week of the course. Although this presentation focuses on simple vocabulary review tests, the principles can be applied to any classroom assessment that is administered repeatedly.
Background: Classroom assessment ideally focuses on formative development, part of a set of practices termed assessment for learning (AFL). Dynamic assessment (DA), which focuses on students’ progress (Guàrdia et al., 2016) is a widely practiced form of AFL, but there is a conceptual tension between DA and institutional requirements to provide summative grades and also with the measurement of learning gains.
Method: Weekly vocabulary review quizzes were administered to 447 students at a Japanese university. Target words from each textbook unit were tested for several consecutive weeks and in a final test. This design allowed growth across time to be analyzed as a measurement facet in a many-faceted Rasch measurement (MFRM) analysis (Linacre, 2012).
Research questions:
1. Did students show substantively and statistically significant learning gains across the semester?
2. Did students show cumulatively increasing learning gains each week?
Results and discussion: A large learning gain of 0.90 logits was measured across the semester, with reliability .99 (for Time) indicating extremely precise measurement. More importantly, well-ordered gains were observed across each week of the course, consistent with an effective learning task. Although this presentation focuses on simple vocabulary review tests, the principles can be applied to any classroom assessment that is administered repeatedly, such as in a DA model, including assessments of performances such as writing or speaking assignments. In addition to modelling learning, I will discuss the potential of Rasch
analysis for the diagnostic analysis of students who diverge from the average learning trajectory of the group.
References
Guàrdia, L., Crisp, G., & Jurnet, I. (2016). Trends and Challenges of E-Assessment to Enhance Student Learning in Higher Education. In E. Cano & G. Ion (Eds.), Innovative Practices for Higher Education Assessment and Measurement (pp. 36-56). IGI Global. https://doi.org/10.4018/978-1-5225-0531-0.ch003
Linacre, J. M. (2012). Many-facet Rasch measurement: Facets tutorial.
http://www.winsteps.com/a/ftutorial2.pdf
