Virtual Standard Setting: Setting Cut Scores

The study investigated whether cut scores could be set in two synchronous e-communication media.The findings revealed that virtual cut scores set in both media were valid, reliable, and comparable to ones set in face-to-face environment and provides a framework for conducting and evaluating virtual cut score studies.

Virtual standard setting became more popular since the global outbreak of Covid-19 in 2020. Standard setting practitioners needed to conduct either cut score studies and/or linking studies online. The research presented in this book predates Covid-19 and explores virtual standard setting in two e-communication media (audio and video) and then compares them to the face-to-face environment. The interplay of quantitative methods [i.e., classical test theory (CTT) and Rasch measurement theory (RMT)] and qualitative methods [(i.e., constant comparative method (CCM), and media naturalness theory (MNT)] unravel Ariadne's thread into the labyrinth of virtual standard setting. Illustrative examples of how to conduct and evaluate a virtual workshop are offered to stimulate standard setting practitioners to embrace the opportunities of the virtual environment.

Autorentext
Charalambos Kollias is a research director - psychometrician at an educational research organisation. He has worked in the field of language assessment for over 30 years in roles ranging from examiner trainer, assessment specialist to measurement analyst. He is experienced in conducting and evaluating (virtual) standard setting workshops, linking local and international examinations to global frameworks.

Inhalt

Charalambos Kollias- Introduction - Literature review- Methodology- Cut score data analysis- Survey data analysis- Focus group interview data analysis- Integration and discussion of findings- Implications, future research, and conclusion- Appendices- References- Author index- Subject index