Understanding How Raters Communicate in the Context of Medical High-Stakes, Performance-Based Assessments
Loading...
Authors
Sebok, Stefanie Susane
Date
2015-10-01
Type
thesis
Language
eng
Keyword
Rater-Based Assessments , Competency-Based Assessment , Psychometrics , Medical Education
Alternative Title
Abstract
It is well established in the medical education literature that raters represent a significant source of unexplained variance in performance assessments. To better understand the variance associated with raters’ evaluative judgments, this dissertation investigated how raters, in various assessment contexts, communicate and articulate what they observe during performance assessments. Empirical data collected from three different performance assessments: a multiple mini-interview (MMI), an objective structured clinical examination (OSCE), and workplace-based assessments (WBAs) were analyzed to better understand raters’ behaviours. Relationships that exist between different forms of assessment were also examined. The first study established a proof of concept, showing that raters are not always consciously aware of the associations they make during assessments. The second study examined conditions under which raters find it difficult to make a seemingly uncomplicated judgment. Finally, the third study examined relationships that exist between quantitative and qualitative methods used in the evaluation of performance assessments. The empirical findings from this dissertation suggest that although raters may behave idiosyncratically, there exist patterns of behaviour that could be used to attain better information from raters about a learner’s performance and potentially allow raters to be more suitably matched according to their strengths in performance-based assessments. The findings of this research should help to overcome some of the challenges associated with rater-mediated assessments.
Description
Thesis (Ph.D, Education) -- Queen's University, 2015-09-27 17:35:28.998
Citation
Publisher
License
Queen's University's Thesis/Dissertation Non-Exclusive License for Deposit to QSpace and Library and Archives Canada
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.