Formative Computer-based Assessments: The potentials and pitfalls of two formative computer-based assessments used in professional learning programs

Abstract

A case study approach is used to examine two formative computer based assessments (CBAs) used in a School of Medicine and Faculty of Education at a Canadian university. Both assessments contained primarily scenario-based, multiple-choice items however the number of items and complexity was different. The formative CBAs were examined in terms of how feedback is provided to students, the effectiveness of different types of feedback, and the characteristics of items used for assessment. Feedback was channelled to students in one case, immediately following responding to an item and in the second case, at the end of a set of items. Feedback given to students immediately following an item was more effective given that the timing captured students’ construction of knowledge. The most favoured type of feedback provided additional information. Students also indicated that feedback using a live internet link to direct them to a resource would be acceptable. Although feedback that simply stated a response was correct or incorrect was previously shown to be ineffective, students from the School of Medicine indicated this type of feedback was acceptable for low cognitive items. In both case studies, students reported that more items were required and in one case study, students recommended these items be added at the higher end of the cognitive scale.