Urgh, exams. The epic ‘true-false-no idea’ multiple choicers of my undergraduate days are not a distant enough memory for me. The whole ‘get it right, get 1 point’, ‘get it wrong, lose 1 point’ approach always seemed horrendously unfair, regardless of the statistical basis for the strategy (i.e. examiners don’t want to reward people that guess true or false on every question, as in theory they’ll get the right answer 50% of the time).
According to researchers at University of Nottingham, my personal MCQ fear is well founded. Shona Kelly and Reg Dennick have found that male medical students are much more likely to do well on an exam with some ‘true-false-abstain’ questions than are female students.
The authors of this study looked at seven years worth of results from medical school course assessments – which included course work, essays, in-class assessments, lab studies, Objective Structured Clinical Examinations (OSCEs), short answer tests, single phrase tests, spotter quizzes, single word answer exams, true-false-abstain questions, and Vivas.
Among the 359 course assessments, there was a statistically significant difference in the marks between the genders in a third of the courses in any given year. Univariate analyses indicated that women did better in assessments that included some in-class assessment and some short answer questions, but struggled with exams that included true-false-abstain questions. Men did better on assessments with some true-false-abstain questions and were at a disadvantage in those that included some short answer questions.
The female advantage seen with in-class assessment disappeared in multivariate analyses that took into account the subject area/content of the assessment and calendar year, however, and the advantage that females seemed to have in exams with short answer questions was very small (odds ratio 1.03).
On the other hand, the association between true-false-abstain and male advantage wasn’t affected in multivariate analyses. In fact, males were 16.7 times more likely to score higher than females if at least some true-false-abstain questions were in the assessment. This difference could be down to the fact that women were more likely to pick the ‘abstain’ option in this format exam.
So there’s the answer. My difficulty with university exams clearly had nothing to do with my preparation and was obviously down to the inherent gender bias in the format used for biomedical exams. Cough.
Kelly S & Dennick R (2009) Evidence of gender bias in True-False-Abstain medical examinations. BMC Medical Education 9(1). DOI: 10.1186/1472-6920-9-32