New research has found that the UK Clinical Aptitude Test (UKCAT), introduced to level the playing field in selection for medical and dental schools, favours male applicants, white people, and students from a higher socioeconomic class or who attended an independent or grammar school.
In the UK, students take advanced level (A level) exams aged 18, and it is the grades they get in these exams that primarily determine whether they can secure a place at university. A levels are affected by socioeconomic bias though – students who are academically able but whose education has been compromised by attending a bad school can end up performing badly and miss out on a place.
The UKCAT, however, doesn’t examine acquired knowledge and candidates can’t be “coached” to pass, so in theory it should provide a more fair assessment of aptitude than A level grades.
In addition, medical school selectors aren’t just interested in academic ability – they want to pick out students who have the personality attributes that will make them a good doctor, like excellent interpersonal skills and professional integrity.
The various components of the UKCAT – verbal reasoning, quantitative reasoning, abstract reasoning, and decision making – are supposed to be best measures to pick out such traits, so the test should also help determine the students who have the personality attributes to make a great doctor.
Unfortunately, the research by James et al, published in the BMJ, suggests that the UKCAT doesn’t really provide a more “equitable assessment of aptitude” than A levels.
The authors looked at data from applicants to 23 UK medical schools who took the UKCAT in 2006, the year it was first introduced. People who sat the test were asked to supply demographic and socioeconomic data such as their parents’ or carers’ occupation. The UKCAT data was then compared with A level results, the current “gold standard” in selection.
From the 18,582 individuals who took the UKCAT in 2006, the authors identified a subgroup of 9884 (53%) students who lived in the UK and for whom they had results data on at least three recent A levels. There was a modest correlation between A level grades and UKCAT scores, which indicates that the test could be used as an alternative to A level grades in the selection process
As far as bias in A level results went, sex didn’t have much of an effect on whether a student scored AAA or AAB – the minimum requirement for medical school admission. However, white students and those whose parents had a managerial or professional job were more likely to get top scores.
Startlingly, applicants who went to an independent or grammar school were more than twice as likely to get top A level grades than those who went to a comprehensive school, sixth form college, or some other sort of higher education institution.
The UKCAT was slightly less subject to socioeconomic bias than A level results. However, male applicants were more likely to get a high score (i.e. a score within the top 30% out of all applicants) than were females. White students were twice as likely to get a high score than students in other ethnic groups, and “professional or managerial background” and “independent or grammar schooling” were likewise independent predictors of a top score.
So although the test is less biased than A levels, the UKCAT still has “an inherent favourable bias to male applicants and those from a higher socioeconomic class or from independent or grammar schools,” conclude the authors.
One major limitation of the study is that about 30% of participants did not provide socioeconomic data when they sat the UKCAT, so a considerable proportion of applicants covered in the A levels calculations were excluded from the UKCAT analyses. Those who withheld socioeconomic status data were more likely to be male, non-white, and from non-selective schools, and were less likely to have top A levels grades and high UKCAT scores. “Arguably, this group contained those candidates who were more likely to benefit from widening participation,” say the authors.
In a rapid response to the research, Rachel Greatrix, chief operating officer of UKCAT, rather unexpectedly welcomes the findings. “It is reassuring that medical and dental schools can use the test as a reasonable proxy for A levels given the on-going debate regarding the speculative nature of predicted grades and discussions regarding post-qualification admissions,” she says. “However, the fact that UKCAT scores are less subject to bias than A level results alone, potentially indicates that if combined with A level achievement, they may offer a fairer tool for selection.”
James D, Yates J, & Nicholson S (2010) Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study. BMJ 340 (feb16 1). DOI: 10.1136/bmj.c478