What affects the reliability of vocational examiners marking?

Abstract

The results of examinations need to be accepted as just, appropriate and reflect the ability of the candidates sitting an examination, and ‘examiners’ have a key part to play in this process. Examiner’s marking to a common standard and a common interpretation of mark schemes is important so as to not disadvantage or favour clusters of students. In addition to disadvantaging or benefitting those sitting an examination, aberrant marking can also affect the integrity of an award and / or qualification by inflating or deflating pass rates. The purpose of this study was to identify what affects the reliability of vocational examiners marking; with ‘vocational’ being interpreted as subject experts rather than educationalists. Although there has been extensive research into what affects marking e.g. increased monitoring, clearly structured mark schemes, little research has been undertaken as to what awarding bodies using ‘vocational’ subject experts have found to be most effective in improving the reliability of the examiners they use. The assumption of this study was that vocational examiners would need managing differently in terms of selection, training, support and moderation so as to affect the reliability of their marking. What became evident was that the examiners used by the participants in this 2 study needed no more or less management than any examiner, working for any examination board or professional organisation offering public examinations, regardless of their background i.e. not being experienced educationalists. The study identified that whenever examiners are being used and from whatever field e.g. experienced examiners, teachers, lecturers etc, they are all potentially fallible and that they need support, guidance and monitoring to be able to fulfil the task of examining reliably and effectively.