The QAA Code of Practice contains some general guidance as follows:
Internal moderation is important in assuring that examiners apply assessment criteria consistently, and that there is a shared understanding of the academic standards students are expected to achieve. Evidence of moderation is an important feature of internal procedures. Different methods of internal moderation are more or less appropriate for particular situations. In some circumstances, moderation may be limited to sampling a representative number of scripts from a cohort of students, perhaps with emphasis on borderline cases. In other cases, moderation may involve double, or second, marking.
It also suggests a number of factors to be taken into account in formulating policy, including:
- when double or second marking should be used and what approach should be taken, for example, whether or not the second marker normally has access to the first marker’s comments and/or marks and highlighting the importance of demonstrating that double or second marking has taken place
- the methods to be used when assessments from larger groups are sampled by internal or external examiners
- the processes governing and recording any internal moderation and verification of marks and the procedure to be followed when an internal or external moderator disagrees with the original marks
The guidance set out below takes account of this rather general guidance and separates out
(a) assessments in PGT programmes and those which contribute towards a final degree classification for honours and (b) other assessments
Assessments are independently marked by more than one marker with neither having access to the grades or the comments of the other.
This involves independent marking of an assessment by more than one marker. The second marker will have access to the comments of the first marker and to the grades awarded by that marker.
This is really a process of review to check consistency of grades awarded for an assessment, normally through sampling the assessment. This may involve some second marking and will require some additional procedures (e.g. in respect of selection of the sample and resolving disagreements). An important aspect of moderation is checking that feedback has been given consistently by all markers and that any minimum requirements in terms of feedback have been met.
Best practice in moderation
- Every individual summative assessment which forms part of the assessment scheme for a course must be moderated. The process of moderation will depend to some extent on how each particular assessment is marked. Where it has been marked by one marker, then a sample of the marked assessments should be reviewed by another marker, who should also have access to a complete list of the grades awarded for the assessment.
- The sample should consist of 10% of the marked assessment (subject to a minimum of 3 and a maximum of 25) plus all of those assessments which have been graded at Fail or equivalent.
- The sample should cover the whole range of grades awarded by the initial marker.
Where marking is shared between a number of different markers moderation should involve two processes.
- First there should be an initial discussion involving those who are undertaking the marking. This could take place before marking has started and be focussed on an outline answer and marking plan.
- Alternatively, the meeting could take place after a small number of assessments have been graded by each marker and considered by the person with overall responsibility for marking the assessment.
- The second stage in the moderation process will involve a moderator considering a sample drawn as before. This sample should include assessments marked by each of the initial markers.
Both processes of moderation can result in disagreement between the initial marker and the moderator which may take one of a number of forms:
- there may be minor disagreements in isolated cases; or
- there may be a consistent disagreement pointing in one direction; or
- the disagreement may reveal an apparent variability in the standards adopted by the initial marker.
The procedures described below anticipate the possibility of agreement being reached as to what is to be done at this stage with resolution of continued disagreement considered in a later section.
Minor disagreements involve a difference of no more than two secondary bands in the grades awarded for an individual question (even if it goes across the primary band boundary).
Where there are only one or two of these minor disagreements there should be a discussion with the initial marker and, in individual cases the grade may be adjusted with the basis for the agreement on the mark awarded being noted.
If discrepancies are not minor a third person should be brought in to help in the process of resolving the dispute. In cases where there is a consistent disagreement in one direction, a further sample should be selected and reviewed. If this discloses the same consistent variation then there should be a discussion with the initial marker and agreement should be reached on a course of action, which may involve all of the assessments marked by the marker concerned being adjusted as suggested by the consistent variation (e.g. increased by one secondary band).
Where there is variation between the marking of the moderator and that of the initial marker which does not involve consistent variation in one direction a further sample should be reviewed by another moderator. If this suggests a similar pattern all of the grades awarded by the initial marker must be reviewed by him/her in light of the feedback from the two moderators.
Conflicts of interest
Where a first marker has a personal interest in, or relationship with a particular student being assessed they must declare their interest to the Education Director who will ensure that, wherever possible, other members of the team mark and moderate the relevant work. Any conflict of interest must also be recorded on the Record of Moderation form. Where it is not possible to select an alternative maker the assessed work must always be included in the moderation sample for each assessment. Where the assessment contributes to the final award it must always be moderated by the External Examiner. Further guidance on conflicts of interest can be found in E13: Conflict of Interest Policy and para 2.1 D1a: Assessment Procedures
There should be an initial attempt to resolve differences and agree on grades between the moderator and the initial marker (or between the two markers involved in marking a presentation). If this is not possible then the course leader (or, if s/he is one of those involved in the marking/moderation process, the Quality Manager) should seek to establish agreement or, if this is not possible, take a decision on which grades should be awarded. In such cases the sample seen by the External Examiner should include some of those assessments where there was disagreement and the External should be asked for their views on the resolution adopted. The final decision on grades is, of course, for the Board of Examiners in light of the advice of the External Examiner.
Recording of the process
It is important that the moderation process is recorded in some way. This may take various forms, for example:
(a) a moderator writing on non-honours assessments that have been moderated; or
(b) a marker keeping a list of grades and comments.
Where there is disagreement, this should be recorded, as should the process by which the disagreement was resolved and the outcome of this. If agreement was not possible the steps taken, for example reference to the External Examiner, should be recorded.
Guthrie, T., 2011. Moderation and Second Marking. [pdf] Glasgow, University of Glasgow. Available at: http://www.gla.ac.uk/media/media_216411_en.pdf [Accessed 29 March 2017]