---

Education: Assessments

Answer

The National Assessment Agency (NAA) is responsible for the standardisation of markers for National Curriculum tests. I have therefore asked David Gee, Managing Director, NAA, to write to you in response to your questions. A copy of his letter has been placed in the Library.Letter from David Gee, dated 29 May 2008:"Further to your two parliamentary questions to the Secretary of State the National Assessment Agency (NAA) has been asked to place this letter in the House of Commons library as a formal response.""Marker Standardisation & Mentoring""The process for the standardisation of markers for National Curriculum Tests has changed in 2008 and therefore there are differences in the way the margin of errors are recorded for the period from 2005 - 2007 and 2008. These are defined in the table enclosed.""From 2005 - 2007 standardisation and the first sample, submitted at the same time, were the main method for detecting markers who were unable to consistently apply the mark scheme. The second sample, two-thirds of the way through the process, was a final check, but it was rare for a marker to be stopped at this stage. This manual process involved supervising markers checking the marking of those in their team and was very dependent on the individual consistency of the team leaders.""From 2008, the introduction of the online standardisation and benchmarking processes means that monitoring of markers has become an ongoing process and poor marking can be identified and corrected at several stages throughout the marking activity. The procedures have been designed to detect those markers that are unable to consistently apply the mark scheme and stop them. It also allows team leaders to identify where markers may require mentoring support to improve their marking to ensure that all markers consistently apply the mark scheme.""The online system for standardisation and benchmarking allows ETS, the test delivery agency, to monitor all markers against an agreed national standard set of scripts at all points in the process; monitoring both marking quality, through benchmarking, and marking progress on a daily basis. This will highlight were additional support may be necessary.""Team leaders should provide feedback on the standardisation papers completed to all markers in their team. Any markers who failed their first attempt at standardisation will receive mentoring ""before a second attempt is made. For those markers who have passed standardisation, feedback should take place either before marking starts or very early in the process. Higher priority is given to those markers who have only just managed to pass standardisation. If after mentoring, a marker fails the second attempt at standardisation or benchmarking, the marker is stopped and removed from marking.""If you are interested further I would be delighted to meet with you once this years tests are completed to give you a full briefing on the work of the NAA in all areas of assessment and school support.""Please contact my office if you would like to arrange this."

Standardisation and Sampling Metrics 2005 to 2007¹
		2005 to 2007¹
			Absolute mark difference allowed
Key stage	Subject	Number of scripts in sample	Discretional second sample	Compulsory second sample
2	English reading	10	6-7	8+
	English writing	10	13-22	23+
	Mathematics	10	16-19	20+
	Science	10	11-23	24+
3	English reading	10	13-22	23+
	English writing	10	24-33	34+
	Mathematics	10	10-19	20+
	Science	10	19-30	31+
¹ Data derived from 2007 NCT Policy Document.

Standardisation and benchmarking metrics 2008
		2008 Standardisation	2008 Benchmarking (current 27 May) and may be subject to further tightening. Benchmarking is conducted typically every 90 scripts
Key Stage	Subject	Number of scripts in sample	Percentage exact agreement	Percentage adjacent	Percentage discrepant	Number of scripts in sample	Percentage exact agreement	Percentage adjacent	Percentage discrepant
2	English	5 reading and 10 writing	80	20	6	90	10
	Mathematics	5	90	0	10	4	97	0	3
	Science	5	90	0	10	4	95	0	5
3	English reading	10	80	20	6	85	15
	English writing	10	80	20	6	90	10
	Mathematics	6	90	0	10	4	97	0	3
	Science	6	90	0	10	4	93	0	7

Secondary information

Type

Written question

Reference

476 c1158-60W; 207868

Session

2007-08

Subjects

Assessments National curriculum tests

Link

View this Written question on www.publications.parliament.uk