A look at the new GCSE dictation mark schemes

I’ve been interested to see how the Pearson Edexcel and AQA awarding bodies (exam boards) will mark the dictation section of the Listening exams. As an aside, it’s both interesting and unfortunate that the dictation is part of the Listening paper, since dictation is not much about listening comprehension at all, but about accurate written transcription of words and phrases. Which leads me to the Pearson Edexcel mark scheme… Because the Ofqual/DfE brief, based on the TSC ‘three pillars’ view of language learning (phonics, vocabulary and grammar), is that GCSE should assess students’ knowledge of sound-spelling correspondences (SSCs). So the mark scheme should assess precisely that - how sounds convert to spelling. Pearson have gone to extremes on this, I would suggest.

Now, this creates a problem, particularly for French where the same sounds can be spelled accurately in many ways. Just to take an example the /o/ sound can be spelled variously as o, ot, os, au, aux, eau, eaux, ault, to name just a few. To address this issue, and to ensure that it is SSCs which are rewarded, rather than morphological knowledge, Pearson Edexcel have come up with the following solution. Here are two screenshots from the mark scheme for the Higher Tier Listening paper. 

You’ll see that Pearson give marks for words which are incorrectly spelled, but where the meaning recognisable (in isolation) and the SSC is acceptable. So in Pearson mark scheme if a studen hears intéressant and writes intéresant it is wrong (the 's' is not a correct SSC). If they write intéressent it is alright (phonically sound, though grammatically incorrect and potentially ambiguous). In practice, this may give markers a few headaches, and mark schemes for specific papers may end up providing examples of all the feasible spellings where the meaning may be deemed to be recognisable. Standardisation meetings might be fun.

It’s worth a reminder that dictation is not really about SSCs, so is a dubious way of testing phonics knowledge. When students transcribe a French sentence they are using recall of written words, phonological memory of words and morphological knowledge (e.g. knowing the ils jouent requires ‘nt’ at the end if the verb because of subject-verb agreement). Perhaps a better way to test phonics skill would be to provide non-words where lexical recall and morphological knowledge cannot be used. This approach is used in primary tests, I believe. Would we want to see this is a GCSE exam? I wouldn’t.

So Pearson Edexcel mark as correct spellings which are incorrect. I don’t blame them for this. You can see why they came up with that solution.

How does AQA mark the dictation?

To start with, they take quite a different approach, using a level-based set of descriptors under two categories (Assessment Objectives - AOs). AO1 is Communication of Meaning and AO3 is Transcription and grammatical accuracy. the grid below is followed by two examples of student performance and how it would be marked.

There are pros and cons with the AQA approach. Any level of response-based mark scheme of this type is open to interpretation by the marker, so may be less reliable. On the other hand, there is a greater focus on meaningful communication and some leeway given to the marker to assess the overall ability to recreate the message accurately. The AQA mark scheme rightly acknowledges that there is much more to dictation that the ability to apply SSCs accurately. The examples of student answers are useful too. In theory, the Pearson mark scheme can award equal marks for less grammatically accurate transcriptions.

What does this mean in practice? Let's try out the following French sentence and see what marks would be allocated by Pearson and AQA in a couple of scenarios.

Correct transcription:   Les enfants jouent avec la souris noire.

Imaginary student 1:   Les infants joue avec la souri noir.

    

Pearson - only one word is incorrect in SSC terms (infants), though it is understandable both in isolation and in context. So 6 out of 7 words are rewardable. (I'm assuming souri is okay.) If this level of accuracy were extended over more sentences, using the Pearson grid above, then this students may be scoring 9/10.

AQA -  for AO1 I'd say full marks. For AO3 it looks like a 2 or maybe 3 out of 5. Overall, if that level of performance were extended over more sentences, that looks something like 7/10 or 8/10.

AQA student 2

Look  again at the AQA Student 2 exemplar above. AQA would award 7/10. What would Pearson award for that same student? I can only see one word incorrectly transcribed (in terms of correct SSCs) - pilot. So Pearson would award that candidate 10/10.

Does that mean that the Pearson mark scheme is more generous? Possibly, but you would need to check other scenarios and, in any case, you would need to look across the whole Listening paper to see if one paper were easier than another. Furthermore, you then need to look at the fact that Ofqual do their best to ensure consistency between exam boards, so grade boundaries can be moved. A harder paper does not mean a lower grade. 

Which mark scheme is better? My instinct tells me AQA, since I like that AO1 refers to communication and AO3 acknowledges the role of grammar in dictation, as well as SSC's. On the other hand, the Pearson mark scheme may be more reliable (since AQA's level based grid is open to interpretation - as soon as you use words such as always, frequently, generally, occasionally and rarely, different markers will come up different marks). Standardisation meetings are meant to handle issues like this, but as I said above, level of response-based schemes are inherently not objective. Overall I am uncomfortable with Pearson's position that dictation is all about SSCs. It isn't.

Of course, we are only in the position of evaluating these fine details because dictation is forms 20% of the marks and is in the exam at all! I believe this is a temporary aberration.

Incidentally, AQA also use a level of response-based mark scheme for the Reading Aloud section of the Speaking assessment, again using AO1 (10 marks) and AO3 (5 marks). Pearson also uses a level of response-based grid (AO1 only), but makes specific mention of SSCs in its descriptors. You could see from both this and the descriptors that Pearson is more explicitly aligned with the DfE/Ofqual/Ofsted 'three pillars' language. Pearson spells out that anglicised accents are fine in the read-aloud section as long as the message is clear.