I watched some of the Education Select Committee hearing with Glenys Stacey this morning, having followed this story with great interest from the night before grades were originally released. There was already a storm brewing on Twitter when Heads had their sneak preview of results.
Ofqual have changed their policy on grading. After years of undoubted grade inflation they have decided to impose a policy of "comparable outcomes". In practice what this means is that Ofqual look at prior attainment data (KS2 SATS performance) and other factors to do with the cohort for that year and pretty much predetermine what the grades will be. They make a prediction which exam boards have to stick closely to. If the boards disagree with Ofqual, as Edexcel clearly did, they have to tow the Ofqual line.
This year, for English, Ofqual took into account KS2 data and, importantly, the lower number of candidates from schools who had entered students for the IGCSE - principally independent schools - and produced a prediction for grades to which boards had to adhere. The result was that grades fell for the first time in many years (ever?) and not just at the C/D borderline.
So let's be clear: what we now have is effectively an adjusted norm-referenced system of grading. More precisely it is described, so I have learned, as limen referencing. * Outcomes are largely predetermined by prior attainment at KS2 and assumptions about predictable progress between KS2 and the end of KS4. To me this looks like secondary school teachers are not allowed, on average, to add significant value.
I would ask whether we can be sure that the KS2 data are accurate. They are not influenced by prior attainment data from a younger age so we are depending on tests being consistent and raw marks and levels being accurate. History has shown us that KS2 marking is sometimes inaccurate. In addition, children from independent schools and from Wales do not do KS2 tests at all.
Experienced heads and heads of English report that many of their students were unfairly graded this year. They argue that they know what C grade work looks like and many "C grade" candidates were awarded Ds. Ofqual defend their position by pointing to the numbers. This year, overall, results fell by only 1.5% with a cohort missing a significant proportion of more able candidates, so they argue. Edexcel disagreed with Ofsted citing in the recently leaked letters their own statistical evidence.
We can also safely assume that the other exam boards, not just Edexcel, would have given higher grades had it not been for the intervention of Ofqual. Experienced examiners are not being trusted to do their job.
There are interesting consequences to this new era of "comparable outomes". If, generally speaking, results will not increase, then it will appear impossible for schools, on average, to show improvements in accountability measures such as Raise Online and league tables. If one school improves, it will have to be at the expense of another. On average teachers and schools will be wasting their time if they think they can improve results overall.
The analogy with athletics is often mentioned. Not allowing students to improve is like saying to a sprinter that they cannot break the world record given their previous performances. I am not sure this is a fair analogy, since noone is bothered about world records inflation, whereas there is a genuine concern about grade inflation and the alleged "race to the bottom", as Glenys Stacey put it today.
But if we are to have a kind of norm-referencing let's be honest about it and say that only a certain number can pass with the same proportion of candidates at each grade each year.
English has been highlighted in recent weeks, but languages are not immune from comparable outcomes. But with languages the cohorts have changed in size and nature so much over the years that any unfairness or inconsistency has been more masked. What we do know for sure is that languages continue to get an even worse deal than English as far as grading is concerned.
* For a detailed study of types of assessment see:
http://www.cambridgeassessment.org.uk/ca/digitalAssets/113939_What_Happened_to_Limen_Referencing._An_Exploration_of_how_th.pdf
Ofqual have changed their policy on grading. After years of undoubted grade inflation they have decided to impose a policy of "comparable outcomes". In practice what this means is that Ofqual look at prior attainment data (KS2 SATS performance) and other factors to do with the cohort for that year and pretty much predetermine what the grades will be. They make a prediction which exam boards have to stick closely to. If the boards disagree with Ofqual, as Edexcel clearly did, they have to tow the Ofqual line.
This year, for English, Ofqual took into account KS2 data and, importantly, the lower number of candidates from schools who had entered students for the IGCSE - principally independent schools - and produced a prediction for grades to which boards had to adhere. The result was that grades fell for the first time in many years (ever?) and not just at the C/D borderline.
So let's be clear: what we now have is effectively an adjusted norm-referenced system of grading. More precisely it is described, so I have learned, as limen referencing. * Outcomes are largely predetermined by prior attainment at KS2 and assumptions about predictable progress between KS2 and the end of KS4. To me this looks like secondary school teachers are not allowed, on average, to add significant value.
I would ask whether we can be sure that the KS2 data are accurate. They are not influenced by prior attainment data from a younger age so we are depending on tests being consistent and raw marks and levels being accurate. History has shown us that KS2 marking is sometimes inaccurate. In addition, children from independent schools and from Wales do not do KS2 tests at all.
Experienced heads and heads of English report that many of their students were unfairly graded this year. They argue that they know what C grade work looks like and many "C grade" candidates were awarded Ds. Ofqual defend their position by pointing to the numbers. This year, overall, results fell by only 1.5% with a cohort missing a significant proportion of more able candidates, so they argue. Edexcel disagreed with Ofsted citing in the recently leaked letters their own statistical evidence.
We can also safely assume that the other exam boards, not just Edexcel, would have given higher grades had it not been for the intervention of Ofqual. Experienced examiners are not being trusted to do their job.
There are interesting consequences to this new era of "comparable outomes". If, generally speaking, results will not increase, then it will appear impossible for schools, on average, to show improvements in accountability measures such as Raise Online and league tables. If one school improves, it will have to be at the expense of another. On average teachers and schools will be wasting their time if they think they can improve results overall.
The analogy with athletics is often mentioned. Not allowing students to improve is like saying to a sprinter that they cannot break the world record given their previous performances. I am not sure this is a fair analogy, since noone is bothered about world records inflation, whereas there is a genuine concern about grade inflation and the alleged "race to the bottom", as Glenys Stacey put it today.
But if we are to have a kind of norm-referencing let's be honest about it and say that only a certain number can pass with the same proportion of candidates at each grade each year.
English has been highlighted in recent weeks, but languages are not immune from comparable outcomes. But with languages the cohorts have changed in size and nature so much over the years that any unfairness or inconsistency has been more masked. What we do know for sure is that languages continue to get an even worse deal than English as far as grading is concerned.
* For a detailed study of types of assessment see:
http://www.cambridgeassessment.org.uk/ca/digitalAssets/113939_What_Happened_to_Limen_Referencing._An_Exploration_of_how_th.pdf
Comments
Post a Comment