Measuring attainment and disadvantage in schools

And why the 'John Harvey Jones' model of school improvement isn't the way forward.

An awful lot of analysis of the relationship between educational outcomes and income uses eligibility for free school meals (FSM) as its starting point.

But, as researchers will happily point out, there are two problems with this. First, it’s a binary variable - all or nothing. Second, eligibility for free school meals varies massively over time with welfare reforms.

When fewer pupils are eligible, that means it is a poorer, more concentrated group - and vice versa. You can’t draw a straight line with a wonky ruler, and the population of kids on free school meals is a very wonky ruler. That’s true even in aggregate, never mind for subgroups, as this chart from FFT shows:

Other measures are available. You don’t see them so often, as digging them out of the DFE is a pain, but the charts below look at the Income Deprivation Affecting Children Index (IDACI), which avoids the all-or-nothing nature of FSM, and lets us see what happened across the income spectrum.

Over the period 2010-2024 improvement was strongly concentrated in the bottom half of the IDACI distribution. The share achieving grades 4 or above in English and maths GCSEs grew faster among poorer pupils. But the distribution is important. The very bottom decile did better than the top 30%, but the really big gains were in deciles 2-5.

And this wasn’t just grade inflation - in fact it took place in a period where the previous grade inflation was tamed. While GCSEs are not norm referenced (i.e. only X% can get a certain grade) they are now cohort-referenced so that grades are anchored in prior attainment.

The gaps are still too large, and England still has a long way to go to catch up with the leading school systems in Asia, where they simultaneously have very high average achievement and see very few not gaining the basics.

But things in England are much better than in Wales, where the reforms of 1988-2024 were not implemented.

As the IFS note in their report “Major Challenges for Education in Wales”:

PISA scores declined by more in Wales than in most other countries in 2022, with scores declining by about 20 points (equivalent to about 20% of a standard deviation, which is a big decline). This brought scores in Wales to their lowest ever level, significantly below the average across OECD countries and significantly below those seen across the rest of the UK.

Lower scores in Wales cannot be explained by higher levels of poverty. In PISA, disadvantaged children in England score about 30 points higher, on average, than disadvantaged children in Wales. This is a large gap and equivalent to about 30% of a standard deviation. Even more remarkably, the performance of disadvantaged children in England is either above or similar to the average for all children in Wales.

This is a useful reminder that the absolute level is what we really care about. It is better to be disadvantaged at a school in England than average at a school in Wales, which is incredible. Everyone in Britain is increasingly competing in a global market, we can all do better.

So how do we catch up with the Asian nations, which combine the world’s highest levels of achievement with very few kids left behind?

Looking for answers

The current government see the answer as intervention from the centre. People appointed by DFE ‘RISE’ teams will go in to “stuck schools” and give advice. This idea has been tried multiple times in various ways, from “Beacon Schools” to “National Leaders of Education”.

This approach reminds me of the old TV programme “Troubleshooter”, where gloriously rotund industry bigwig Sir John Harvey-Jones would go into struggling firms and tell them what to do, and then sail off into the sunset.

While this was good TV, this is not how real improvement works.

It’s better for a school to become part of a really good trust. When a school is in a trust everyone is on the same side, and the relationship is for the long haul. The people who come to help from a trust aren’t going away. This avoids a perception of “us and them”, with people appearing from outside and then going away again,

There are multiple other problems with the “stuck school” concept.

As Pepe Di’Iasio, general secretary of the Association of School and College Leaders, points out that it’s “unhelpful to persist with labelling hundreds of schools, in a range of different contexts, simply as ‘stuck’”.

Moreover, for the stuck schools Ofsted plans to conduct five monitoring visits “within 18 months, unless the issues have been resolved earlier”.

If you are a young teacher or moving up into a leadership role, will you want to go and teach in a “stuck” school which is about to have five visits from Ofsted?

And there are big questions about the definition too. Analysis by the Times Education Supplement (TES) found that more than half of what DFE calls “stuck” schools have either moved trusts or improved their Ofsted grade since the inspection that put them in that category. So in many cases DFE will be aiming at the tail of the comet, so missing the target.

All this is compounded by the government’s changes to Ofsted which have left neither parents nor school leaders happy, and are likely to make verdicts less reliable.

I am the first to say we needed to go further and will need to if we are returned to government. Not all MATs are good, and more needs to be done to support the growth of the best MATs and school leadership.

More likewise to be done to ensure struggling schools move quickly into MATs that will actually change things. And indeed, to more generally grow trusts to get the economies of scale and depth of support we need. Less than half of schools are academies and three-quarters of MATs run between two and nine schools: only 7% have 20+ schools. Scale opens up not just opportunities for savings but the development of really strong school improvement capacity.

The government’s decision to axe the £126m Trust Capacity Fund is the polar opposite of what is needed, likewise the decisions to end the CEO leadership development programme and the axe the multiple programmes that schools can draw down on for Advanced Maths, Physics, Latin, Computing and Behaviour Hubs.

I cannot pretend to have all the answers at this point. I visited the Harlem Children’s Action Zone years ago and was struck by their efforts to change the wider community and all the things that happen before their high schools, and I have long thought there’s something there. But I know from speaking to numerous school leaders that the current shift away from school-led school improvement to centrally-led is causing a lot of confusion and uncertainty for schools.

One of my enduring memories of school is from around age 14, listening to a classmate who had limited ability to read slowly trying to read out a passage from A Midsummer Night’s Dream. It was excruciating then, and still is now, albeit in a different way. There are still far too many like him for whom we fail to secure the basics. We know it can be different and that things can get better and in recent years we have seen a crop of new schools (particularly free schools) where far fewer are left behind. We must do better - but we know we can.