Migration: even the data we do have is broken

Another way in which the APS is broken

Previous readers will be familiar with my complaints about the lack of meaningful data on migration, and the lack of any serious assessment of migration policy. We don’t even have up to date estimates of how many non-UK nationals are in the country. The government has ceased to publish basic data on welfare claims and tax payments by nationality, and refuses to conduct the kind of life-cycle analysis many other countries do.

But even what data we do have seems increasingly broken. I was hoping to use the Annual Population Survey (APS) to look at migration trends to different places since the last Census in 2021. In principle the APS can do this: it records information on migration and ethnicity.

However, in practice I think it is bust.

I am obviously not the first to say this. In the last week alone we have seen the Bank of England slate APS for failing to produce accurate labour market data, and the Resolution Foundation warn it has “lost” nearly a million workers and overstated problems with youth employment.

The ONS is well aware of the problems, which are caused by a dramatic drop-off in responses to the survey since the pandemic. The LFS has been downgraded to an “official statistic in development”. They plan to replace it with a streamlined, online-first version (the Transformed Labour Force Survey) - though we learned this week that TLFS might not be fully up and running until 2027.

To say this is a bit of a shame is a massive understatement: the LFS is a workhorse survey used for many years in loads of different types of analysis, and it would have been our only real way into up to date local data on migration. Other datasets like ASHE and RTI only cover people in work.

But from the point of view of migration, as with other issues, the APS seems broken.

Broken data

According to the APS data, the share of London’s population who are ethnic minority and not UK-born did not increase between 2011 and 2024. It was 22.7% and still is apparently.

Despite record amounts of migration, and a record proportion of it coming from non-EU locations… nothing has changed.

To Londoners, this will seem… a bit implausible.

Londoners would be right to feel this is implausible. The Census suggests quite a different trend. While in 2011 the Census and APS estimates were similar, by 2021 a large gap had opened up, with the APS suggesting there were far more white British people in London than the Census:

You can look at the same data as a line chart - the gap has grown:

Though the gap is biggest in London, the issue is present in every region. Karl Williams at the Centre for Policy Studies has compiled this chart looking at the working age population. In 2011 APS and Census numbers are similar. But in 2021 the Census shows more people born overseas than APS in every region:

The Census is obviously way more granular, with a sample size of… everyone.

But it isn’t just that. Other surveys point in the same direction - from NINO registrations, to the HMRC Real Time Information (RTI) data.

The RTI data is shown below. These are different concepts in both dimensions (ethnicity vs nationality, population vs employments). But as a piece of circumstantial evidence it is striking that there is a large increase in the share of employments in London held by non-UK nationals, driven more or less wholly by people from outside the EU (up 6.1%). But over the same period APS shows a very small increase in the share of the working age population not born in the UK (1.6%) and a particularly small increase in non-white people not born in the UK (1.1%).

Karl Williams also highlighted this recent ONS response to a FOI request, which acknowleges the problem - to some extent:

Historically, the LFS has been based primarily on face-to-face interviews at a first wave of contact, followed up by telephone, or face-to-face interviews in subsequent waves. At the start of the COVID-19 pandemic the ONS had to suspend face-to-face interviewing as an initial form of contact and rely on telephone-based collection only.

This change in collection mode for the initial collection wave led to a significant change in the pool of people who responded to the survey. One very noticeable characteristic of this change in non-response bias was the poor response from migrants, leading to implausible movements in our country of birth and nationality statistics. This led to us suspending our labour market statistics relating to levels of migrants in 2021.

Following this initial suspension, a new temporary methodology was developed for the LFS relating to migrant counts, making use of information from HM Revenue & Customs (HMRC) Pay As You Earn (PAYE) Real Time Indicators (RTI) data. This method uses calibration constrained to UK, EU and Non-EU groups, which was identified as the priority level of disaggregation required. Whilst allowing for the production of migrant information for those three groups, this methodology is not designed to support more granular country breakdowns.

I say “to some extent”. Whatever has been to the publicly-available APS data so far has not brought it in line with Census data, nor do I find the results intuitively plausible.

Conclusion

Migration seems like another dimension of the APS which is broken, as the ONS acknowledge to some extent.

But I am not convinced whatever they are doing to patch it up is giving us accurate numbers yet.

This is a massive shame. The UK lacks the most basic data we need to have a sensible conversation about migration. This partly reflects (bad) choices by government not to release data it holds.

But it is very disappointing that the only really up-to-date and comprehensive public data source also seems to be broken, leaving us all guessing.

Surely we can - and must - do better than this?