Small Area Estimation for UK Local Authorities

Small Area Estimation for UK Local Authorities#

Borrowing strength when the sample runs out

A Fay–Herriot area-level model for APS unemployment rates, compared with ONS’s published model-based estimates

Project Overview#

Direct survey estimates of unemployment become too volatile to publish once the Annual Population Survey is broken down to local authorities: the numerator sample is often a handful of unemployed people. ONS has long addressed this with model-based estimates that borrow strength from the claimant count. This project is an open-source reimplementation of that class of model — classical EBLUP and a Bayesian hierarchical version — validated against the Nomis-published official series.

The analysis uses only openly licensed aggregates (no Accredited Researcher / SDS access). Sampling variances \(\psi_i\) are recovered from published 95% confidence intervals. The linking model is the textbook Fay–Herriot specification: a linear mean in the claimant-count rate and iid area random effects.

Project Structure#

Six notebooks build the analysis step by step:

1. Introduction

Explore →

Why direct estimates fail; Fay–Herriot shrinkage; EBLUP vs Bayes

2. Data acquisition

Explore →

Nomis API: APS rates & CIs, claimant count, ONS model-based series

3. Direct estimates

Explore →

Construct $y_i$ and $\psi_i$; instability versus area size

4. Classical EBLUP

Explore →

REML $\sigma_u^2$, Prasad–Rao / Datta–Lahiri MSE, jackknife

5. Bayesian FH

Explore →

Gibbs sampler and PyMC NUTS; posterior of $\theta_i$ and $\sigma_u^2$

6. Validation

Explore →

Direct vs EBLUP vs Bayes vs ONS; choropleths; shrinkage

Key Questions#

  • When does the APS local-authority unemployment rate stop being a publication-grade statistic?

  • How much should a noisy area shrink toward the claimant-count regression, and does that match ONS?

  • Do REML EBLUPs and Bayesian posterior means disagree once \(\sigma_u^2\) is uncertain?

  • Where do an open Fay–Herriot and the official model-based series diverge — and why might that be expected?

Preview Figures#

Shrinkage weight vs sampling variance

Direct-estimate CV vs area size

EBLUP vs ONS model-based estimates

Dataset#

  • Direct estimates: Nomis APS (NM_17_5, unemployment rate aged 16–64) with published confidence intervals

  • Benchmark: Nomis model-based estimates of unemployment (NM_127_1)

  • Auxiliary: Claimant count as % of residents aged 16–64 (NM_162_1); APS inactivity as a sensitivity covariate

  • Boundaries: ONS Open Geography LAD December 2023 BGC

  • Licence: Open Government Licence v3.0 — see data/ATTRIBUTION.txt

Refresh the bundled panel with python projects/small-area-estimation/_build_data.py.

Technical Stack#

Python • Fay–Herriot • REML EBLUP • PyMC • Nomis API • geopandas • Plotly • Official Statistics

Ready to explore?

Start with Introduction →

Contents