# Small Area Estimation for UK Local Authorities
Borrowing strength when the sample runs out
A Fay–Herriot area-level model for APS unemployment rates, compared with ONS’s published model-based estimates
## Project Overview
Direct survey estimates of unemployment become too volatile to publish once the Annual Population Survey is broken down to local authorities: the numerator sample is often a handful of unemployed people. ONS has long addressed this with **model-based estimates** that borrow strength from the claimant count. This project is an open-source reimplementation of that class of model — classical EBLUP and a Bayesian hierarchical version — validated against the Nomis-published official series.
The analysis uses only openly licensed aggregates (no Accredited Researcher / SDS access). Sampling variances $\psi_i$ are recovered from published 95% confidence intervals. The linking model is the textbook Fay–Herriot specification: a linear mean in the claimant-count rate and iid area random effects.
## Project Structure
Six notebooks build the analysis step by step:
1. Introduction
Explore →
Why direct estimates fail; Fay–Herriot shrinkage; EBLUP vs Bayes
2. Data acquisition
Explore →
Nomis API: APS rates & CIs, claimant count, ONS model-based series
3. Direct estimates
Explore →
Construct $y_i$ and $\psi_i$; instability versus area size
4. Classical EBLUP
Explore →
REML $\sigma_u^2$, Prasad–Rao / Datta–Lahiri MSE, jackknife
5. Bayesian FH
Explore →
Gibbs sampler and PyMC NUTS; posterior of $\theta_i$ and $\sigma_u^2$
6. Validation
Explore →
Direct vs EBLUP vs Bayes vs ONS; choropleths; shrinkage
## Key Questions
- When does the APS local-authority unemployment rate stop being a publication-grade statistic?
- How much should a noisy area shrink toward the claimant-count regression, and does that match ONS?
- Do REML EBLUPs and Bayesian posterior means disagree once $\sigma_u^2$ is uncertain?
- Where do an open Fay–Herriot and the official model-based series diverge — and why might that be expected?
## Preview Figures
## Dataset
- **Direct estimates:** Nomis APS (`NM_17_5`, unemployment rate aged 16–64) with published confidence intervals
- **Benchmark:** Nomis model-based estimates of unemployment (`NM_127_1`)
- **Auxiliary:** Claimant count as % of residents aged 16–64 (`NM_162_1`); APS inactivity as a sensitivity covariate
- **Boundaries:** ONS Open Geography LAD December 2023 BGC
- **Licence:** [Open Government Licence v3.0](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/) — see `data/ATTRIBUTION.txt`
Refresh the bundled panel with `python projects/small-area-estimation/_build_data.py`.
## Technical Stack
**Python** • Fay–Herriot • REML EBLUP • PyMC • Nomis API • geopandas • Plotly • Official Statistics
---