Appendix B
Scoped Derivations
Linearization, the Moulton factor, implicit regression weights, and four effective sample sizes, each with its conditions.
You will need this appendix when a chapter's claim is one you would have to defend: when a referee asks why a cluster-robust standard error should equal the survey package's, or whether the Moulton factor is "the design effect," or how a regression with positive effects in every cell can return a negative coefficient. Each question has a short answer in the chapter and its conditions here. Come for the derivation you need; the table below says which one that is.
Five derivations carry the chapters' arguments, and none of them is new. What this appendix adds is the scope: each result is stated with the conditions under which it holds, next to the algebra, with a note on what happens when a condition fails. Every identity and approximation was also checked numerically on synthetic data built to satisfy its conditions; Figure B.1 at the end summarizes the checks.
| Derivation | The question it answers | Used in | Kind of result and the condition that matters most |
|---|---|---|---|
| Horvitz–Thompson and Hájek | Why does weighting by inverse inclusion probabilities point an estimator at a population total or mean? | Chapters 0 and 2 | Exactly unbiased for a total, approximately for a mean; needs known, positive inclusion probabilities and complete response |
| Taylor linearization and CRV1 | When does a cluster-robust standard error equal the design-based one? | Chapters 3 and 7 | Identical with one stratum; with several strata, close only in expectation, and only when the strata do not predict the outcome |
| The Moulton factor | How much does clustering inflate a conventional regression variance? | Chapters 3 and 4 | Exact with equal cluster sizes and equicorrelated errors; an approximation otherwise |
| Variance weighting in OLS | Which average of heterogeneous contrasts does a regression on a binary treatment report? | Chapters 0, 4, and 5 | Exact in sample with a saturated set of cells, also for adjusted outcomes when other controls enter additively; with linear controls, weights can turn negative |
| Four effective samples | What does each "effective sample size" measure? | Chapters 0, 2, 3, and 5 | Each holds under its own conditions, and none converts into another |
Horvitz–Thompson and Hájek
A finite population of units carries fixed values . A probability design selects a sample ; unit is included with probability , and units and together with probability . Write and .
The Horvitz–Thompson estimator of the total is design-unbiased (Horvitz and Thompson 1952). The Hájek estimator of the mean is a ratio of two such estimators; it is consistent and approximately unbiased, and it is what software computes when asked for a weighted mean.
Derivation
The expectation is over repeated samples from the design with the held fixed. Its variance is
which can be estimated without bias when every .
The Hájek mean divides by the estimated population size:
The right-hand side is a Horvitz–Thompson estimator of the total of the residuals , divided by . So the Hájek mean is unbiased to first order, with a bias of order , and its variance is approximately the Horvitz–Thompson variance of the residuals. When is roughly constant but varies from sample to sample, as under Poisson sampling, that variance is far smaller than the variance of .
Conditions. The design must give every unit of the target population a positive, known , and the weights must be . Units with are outside what any weighting can recover: that is coverage, not estimation. Responses must be complete and measured without error. Once the weights are with nonresponse and calibration adjustments, the estimator is a calibration estimator: design-consistent under conditions on the control totals, approximately unbiased only if the response model holds, and with a variance that should reflect the calibration, either by linearizing with the calibration residuals or with replicate weights that repeat the weighting. The binding statement follows: correct inclusion weights give design-unbiased or consistent estimators for specified finite-population quantities, and which of the two depends on the estimand and the estimator.
Taylor linearization for a ratio mean, and when it coincides with CRV1
Take a stratified design with strata , with primary sampling units in stratum treated as sampled with replacement, and weights for unit of PSU in stratum . The ratio is estimated by , with and . A weighted mean is the case .
Derivation: the linearized variance
A first-order expansion around gives
Define the linearized values , their weighted PSU totals , and stratum means . The with-replacement variance estimator is
with the common design degrees of freedom (Binder 1983; Wolter 2007).
Now compare the cluster-robust variance of a weighted least squares regression of on a constant, with the PSUs as clusters (), strata ignored, and residuals :
Derivation: when the two coincide
With , , so , the PSU total of the linearized values. The normal equations give exactly.
One stratum. With , and , so
Several strata. Decompose the uncentered sum by strata:
Taylor linearization keeps only the within-stratum part, with the factors . The term is between-stratum variation that the stratified design removed by construction. CRV1 keeps it, so it omits the stratification gain and is conservative when strata predict the outcome. When the stratum means of the linearized totals are near zero, the two estimators differ only through the factors and . With two PSUs per stratum the latter equals two; it is not a small correction, and it compensates, in expectation, for centering each stratum on a two-PSU mean.
Regression coefficients. The same structure holds with in place of the scalar linearized value. With one stratum, CRV1 equals Taylor linearization times .
Conditions. The two are identical in a given sample with one stratum, the same clusters (PSUs nested in strata and identified by the stratum-PSU pair), the with-replacement approximation in both (no finite population correction), the same weights, and either an intercept-only model or acceptance of the factor. With several strata they are not identical. When the strata do not predict the outcome, the two agree approximately in expectation over repeated samples, not sample by sample: CRV1's between-stratum term and Taylor linearization's larger finite-sample factors offset on average. When the strata do predict the outcome, CRV1 keeps between-stratum variation that the design removed, and it is conservative. Intervals also need matching degrees of freedom: PSUs minus strata for Taylor linearization, commonly for CRV1. On real data the difference is visible. For adult cigarette smoking in NHIS 2024, CRV1 on the stratum-PSU key reproduces Taylor linearization with the strata ignored, and it is 1.11×1.11×EstimandCRV1 (composite PSU key) SE divided by the Taylor SE with strata, NHIS 2024 adult smokingVarianceNone reportedSourceIPUMS NHIS 2024 (ipums/analysis/nhis/nhis_core/part_2015_2024.parquet) the Taylor standard error that uses the strata; clustering on the PSU code alone, which merges PSUs from different strata, gives 1.21×1.21×EstimandCRV1 (PSU code alone) SE divided by the Taylor SE with strataVarianceNone reportedSourceIPUMS NHIS 2024 (ipums/analysis/nhis/nhis_core/part_2015_2024.parquet) (Chapter 8).
The Moulton factor as a special case
Predict first
Groups of 20 people share a common shock that accounts for a tenth of the outcome's variance, and the regressor of interest takes one value per group. By how much does the conventional OLS standard error understate the true one?
Moulton's factor answers that question for a regression coefficient (Moulton 1990). The survey design effect answers a related question for an estimator under a sampling design. They coincide in one special case.
Derivation
Let with clusters of equal size and , where the and are independent with variances and , , and .
When is constant within clusters, ,
so
The denominator is the conventional OLS variance. With unequal cluster sizes and a regressor that varies within clusters, the standard approximation is
where is the within-cluster correlation of the regressor.
Now take single-stage cluster sampling: clusters of size drawn with equal probability and with replacement, equal weights, and the sample mean of . Its variance relative to a simple random sample of the same size is , with the intraclass correlation of (Kish 1965). The sample mean is the OLS coefficient on a constant, a regressor that is constant within clusters, so Moulton's ratio with reproduces this clustering component.
For the prediction above, the variance ratio is , so the conventional standard error is too small by a factor of about .
Conditions and scope. The factor needs equicorrelated errors with a common (a random-effects structure), homoskedasticity, independent clusters, a fixed regressor, and known cluster sizes. It concerns an unweighted regression coefficient and a model-based repetition, while the design effect concerns an estimator under a sampling design. A full design effect also reflects stratification, which usually lowers variance, and unequal weighting, which Kish's approximates under conditions of its own. The product is a model-based approximation to the whole, with its own assumptions (Gabler, Häder, and Lahiri 1999). The accurate sentence is therefore the binding one: Moulton's factor is a special-case regression analogue of the clustering component of a design effect, not the design effect itself.
Variance weighting in OLS with a binary treatment
With a binary treatment and a discrete covariate whose cells are , the saturated regression gives a coefficient that is a variance-weighted average of within-cell contrasts (Angrist 1998):
Derivation
By the Frisch–Waugh–Lovell theorem, is the coefficient from regressing on , the residual of on the cell dummies. Because the dummies are saturated, exactly, so
and averages these over the cells. In a sample the identity is exact, with cell counts, treated shares, and differences of cell means in place of the population quantities.
The weights favor cells in which treatment is split evenly; cells in which nearly everyone or nearly no one is treated contribute almost nothing, whatever their size. The targets one might have wanted weight cells differently: a population average effect weights by , the effect on the treated by , which is proportional to . A constant , or the same treated share in every cell, makes all three coincide. Equal treatment variance, the same in every cell, is weaker. It makes the regression's weights proportional to , so the regression then targets the population average effect, but it does not bring the effect on the treated along: cells with treated shares and have the same variance and different weights .
Chapter 5 adds age, age squared, and sex to the cell dummies, entered additively. The identity survives, for adjusted outcomes.
Derivation: additive controls alongside the cells
The model is . Because minimizes the (weighted) sum of squared residuals jointly, also minimizes it with held at . That restricted problem is the saturated regression of the adjusted outcome on and the cell dummies, to which the derivation above applies, with or without survey weights. So is the variance-weighted average of within-cell differences in mean adjusted outcomes, exactly in sample. The contrasts are net of one profile fitted on the whole sample, whose coefficients depend on how and covary within cells; they are not the contrasts that separate regressions within each cell would give.
Survey weights change the distribution the average is taken over, not its form. Weighted least squares with inclusion weights replaces the sample's cell shares and treated shares with their population counterparts, so the survey-weighted coefficient estimates the population version of the variance-weighted average: the population linear projection coefficient of this saturated model, which in general is not the population average effect. Solon, Haider, and Wooldridge put the point in estimand terms: under unmodeled heterogeneity, weighting delivers the population projection rather than the population average of the heterogeneous effects, and the constructive response is to model the heterogeneity and then summarize it over the population of interest (Solon, Haider, and Wooldridge 2015). A gap between the weighted and unweighted coefficients is then a diagnostic, of heterogeneity interacting with the design, of misspecification, or of endogenous sampling, rather than a verdict on which is right (DuMouchel and Duncan 1983).
Conditions. The result requires a binary treatment and a saturated set of controls for a discrete covariate; it is exact in sample and holds in the population under the usual large-sample conditions. Reading as a causal effect requires conditional ignorability; without it the are descriptive within-cell contrasts, which is how Chapters 4 and 5 use them.
With controls entered linearly rather than saturated, Aronow and Samii show that if is linear in , the coefficient converges to with , whose conditional mean is (Aronow and Samii 2016). That is the regression-effective sample of Chapter 5. Słoczyński shows the complementary result that, with linear controls, the coefficient is approximately a convex combination of the effects on the treated and the untreated in which the weight on the effect on the treated is roughly the untreated share: the smaller group gets the larger weight (Słoczyński 2022).
Predict first
Every cell's treatment effect is positive: one, one, and five. Can the OLS coefficient on the treatment be negative?
It can, once the control is linear and the treatment probability is not: with three cells whose effects are one, one, and five, a linear control and a treatment probability that bends produce a negative coefficient.
The arithmetic of a negative implicit weight
Let denote the linear projection of on . If treatment is ignorable given and is linear in , then
and the implicit weight is negative wherever . Take with equal probabilities and . The projection is , so , and the normalized weights are . With cell effects ,
Every effect is positive and the coefficient is negative. Saturating the control restores the nonnegative weights . With linear controls, the implied weights on individual units can also be negative, the regression extrapolating beyond the data, which is why they are worth computing (Chattopadhyay and Zubizarreta 2023). The binding statement follows: equal case weights do not imply equal weighting of heterogeneous conditional effects.
Four effective samples
The Kish effective sample size
Hold the weights fixed and let the be independent with common variance , unrelated to the weights. Then
where uses the divisor . The second equality follows from and .
Conditions: an outcome unrelated to the weights, no clustering, and weights treated as fixed. When the weights are correlated with the outcome, as with selection proportional to a size that predicts the outcome, weighting can reduce variance and Kish's formula misleads. It never says anything about bias (Kish 1965).
The design effective sample size
The design effect of an estimator is
where the denominator is the variance under simple random sampling of the same size, with or without replacement; the choice should be stated. The numerator carries clustering, stratification, and weighting at once, and it depends on the outcome through its intraclass correlation and its relation to the strata and weights. A survey therefore has one design effect per estimator and outcome, not one per file.
The regression-effective sample
Under the conditions of the Aronow–Samii result above, the coefficient is with the squared residual from regressing on . The regression-effective sample is the analytic sample reweighted by . It is summarized by comparing -weighted covariate means with population means, not by a count. A Kish-type scalar measures how concentrated the implicit weights are, not how precise the coefficient is, and with a misspecified treatment model the weights on cells can be signed.
Meng's data-defect effective sample size
Let indicate that unit of the finite population is recorded, , and . With population moments computed with divisor ,
an algebraic identity that holds for every recording pattern (Meng 2018). Averaging its square over the recording mechanism gives a mean squared error of . A simple random sample of size has variance close to when is small relative to , so the sample of equal mean squared error has size
With , a recorded fraction , and a correlation of between recording and the outcome, .
Conditions: the estimand is a finite-population mean; the identity is exact, but the correlation is unknown and must be bounded with outside information; weighted estimators have an analogous identity with the recording indicator replaced by a weighted one. The Kish effective sample size prices weight dispersion and the design effective sample size prices the whole design, both under the assumption that the design is a probability design; the data-defect effective sample size prices what happens when it is not.
Numerical checks
Numerical checks of the stated results
Each derivation's claim, checked on synthetic data built to satisfy its conditions
| Block | Claim as stated | Check | Result | Status |
|---|---|---|---|---|
| B.1 | HT total is design-unbiased when every pi_i > 0 and is known | Poisson sampling, 20,000 draws, outcome unrelated to pi | HT mean minus T = -0.88 Monte Carlo SEs | consistent |
| B.1 | Hajek mean is not exactly unbiased; its variance can be far below HT/N's | same draws, outcome unrelated to pi | Hajek bias -0.0002 (MC SE 0.0006); Var(HT/N) / Var(Hajek) = 102.15 | consistent |
| B.1 | HT total is design-unbiased when every pi_i > 0 and is known | Poisson sampling, 20,000 draws, outcome proportional to pi | HT mean minus T = -0.82 Monte Carlo SEs | consistent |
| B.1 | Hajek mean is not exactly unbiased; its variance can be far below HT/N's | same draws, outcome proportional to pi | Hajek bias +0.0068 (MC SE 0.0008); Var(HT/N) / Var(Hajek) = 1.42 | consistent |
| B.2 | One stratum, weighted mean: Taylor (with replacement) equals CRV1 exactly | hand formula vs svy vs pyfixest CRV1, 40 PSUs | svy vs hand 0.0e+00; CRV1 vs svy 0.0e+00 (relative) | consistent |
| B.2 | Informative strata: CRV1 omits the stratification gain and is conservative | outcome with stratum means 0, 3, 6, 9 | CRV1 SE / Taylor SE = 2.22; with uninformative strata 0.98 | consistent |
| B.2 | Regression (K = 2): CRV1 = Taylor x (N - 1)/(N - K), a small-sample factor | slope SE: svy GLM vs hand sandwich vs pyfixest CRV1 | svy vs hand 1.3e-16; CRV1 / svy = 1.000748 vs sqrt((N-1)/(N-K)) = 1.000748 | consistent |
| B.3 | Moulton: Var(OLS slope) / conventional OLS variance = 1 + (m - 1) rho_x rho_u | equal clusters (m = 20), rho_u = 0.1, regressor constant within clusters | Monte Carlo ratio 3.01 vs formula 2.90 | consistent |
| B.3 | Moulton: Var(OLS slope) / conventional OLS variance = 1 + (m - 1) rho_x rho_u | equal clusters (m = 20), rho_u = 0.1, regressor half within-cluster | Monte Carlo ratio 1.89 vs formula 1.93 | consistent |
| B.4 | Saturated OLS slope = sum over cells of n_x p_x(1 - p_x) Delta_x / sum n_x p_x(1 - p_x), exactly in sample | three cells, p = 0.1, 0.5, 0.9; unweighted | |OLS - formula| = 4.4e-16 | consistent |
| B.4 | Same identity for WLS with the weighted cell shares and treatment rates | same data with unequal weights | |WLS - formula| = 2.2e-16; WLS 1.955 vs weighted cell-average (PATE analogue) 1.908 vs ATT analogue 2.440 | consistent |
| B.4 | Linear control, nonlinear propensity: implicit cell weights p(x)(1 - L(x)) can be negative | X in {0,1,2}, p = 0.05, 0.90, 0.95, cell effects 1, 1, 5; n = 2,000,000 | weights +0.140, +1.131, -0.271; population slope -0.0857, simulated OLS -0.0936 | consistent |
| B.5 | n_Kish = (sum w)^2 / sum w^2 = n / (1 + CV(w)^2) | 5,000 lognormal weights | relative difference 1.8e-16 | consistent |
| B.5 | Meng: sample mean error = rho(R,Y) x sqrt((1 - f)/f) x sigma_Y, an identity | N = 10,000, recording depends on Y | error 7.9064 vs identity 7.9064 | consistent |