Skip to content
Draft: numbers and prose are under review
Contents
Weights, Design, and Estimands

Appendix A

Notation and Crosswalk

One notation for weights, and a dictionary between the survey, econometrics, and causal-inference literatures.

You will need this appendix when a symbol in a chapter is not the one your field uses, or when a referee from another field uses a word you do not: when an economist's "cluster" has to be matched to a survey statistician's "PSU," when [pweight=] and [aweight=] give the same coefficient and different standard errors and you need to know which one asserts what, or when you want the book's nine weight symbols on one page. It is a reference, not a chapter: read the table you need and go back.

One notation runs through the booklet, and it is stricter than software. Stata will accept the ACS person weight as [pweight=perwt], [aweight=perwt], or [fweight=perwt] and return the same regression coefficient three times with three different standard errors, because the three keywords assert three different things about what the column means. This book gives each of those meanings its own symbol and never reuses one for another.

The notation contract

Nine symbols cover every weight the booklet uses. The first four are the stages of a survey weight, from the design's inclusion probability to the number shipped on the file; the next three are the other things software calls a weight; the last two are weights that an analysis constructs rather than receives.

SymbolNameWhat it isWhere it does its work
πi\pi_iinclusion probabilitythe probability that unit ii is selected under the designChapters 0 and 2
di=1/πid_i = 1/\pi_ibase or design weightundoes unequal selection probabilitiesChapter 2
gig_iadjustment factornonresponse and calibration adjustment: raking, poststratification, trimmingChapter 2
wi=digiw_i = d_i g_ifinal survey weightthe weight shipped on the fileChapters 2 and 3
aia_iprecision weightinversely proportional to a row's variance, as for rows that are group meansChapters 2 and 4
fif_ifrequency weightthe number of identical observations a row stands forChapter 2
wi(r)w_i^{(r)}replicate weightthe weight of unit ii in replicate r=1,…,Rr = 1, \dots, RLab 3
sas_astandardization weightthe share of age group aa in a fixed standard populationChapter 3
ωi\omega_iimplicit weightthe weight a regression places on unit ii's conditional effectChapters 4 and 5

A frequency weight changes the number of observations; a precision weight changes the variance attributed to each observation; a survey weight changes the population a sum represents. The three can give the same point estimate and three different standard errors, which is why ww is never overloaded.

The design and the regression each need a few more letters. A stratified two-stage survey has strata indexed by hh and PSUs by jj; a cluster-robust variance has GG clusters of average size mˉ\bar m; a treatment-effect regression has cells xx with a treated share and a contrast in each.

SymbolMeaning
UU, NN; ss, nnthe finite population and its size; the sample and its size
h=1,…,Hh = 1, \dots, Hstrata
j=1,…,nhj = 1, \dots, n_hprimary sampling units (PSUs) sampled in stratum hh
GGclusters in a cluster-robust variance
ρ\rhointraclass correlation
mˉ\bar maverage cluster size
DEFF(θ^)\DEFF(\hat\theta)design effect of the estimator θ^\hat\theta: its design-based variance over its variance under simple random sampling of the same size
nKish\nKish, ndesign\ndesign, nDDC\nDDCthe three scalar effective sample sizes of the Chapter 0 box
DiD_i, XiX_ia binary treatment and covariates
pxp_x, τx\tau_xthe treated share in covariate cell xx; the treated-minus-untreated contrast in that cell
PATE\PATE, ATT\ATT, ATU\ATUthe population average treatment effect; the average effect among the treated; among the untreated

Estimands and populations

Three objects are kept apart throughout. The estimand is the population quantity an analysis targets, defined without reference to any data. The estimator is the rule that maps a sample to a number. The estimate is the number the rule returns on the sample in hand. A weighted and an unweighted mean of the same file are two estimators; whether they target the same estimand depends on the design.

Five populations are named because each gap between them is a place where an analysis can drift:

  • The target population is the one the result is meant to describe.
  • The frame population is the list, area sample, or telephone-number bank the survey draws from.
  • The sampled population is the part of the frame with a positive probability of selection.
  • The respondent population is those in the sampled population who would respond if selected.
  • The analytic population is who remains after the analyst's restrictions and missing-data rules.

The sample, the units actually selected, and the respondents, the selected units that answered, are realizations of the sampled and respondent populations, not populations of their own. Chapter 1 follows all five populations through one estimate.

Coverage honesty follows from the list: when a release omits a state, the target population is the participating jurisdictions, and the result is labeled that way (Chapter 2, Chapter 8).

What software calls a weight

IPUMS's Stata example for the CPS ASEC declares the replicate weights as [iweight=], not [pweight=], because some of them are negative and Stata refuses a negative probability weight. That is the clearest sign that a weight keyword is an assertion about the column, not a formatting choice. The table maps each keyword the booklet's three languages use to the symbol it asserts.

Software syntaxSymbol in this bookWhat the software assumes
Stata [pweight=]wiw_isampling weights; implies a robust (linearized) variance
Stata [aweight=]aia_ianalytic weights inversely proportional to a row's variance; rescaled to sum to the number of rows
Stata [fweight=]fif_iinteger frequency weights; the sample size becomes their sum
Stata [iweight=]noneimportance weights with no fixed statistical meaning; IPUMS uses them for CPS replicate designs because some replicate weights are negative
R survey::svydesign(weights =)wiw_isampling weights inside a declared design
R lm(weights =)aia_iprecision weights: weighted least squares with error variance proportional to one over the weight
Python svy.Design(wgt =)wiw_isampling weights inside a declared design
Python pyfixest.feols(weights =)aia_i by defaultanalytic weights (weights_type="aweights"), or frequency weights with "fweights"; paired with a robust or cluster-robust variance, the coefficients equal the survey-weighted ones
Python statsmodels WLS(weights =)aia_iprecision weights

The table's lesson is narrow. A weighted regression coefficient is the same number whichever of wiw_i, aia_i, or fif_i the software thinks it was given; the standard error is not. Which variance is right depends on what the weight means and on how the data were drawn, which is the subject of Chapter 3 and Chapter 6.

The master crosswalk

A survey statistician's primary sampling unit, an econometrician's cluster, and a trialist's unit of assignment are one object described in three seminar rooms; a stratum, by contrast, has no standard econometric name at all, which is one reason the stratification gain goes missing from cluster-robust standard errors. Each row of the table names one such object in the three dialects. The last column states the condition under which the translation holds; where a condition is missing, the translation is an analogy, not an identity.

Survey statisticsEconometricsCausal inferenceCondition or caveat
finite-population quantity (total, mean, ratio, census regression coefficient)population parameter; probability limit of an estimatorestimand (PATE, ATT, CATE)name the population and the unit before the estimator
inclusion probability πi\pi_isampling rate; selection probabilitypropensity score e(x)e(x)the causal analogy holds for inverse weighting, not for design-based variance
design weight di=1/πid_i = 1/\pi_iprobability weight (pweight)inverse-propensity weight 1/e(x)1/e(x)πi\pi_i is known by design; e(x)e(x) is estimated from a model
nonresponse and calibration adjustment gig_i (raking, poststratification)reweighting to known margins; inverse probability weighting for missing data (Wooldridge 2007)balancing weightscalibrated weights are no longer inverse probabilities; their validity rests on a response or outcome model
Horvitz–Thompson estimator of a totalinverse-probability-weighted suminverse-probability-weighted (Horvitz–Thompson) estimator of mean potential outcomesdesign-unbiased for a finite-population total when every πi>0\pi_i \gt 0 and is known (Horvitz and Thompson 1952)
Hájek (ratio) estimator of a meanweighted mean; WLS on a constantnormalized inverse-probability weightingapproximately unbiased and consistent, not exactly unbiased
stratumnone standard (sampling stratification)block in a blocked designstrata reduce variance when they predict the outcome
primary sampling unit (PSU)clustercluster of jointly assigned unitsclustering is a property of the sampling or assignment design, not of correlated errors alone (Abadie et al. 2023)
intra-PSU correlationintraclass correlation; the Moulton problemintraclass correlation of outcomesthe Moulton problem also needs a regressor that is correlated within clusters
Taylor linearization over PSU totals within strata, with replacementcluster-robust sandwich (CRV1)design-based variance for cluster-randomized designsidentical for a weighted mean with one stratum; with informative strata CRV1 omits the stratification gain (Appendix B)
design effect DEFF(θ^)\DEFF(\hat\theta)ratio of the correct variance to an iid variancenone standardspecific to an estimator and an outcome
clustering component of a design effect for a mean, 1+(mˉ−1)ρy1 + (\bar m - 1)\rho_yMoulton factor for a regression coefficient, 1+(mˉ−1)ρxρu1 + (\bar m - 1)\rho_x\rho_u with equal cluster sizesnone standardMoulton is a special-case regression analogue of the clustering component of a design effect, not the design effect itself (Moulton 1990)
design degrees of freedom: PSUs minus strata (Taylor); replicates minus one (ACS and CPS replicates)G−1G - 1 in few-cluster inferencenumber of randomization unitsa common rule, not a law; NCHS notes it is not directly applicable to the NHIS design
replicate weights (BRR, Fay, jackknife, successive difference, bootstrap)resampling variance estimatorsrandomization inference is a different repetitionreplicate weights are pre-packaged repeated sampling; the five methods are distinct procedures, and none is "a bootstrap" in general (Fay and Train 1995)
declare the design before analyzingcluster at the level of sampling or assignmentanalyze as randomizedidentify every mechanism that induces common uncertainty; the "coarser level" rule holds for nested hierarchies (Abadie et al. 2023; Cameron and Miller 2015)
domain (subpopulation) estimation on the full designsubgroup estimate with full-sample clustering; interaction with a group indicatorconditional average effect for a subgroupdeleting rows gives the same point estimate and can change the variance when strata lose PSUs
frequency weight fif_ifweightnone standardchanges the number of observations
precision weight aia_iaweight; WLS for heteroskedasticity or group meansinverse-variance weights in meta-analysisthe precision motive for weighting (Solon, Haider, and Wooldridge 2015; Dickens 1990)
implicit weights ωi\omega_ivariance weighting of OLS under heterogeneity (Angrist 1998)implied regression weights (Aronow and Samii 2016; Chattopadhyay and Zubizarreta 2023)nonnegative with saturated controls; can be negative with linear controls
standardization weights sas_a (direct age standardization)reweighting to a reference distributionstandardization to a target covariate distributionredefines the comparison population rather than repairing representation
design-based inference: yy fixed, the sample randomsampling-based inference: draws from a populationdesign-based inference: assignment randomwhich repetition is intended decides which variance is right (Abadie et al. 2020)
finite population correctionsampled-cluster fractionfinite-sample (Neyman) variancewhen most clusters are observed, conventional cluster-robust SEs can be conservative
Kish effective sample size nKish\nKisheffective sample size of a weighted estimatoreffective sample size of balancing weightsprices weight dispersion only, under stated conditions (Kish 1965)

A note on the seed crosswalk

The booklet began from an article that compared survey statistics and econometrics in a seven-row table. Its table was right about the direction of the analogy and wrong in several of its equivalences; the corrections are binding everywhere in the booklet, and a reader does not need the article to use this appendix.

The seven corrections, and one about NHANES
The seed saidThe crosswalk saysWhy
Weights get the point estimate right; the design gets the uncertainty rightThe weights help define the estimator; the sampling design and the weight construction determine repeated-sampling variancefinal weights are estimated, and replication that repeats the weighting captures that
The weighted estimator is design-unbiased, however convoluted the samplingCorrect inclusion weights give design-unbiased or consistent estimators for specified finite-population quantitiesthe Horvitz–Thompson total is unbiased; the Hájek mean is consistent; calibrated weights are not inverse probabilities
Design effect = Moulton factor 1+(mˉ−1)ρ1 + (\bar m - 1)\rhoMoulton is a special-case regression analogue of the clustering component of a design effecta design effect also reflects strata and weighting, and the Moulton factor involves the regressor's clustering
Replicate weights are the survey world's pre-packaged cluster bootstrapReplicate weights are pre-packaged repeated sampling; BRR, Fay, jackknife, SDR, and bootstrap are distinctthe ACS and CPS use successive difference replication, not a bootstrap
Taylor linearization is CRV1 with strata as an extra refinement; weights plus clustering on the PSU get you most of the wayThe two coincide exactly for a weighted mean in one stratum; with informative strata CRV1 omits the stratification gain and is conservativestated with its conditions in Appendix B
df = PSUs minus strataA common rule for Taylor linearization; replicate designs use the producer's valueACS replicates carry 7979 degrees of freedomEstimandDesign degrees of freedom for ACS replicate-weight inference (replicates minus 1)VarianceNone reportedSourcedesign.json: ipums/analysis/usa/metadata/acs.design.jsonNote80 replicates - 1, per IPUMS USA repwt FAQ degrees of freedom whatever the number of strata
"Cluster at the level of treatment variation"Identify every sampling or assignment mechanism that induces common uncertainty; the coarsest relevant level governs within a nested hierarchythe full statement is in Chapter 6

The seed also described NHANES as roughly thirty strata of two PSUs each, with about fifteen degrees of freedom. For the August 2021–August 2023 cycle the public file has 1515 strataEstimandMasked variance pseudo-strata, NHANES August 2021-August 2023VarianceNone reportedSourceNHANES August 2021-August 2023 (NHANES/data/processed/nhanes_analysis.parquet) pseudo-strata and 3030 PSUsEstimandMasked variance pseudo-PSUs, NHANES August 2021-August 2023VarianceNone reportedSourceNHANES August 2021-August 2023 (NHANES/data/processed/nhanes_analysis.parquet) pseudo-PSUs, so the design degrees of freedom are 1515 degrees of freedomEstimandDesign degrees of freedom (PSUs minus strata), NHANES August 2021-August 2023VarianceNone reportedSourceNHANES August 2021-August 2023 (NHANES/data/processed/nhanes_analysis.parquet).