Part I Chapter 2

Where Survey Weights Come From

How one phone-survey respondent comes to stand for thousands of adults: design weights, adjustment, raking, and trimming.

DataBRFSS 2024 · ATUS · effective sample sizes

What is this weight column?

You have decided to use the weights. Now the codebook lists six of them. The 2024 Behavioral Risk Factor Surveillance System (BRFSS) file carries _STRWT, _RAWRAKE, _WT2RAKE, _DUALCOR, _LLCPWT2, and _LLCPWT, and the documentation says to use the last. The American Time Use Survey file carries WT06 and a page of text about weekends. Before you attach either to an estimate, the question worth an hour is: what is this column, and what did the producer do to it? Because the answer decides what the weighted number means, whose chance of selection it undoes, and what it silently assumes about the people who never answered.

Two weights, two logics. The ATUS weight begins as something you can see: BLS assigns half of its diary days to weekends on purpose, and a weekend diary therefore stands for fewer days than a weekday diary. That part of WT06 is an inverse probability, and you could rebuild it from the design description. The BRFSS weight ends as something you cannot see. Somewhere in the 2024 file is a respondent who answered a telephone interviewer's questions and, in the estimates CDC publishes, stands for 49,582 adults. The median respondent stands for 264. The average Vermont respondent stands for 83 adults and the average California respondent for 3,662, because Vermont supplied 6,520 interviews and California 8,382 for adult populations of very different size. These numbers are _LLCPWT, and across the 450,353 respondents in the states and the District of Columbia they add up to 260.9 million adults. That respondent does not have a one-in-49,582 chance of selection. The number is the product of five steps, and only the first three are probabilities.

The weight decides how much each answer counts. Among the 442,292 respondents who gave a valid count of poor mental health days, 13.9% reported frequent mental distress: 14 or more of the past 30 days on which their mental health was not good. Weighted, the same answers give 15.6% (95% CI 15.3%–15.8%). No answer changed. The 1.7 percentage points between the two figures come from the weight. This chapter takes the BRFSS weight apart step by step, asks how much of it can be rebuilt from the public file, sets the ATUS design weight beside it as the case where the probability is visible, and then asks what a person's weight means when it is no longer the inverse of a probability.

15.6%

Frequent mental distress, adults in the 49 states and DC that released 2024 BRFSS data

95% CI 15.3%–15.8%

Frequent mental distress (FMD) is a self-reported count of days, not a clinical diagnosis. BRFSS is the state-based telephone survey run by CDC and state health departments, and every record carries the final weight along with every intermediate weight, so the estimate can be computed at each step of the construction. The first three steps are design weights: they undo known differences in the chance of being sampled, arising from the telephone stratum, the number of adults and phones in the household, and the chance of being reachable on both a landline and a cell phone. The fourth truncates the design weight and puts it on the population scale. The fifth rakes the weights until they reproduce population totals by age, sex, race and ethnicity, education, and other characteristics, state by state.

Five links in a chain

Predict first

Young adults report frequent mental distress more often than older adults and are harder to reach and interview. Which link do you expect to move the estimate most, and in which direction? Does every link push the same way?

Frequent mental distress, one weight at a time

BRFSS 2024, adults in the 49 states and DC that released data; share reporting 14+ poor mental health days

Unweighted
Design-based
Weighted

Show data

Group	Estimate for	Estimate	n
No weights	Unweighted	13.9%	442,292
Rebuilt from public variables	Stratum weight (_STRWT)	14.7%	442,292
Rebuilt from public variables	× adults ÷ phones (_WT2RAKE)	14.4%	442,292
Rebuilt from public variables	× dual-frame factor (_WT2RAKE × _DUALCOR)	15.0%	442,292
Documented by CDC, not rebuildable	Raking input (_LLCPWT2)	14.5%	442,292
Documented by CDC, not rebuildable	Final weight (_LLCPWT)	15.6%	442,292

Figure 2.1.The estimate moves at every link of the weight chain, in both directions, and the final raking step moves it most. Only the final value is an estimate; the intermediate values are diagnostics.Note: Stages 1-3 are rebuilt from public variables; the raking input and final weight are shipped on the file and documented by CDC, but cannot be rebuilt record by record. Intermediate stages are diagnostics, not estimates: their totals are not population counts.Source: CDC BRFSS 2024 public file; weights _STRWT, _WT2RAKE, _DUALCOR, _LLCPWT2, _LLCPWT

The path is not monotone (Figure 2.1). The stratum weight raises the estimate from 13.9% to 14.7%. The adults-and-phones factor lowers it to 14.4%. The dual-frame factor raises it to 15.0%. The raking input lowers it to 14.5%. Raking lifts it to 15.6%.

The design stages are not even on the population scale: summed over respondents, _STRWT totals 16.8 million, telephone numbers thinned by nonresponse rather than adults. Part of the early movement is geography. BRFSS gives every state a sample large enough for state estimates, so small states are heavily oversampled; weighting respondents equally within each state, but holding every state at its final weighted total, already moves the unweighted figure to 14.3%.

Raking is the largest move, and the one that reallocates people (Figure 2.2). Adults aged 18–24 are 6.5% of respondents and 12.2% of weighted adults; adults 65 and older fall from 37.9% to 23.5%. College graduates fall from 42.0% to 31.7%, and adults without a high school diploma rise from 5.9% to 11.0%; the design weights barely touch education. Cell-only adults rise from 67.2% to 71.9%.

Who gains and loses weighted share along the chain

Table of the weighted share of age, sex, phone-status, and education groups under each of the six weighting stages, from unweighted to the final weight.
Group	Unweighted	Stratum weight	× adults ÷ phones	× dual-frame factor	Raking input	Final weight
Age: 18–24	6.5%	8.2%	7.8%	8.6%	8.2%	12.2%
Age: 25–34	10.9%	13.3%	12.6%	14.1%	12.9%	16.7%
Age: 35–44	13.2%	15.0%	14.3%	15.8%	14.4%	16.7%
Age: 45–54	14.0%	15.1%	14.7%	15.5%	14.7%	15.3%
Age: 55–64	17.5%	17.2%	17.3%	17.2%	17.3%	15.6%
Age: 65 or older	37.9%	31.3%	33.3%	28.8%	32.5%	23.5%
Sex: Male	47.6%	49.2%	48.7%	49.7%	49.4%	49.1%
Sex: Female	52.4%	50.8%	51.3%	50.3%	50.6%	50.9%
Phone: Landline only	2.6%	1.1%	1.6%	1.8%	2.4%	2.4%
Phone: Both (dual user)	30.2%	21.7%	25.5%	14.8%	25.8%	25.8%
Phone: Cell only	67.2%	77.2%	72.9%	83.4%	71.8%	71.9%
Education: Less than high school	5.9%	6.5%	6.4%	6.8%	7.0%	11.0%
Education: High school graduate	25.3%	24.0%	24.0%	24.1%	23.7%	27.2%
Education: Some college	26.4%	25.5%	25.6%	25.5%	25.3%	29.5%
Education: College graduate	42.0%	43.4%	43.4%	43.0%	43.4%	31.7%

Figure 2.2.Raking, not the design weights, moves young adults up and college graduates down.Note: n = 450,353. Phone status: landline-frame respondents without a cell phone are landline only; cell-frame respondents without a landline are cell only; everyone else is a dual user (_DUALUSE). Education shares exclude the 0.5% with missing education.Source: CDC BRFSS 2024 public file; all respondents in the 49 states and DC

Age shows the mechanism most plainly. Under the final weight, FMD is 21.8% among adults 18–24 and 8.7% among adults 65 and older, so shifting weight toward the young raises the overall estimate. That is an accounting consistent with the direction of the change, not a decomposition of it: raking adjusts every margin at once, and age, education, and phone status overlap. The weights also spread as they move down the chain. Their coefficient of variation is 1.45 for the stratum weight, 1.56 for the raking input, and 2.03 after raking. The respondent who stands for 49,582 adults is a product of that spread.

What each link does

CDC's 2024 Overview documents how each link is made (CDC 2024), and every definition was checked against the file. The stratum weight _STRWT is the inverse of the sampling fraction for telephone numbers in the respondent's stratum. The adults-and-phones factor applies to the 81,372 landline respondents, whose chance of selection falls with the number of adults $A_i$ in the household and rises with its number of landline numbers $k_i$ ; a cell phone reaches a person, so the 368,981 cell-frame respondents get a factor of one. The dual-frame factor handles the 30.2% of respondents who could have been sampled from either frame, with a state-specific compositing factor $\lambda_s$ that ranges from 0.02 to 0.63 for landline-frame dual users. The raking input _LLCPWT2 truncates the composite design weight within geographic regions at the mean plus or minus $1.96$ standard deviations and, as the data show, rescales it state by state. The final weight _LLCPWT rakes to population totals on eight margins in every state: sex by age, race and ethnicity, education, marital status, home ownership, sex by race, age by race, and telephone ownership.

The first three links multiply into the design weight $d_i$ , which reflects each adult's chance of selection, and the last two act as an adjustment $g_i$ , so that the final weight is $w_i = d_i g_i$ . The design weight is

\begin{aligned} d_i &= \texttt{\_STRWT}_{h(i)} \times \frac{\min(A_i,\,5)}{\min(k_i,\,3)} \times c_i, \\[4pt] c_i &= \begin{cases} \lambda_s & \text{dual user reached by landline} \\ 1-\lambda_s & \text{dual user reached by cell} \\ 1 & \text{otherwise,} \end{cases} \end{aligned}

with $A_i = k_i = 1$ on the cell frame.

Each link as CDC documents it, and the fourth link in detail

States sample telephone numbers within strata defined by geography and, for landlines, by the density of listed household numbers; _STRWT is the count of numbers in the stratum divided by the count selected. A landline call reaches a household and the interviewer picks one adult at random, which is where the adults-and-phones factor comes from. Since 2014 every adult reached on a cell phone is eligible, so an adult with both kinds of phone could have been sampled from either frame and gets $\lambda_s$ if reached by landline and $1-\lambda_s$ if reached by cell. Raking is iterative proportional fitting; states with sub-state regions or large counties add regional and county margins, and the control totals come from Claritas population estimates, the 2019–2023 American Community Survey, and NCHS estimates of wireless-only households.

At the _STRWT scale the dual-frame factor cuts the weighted share of dual users from 25.5% to 14.8%. The fourth link then does two things at once, so its movement is not the effect of truncation alone. CDC's Overview describes truncation at this step; the data also show a rescaling. _LLCPWT2 totals 254.8 million, its total in each state is between 0.97 and 1.02 times the final weight's, and the dual-user share is back at 25.8%, essentially its final value. Because the rescaling differs from state to state, it can move the national estimate on its own.

What can be rebuilt

The chain has three kinds of link (Figure 2.4), and the difference matters to anyone who wants to modify a weight, or to understand what a modified weight has lost.

Rebuilt exactly from public variables. _RAWRAKE, the adults-and-phones factor, equals $\min(A_i,5)/\min(k_i,3)$ for every landline respondent with a known count of numbers; the 839 with an unknown count are treated as having one. _WT2RAKE equals _STRWT times _RAWRAKE on all but 12 records, all Illinois cell interviews that carry another stratum's weight. The two compositing factors sum to one in every state. _STRWT itself is shipped but not derivable: the counts behind it are not public.
Documented, not rebuildable. _LLCPWT2 depends on sub-state regions, which are removed from the public file, and on a rescaling the Overview does not describe. _LLCPWT depends on control totals and regional and county margins that are not on the file.
Illustrative only. Raking _LLCPWT2 to the final weight's own state totals on the eight public margins gives 15.51%, against 15.56% for the published weight, yet the rebuilt weights differ from the published ones by a median of 11%. The visible margins reproduce nearly all of the raking step's effect on this estimate, but not the weights themselves.

The BRFSS 2024 weighting chain

Table of the six weighting stages with the variable, what each step multiplies in, whether it can be rebuilt, the FMD estimate, the weight CV, the Kish effective sample, and the sum of weights.
Stage	Variable	What it adds	Rebuildable?	FMD	CV(w)	n Kish	Sum of weights (millions)
Unweighted	none	Each respondent counts once	No weights	13.9%	0.00	442,292	0.4
Stratum weight	_STRWT	Inverse sampling fraction of telephone numbers in the stratum	Rebuilt from public variables	14.7%	1.45	142,104	16.8
× adults ÷ phones	_WT2RAKE	Times adults in household (max 5) over landline numbers (max 3)	Rebuilt from public variables	14.4%	1.48	138,991	17.8
× dual-frame factor	_WT2RAKE × _DUALCOR	Dual users split between the two frames	Rebuilt from public variables	15.0%	1.52	133,564	15.5
Raking input	_LLCPWT2	Truncated within region; on the population scale	Documented by CDC, not rebuildable	14.5%	1.56	128,779	254.8
Final weight	_LLCPWT	Raked to 8 or more state margins	Documented by CDC, not rebuildable	15.6%	2.03	86,231	255.9

Figure 2.4.The design links can be rebuilt from public variables; the raking input and the final weight cannot, and raking spreads the weights most.Note: n = 442,292. _STRWT = NRECSTR/NRECSEL; _RAWRAKE = min(NUMADULT,5)/min(phones,3) on the landline frame and 1 on the cell frame; _DUALCOR is lambda for landline-frame dual users and 1 - lambda for cell-frame dual users.Source: CDC BRFSS 2024 public file; respondents with a valid MENTHLTH answer in the 49 states and DC

Who the number is about

A prevalence needs a population, and this one is not the United States. The 2024 public file holds 457,670 interviews, none of them from Tennessee, which could not collect enough data to meet CDC's minimum for release (CDC 2024). It does include Guam, Puerto Rico, and the US Virgin Islands, which draw simple random samples of landline numbers rather than the stratified samples the states use. The target here is adults in the 49 states and the District of Columbia that released 2024 data. The 7,317 territorial interviews are set aside. BRFSS asks about mental health in one question, MENTHLTH, coded as a count of days, 88 for none, and 77 or 99 for don't know or refused. The 1.8% who answered don't know or refused are excluded from the outcome, so the estimand is prevalence among adults who would answer.

The missing state is not a rounding error. In 2022, the most recent year in which all 50 states and DC released data, Tennessee's respondents represented 5.5 million adults, 2.1% of the weighted total. The set of jurisdictions also changes from year to year, so a trend or pooled estimate labeled "U.S. adults" describes a population that shifts beneath it (Figure 2.3). If your paper pools BRFSS years, this table is the footnote it needs.

Which jurisdictions are in the BRFSS public file

Table listing, for each year 2014 to 2024, how many of the 50 states and DC released BRFSS data, which did not, and which territories did.
Year	States + DC released (of 51)	Not in the public file	Territories
2014	51	none	Guam, Puerto Rico
2015	51	none	Guam, Puerto Rico
2016	51	none	Guam, Puerto Rico, US Virgin Islands
2017	51	none	Guam, Puerto Rico
2018	51	none	Guam, Puerto Rico
2019	50	New Jersey	Guam, Puerto Rico
2020	51	none	Guam, Puerto Rico
2021	50	Florida	Guam, Puerto Rico, US Virgin Islands
2022	51	none	Guam, Puerto Rico, US Virgin Islands
2023	49	Kentucky, Pennsylvania	Guam, Puerto Rico, US Virgin Islands
2024	50	Tennessee	Guam, Puerto Rico, US Virgin Islands

Figure 2.3.The jurisdictions in the BRFSS public file change from year to year: 2024 omits Tennessee, 2023 Kentucky and Pennsylvania, 2021 Florida, and 2019 New Jersey.Note: Read from the files themselves (distinct _STATE by survey year). CDC's 2024 Overview: Tennessee was unable to collect enough data to meet the minimum requirements for the public file.Source: CDC BRFSS 2014-2024 public files (harmonized: BRFSS_2026/cleaned/brfss_multi_rec.parquet)

A second trap: 19,553 records in the 2024 file carry an interview year (IYEAR) of 2025. Select a survey year by file, not by interview date.

Analysis Ledger

Frequent mental distress, adults in 49 states and DC (BRFSS 2024)

Target population: Noninstitutionalized adults 18+ living in private residences or college housing in the 49 states and DC that released 2024 BRFSS data (every state except Tennessee). Guam, Puerto Rico, and the US Virgin Islands are excluded by choice.
Estimand: Share of adults reporting 14 or more of the past 30 days when their mental health was not good (MENTHLTH >= 14; 88 = none), among adults who would answer the question.
Estimator: Hajek weighted mean, sum(w_i y_i)/sum(w_i), over respondents with a valid answer.
Explicit weights: _LLCPWT: stratum weight x adults/phones (landline) x dual-frame compositing factor, truncated within region, then raked to 8 or more state margins.
Implicit weights: None from a model. The weights themselves move leverage toward young, cell-only, and less-educated respondents and toward large states, where one respondent stands for thousands of adults.
Source of randomness: Selection of telephone numbers within strata in two frames, selection of one adult in landline households, and response; the weights are themselves estimated.
Variance estimator: Taylor linearization; strata _STSTR; PSU = respondent (_PSU); 109 singleton strata centred ('adjust'); df = 439,915. Weights treated as fixed.
Assumptions: Nonresponse and coverage bias in FMD run through the raking margins; the small telephone noncoverage is ignorable; self-report measures the construct; Taylor with fixed weights ignores the variability added by estimating the weights.
Numbers reported: 15.6%
13.9%
5.03
86,231

As a check on coding and weights, the file reproduces CDC's own weighted tabulation of the calculated variable _MENT14D: 15.23% of all 2024 records, territories included and nonresponse in the denominator, as printed in the codebook.

How uncertain the estimate is

BRFSS declares strata (_STSTR) and a primary sampling unit (_PSU), but the PSU is the respondent's sequence number: each respondent is its own PSU. The design is stratified and single-stage. The target population has 2,377 strata, and the FMD estimate has 439,915 design degrees of freedom: respondents with a valid answer minus strata. Of the strata, 109 hold a single respondent, where a variance formula that needs two units per stratum has nothing to work with. This chapter centers each lone unit at the grand mean, the rule in CDC's own R example; removing them instead changes the standard error only in its fifth significant digit.

The design-based standard error is 0.122 percentage points. It treats the final weights as fixed, although raking is itself estimated from the sample (Lumley 2010). A calibration-aware linearization, or replicate weights that repeat the raking, would carry that part of the variance too. The public file carries no replicate weights, and not all of the raking margins or their control totals, so it supports neither directly.

In code: the declaration in Python and R

The full national file in Chapter 8, territories included, has more strata and more degrees of freedom. R's survey package stops on singleton strata by default, which is the right default; the option below overrides it.

python

import svy
design = svy.Design(stratum="xststr", psu="psu_key", wgt="xllcpwt")  # psu_key = stratum + _PSU
s = svy.Sample(brfss, design=design).singleton.handle("center")
s.estimation.mean("fmd0", where=svy.col("valid") == 1, deff="wr")

options(survey.lonely.psu = "adjust")
des <- svydesign(ids = ~1, strata = ~xststr, weights = ~xllcpwt, data = brfss)
svymean(~fmd, des, na.rm = TRUE, deff = "replace")

Both give a standard error of 0.122 percentage points and agree to at least six significant digits.

Inverse probabilities, and where they end

The other logic: a design weight you can see

Only the first three links of the BRFSS chain have a probability interpretation, and even they rest on counts the public cannot see. The ATUS weight is the contrasting case, and it is worth holding the two side by side. ATUS asks each respondent about one day, assigned in advance and never substituted. BLS assigns a share $0.10$ of the sample to each weekday and $0.25$ to each weekend day, so weekdays and weekends each receive half (BLS 2026). People work much less on weekends. Here the producer's decision is printed in the user guide, the probability is known, and a reader with the file can check what the weight does about it.

Predict first

Weekend days are about half of the ATUS diaries but two of every seven days in a year. Is the unweighted mean of minutes worked per day too high or too low, and by roughly how much?

Of the 8,548 diaries in 2023, 48.9% describe a Saturday or Sunday. Respondents worked 250.9 minutes on the average weekday diary and 55.6 on the average weekend diary, so the unweighted mean of 155.4 minutes leans toward the weekend and is too low.

The repair is the oldest idea in survey weighting: weight each unit by the inverse of its probability of selection (Horvitz and Thompson 1952).

d_i = \frac{1}{\pi_i}, \qquad \hat{\bar y} = \frac{\sum_i d_i\, y_i}{\sum_i d_i}.

A weekday diary has $\pi_i \propto 0.10$ and gets $d_i \propto 10$ ; a weekend diary has $\pi_i \propto 0.25$ and gets $d_i \propto 4$ . The ratio on the right, the Hájek estimator, needs only relative weights. With $d_i$ alone, weekends shrink to 27.7% of person-days and the mean rises to 196.8 minutes (Figure 2.5).

Weekend diaries are half the sample and two-sevenths of the days

Table comparing weekday and weekend ATUS 2023 diaries: respondents, share of the sample, assignment probability, design weight, weighted share of person-days, and unweighted work minutes.
Diary day	Respondents	Share of sample	Assignment probability, each day	d = 1/π	Share of days under d	Share of days under WT06	Work minutes (unweighted)
Weekday (Mon–Fri)	4,366	51.1%	10%	10	72.3%	71.2%	250.9 min
Weekend (Sat–Sun)	4,182	48.9%	25%	4	27.7%	28.8%	55.6 min

Figure 2.5.Weekend diaries are half the ATUS sample; the design weight shrinks them to about two-sevenths of person-days.Note: n = 8,548. Weights-only extract: standard errors from these weights would not be design-based.Source: IPUMS ATUS 2023; allocation from the BLS ATUS User's Guide, section 3.5

The published weight, WT06, gives 213.8 minutes, which matches the published BLS figure to rounding (BLS 2024). It still carries the day allocation: the average weekday diary has 2.4 times the weight of the average weekend diary, near the design ratio of $10/4$ , and weekends get 28.8% of weighted person-days. The remaining distance is the $g_i$ in $w_i = d_i g_i$ : other design factors, a nonresponse adjustment, and calibration to population totals, none of which a user can rebuild.This ATUS extract has no design variables, so it carries an estimand lesson and no uncertainty lesson. A standard error computed from WT06 alone would leave out the design.

So the two weights differ in where the visible part ends. In ATUS, a known design probability does most of the work and the adjustment finishes it. In BRFSS, the design links are a scaffold, and the step that moves the estimate most is the one that rests on a model of who answers. Both columns are called "the weight," and both give a weighted mean with the same formula. What differs is how much of each weight you can defend from the design alone, and how much rests on the producer's assumptions about nonresponse.

Analysis Ledger

The day-of-week design weight (ATUS 2023)

Target population: All person-days in 2023, civilian noninstitutional population 15+.
Estimand: Mean minutes per day spent working, averaged over every person-day of 2023.
Estimator: Hajek weighted mean of BLS_WORK: first with d_i = 1/pi_i for the diary day alone, then with WT06.
Explicit weights: d_i = 1/0.10 for a weekday diary and 1/0.25 for a weekend diary (the designated-day allocation); WT06 adds the rest of the design weight plus nonresponse and population controls.
Implicit weights: None.
Source of randomness: CPS-based household selection, random assignment of one designated day, and response.
Variance estimator: Not computed: weights-only file; an SE from these weights alone would not be design-based.
Assumptions: The designated-day assignment follows the documented allocation; response on the designated day is ignorable after WT06.
Numbers reported: 155.4minutes per day
196.8minutes per day
213.8minutes per day

Where the probability interpretation ends

The rescaling in _LLCPWT2 and the raking in _LLCPWT make the sample reproduce known population totals, and they rest on a model of nonresponse: within the raked categories, respondents resemble nonrespondents. Gelman (2007) opens by calling survey weighting "a mess." Weights in practice, he stresses, are generally not inverse probabilities of selection: they combine selection probabilities with adjustments for nonresponse, and once they do, a weighted mean's standard error is no longer straightforward and anything beyond a mean or ratio is harder still. The respondent who stands for 49,582 adults does not have a one-in-49,582 chance of selection. That number says how many adults, in the raked categories the respondent belongs to, the sample was short of.

What software calls a weight

The word "weight" also names three different objects in statistical software, and the option you type commits you to one of them.

A frequency weight $f_i$ says the record stands for $f_i$ identical observations; the sample size is $\sum_i f_i$ .
A precision (analytic) weight $a_i$ says observation $i$ has variance $\sigma^2/a_i$ , as when each record is a mean over $a_i$ people.
A sampling weight $w_i = d_i\, g_i$ says the record represents $w_i$ population units, with a variance that depends on the design that produced it.

All three give the same weighted mean and different standard errors. Read as a frequency weight, _LLCPWT treats every weighted adult as interviewed and gives a standard error of 0.0023 percentage points for FMD; read as a precision weight, 0.055; read as a sampling weight with the declared design, 0.122.Stata's fweight, aweight, and pweight, and the weights argument of R's lm (a precision weight), each commit to one reading whether or not the user meant it.

Box

Two effective sample sizes for one BRFSS estimate

Chapter 0 separates four objects called "effective sample size." Two of them can be computed here, and they answer different questions.

The Kish effective sample size, $\nKish = n/(1+\CV(w)^2) = (\sum_i w_i)^2/\sum_i w_i^2$ , prices weight dispersion alone (Kish 1965): the size of a simple random sample that would give a weighted mean the same variance if the outcome were unrelated to the weights and the design had no strata or clusters. The final BRFSS weight has a coefficient of variation of 2.03, so $\nKish/n$ is 0.19, against 0.55 for ATUS and 0.73 for the CPS ASEC (Figure 2.6). Much of the loss is the state allocation: within the median state, $\nKish/n$ is 0.53. A national estimate pays for sample sizes chosen for state estimates.

Kish effective sample sizes in five federal surveys

Table of sample size, weight CV, Kish effective sample size, and their ratio for BRFSS 2024, ATUS 2023, CPS ASEC 2024, CPS Food Security December 2023 households, and CPS Voting 2020.
Survey	Unit	Weight	n	CV(w)	n Kish	n Kish / n
BRFSS 2024 (49 states + DC)	adults	_LLCPWT	450,353	2.03	87,731	0.19
ATUS 2023	person-days	WT06	8,548	0.91	4,674	0.55
CPS Food Security, Dec 2023	households	FSSUPPWTH	30,863	0.65	21,707	0.70
CPS ASEC 2024	persons	ASECWT	144,265	0.61	104,964	0.73
CPS Voting and Registration 2020	persons	VOSUPPWT	111,665	0.55	85,782	0.77

Figure 2.6.Of five federal surveys, BRFSS keeps the smallest share of its sample after weighting.Note: All records with a positive weight; Food Security counts households (PERNUM = 1). Kish's formula ignores the outcome and the strata; it prices only weight dispersion.Source: BRFSS 2024 (CDC); IPUMS ATUS 2023; IPUMS CPS ASEC 2024, Food Security Dec 2023, Voting 2020 (analysis/)

The design effective sample size, $\ndesign(\hat\theta) = n/\DEFF(\hat\theta)$ , belongs to an estimate, not to a survey. For FMD the design effect is 5.03, so $\ndesign$ is 87,931, close to the Kish figure of 86,231 for the same respondents. For the share of adults aged 65 and older, a characteristic strongly related to the weights, the design effect is 3.95 and $\ndesign$ is 114,027.

Neither is "the" effective sample size of BRFSS. The regression-effective sample of Chapter 5 and Meng's data-defect effective sample size answer other questions; Appendix B derives all four.

Trimming: precision bought with bias

Large weights make estimates noisy, so producers trim them, as CDC does to the design weight before raking. Analysts are sometimes tempted to trim the final weight themselves, and the respondent standing for 49,582 adults is the kind of record that invites it. Figure 2.7 caps _LLCPWT at each state's 99th or 95th percentile and then rescales all of the state's weights to restore the state's total, so state totals do not change.

Predict first

Within a state, which respondents carry the largest final weights: the groups raking pushed up, or the groups it pushed down? If the largest weights are capped, will the FMD estimate rise or fall?

Trimming buys little precision and moves the margins

Table of FMD prevalence, standard error, weight CV, Kish effective sample, and three weighted margins under the final weight, caps at each state's 99th and 95th percentiles, and a capped-then-re-raked weight.
Weight	FMD	SE (pp)	CV(w)	n Kish	Aged 18–24	Cell only	Less than high school
No cap (published final weight)	15.56%	0.122	2.03	86,231	12.2%	71.9%	11.0%
Cap at each state's 99th percentile	15.58%	0.118	1.91	95,477	11.9%	72.3%	10.7%
Cap at each state's 95th percentile	15.49%	0.109	1.72	111,302	11.3%	72.9%	10.1%
95th-percentile cap, then re-raked	15.61%	0.115	1.85	100,046	12.2%	71.9%	11.0%

Figure 2.7.Capping the final weight shrinks the standard error modestly, moves the estimate, and undoes part of the raking; re-raking restores the margins and gives back some precision.Note: Caps are within-state percentiles of _LLCPWT; trimmed weight is redistributed within the state so state totals are unchanged. Re-raking targets the final weight's own state margins (sex x age, race/ethnicity, education, marital status, tenure, sex x race, age x race, phone status). Taylor SEs treat weights as fixed.Source: CDC BRFSS 2024 public file; 49 states and DC

The 95th-percentile cap cuts the coefficient of variation from 2.03 to 1.72 and raises $\nKish$ from 86,231 to 111,302, but the standard error falls only to 89% of its untrimmed value: standard errors move with the square root of the effective sample size. The estimate falls by 0.07 percentage points to 15.49% (95% CI 15.28%–15.71%), a shift of 0.5 standard errors. The milder 99th-percentile cap gives 15.58%, with CV 1.91 and $\nKish$ 95,477.

The shift has a visible compositional source: trimming changes which groups carry weight. Within a state the largest weights tend to belong to groups that raking pushed up, so capping them undoes part of the raking. After the 95th-percentile cap, adults 18–24 fall to 11.3% of weighted adults, below the 12.2% the population totals require, while cell-only adults rise to 72.9%. A trimmed weight that no longer matches the margins has traded part of a bias correction for lower variance, and the size of the bias it readmits is unknown. Re-raking the capped weights to the same margins restores them exactly and returns the estimate to 15.61%, with CV 1.85 and $\nKish$ 100,046: part of the precision gain survives, part is given back. Producers iterate between trimming and raking for this reason (Valliant, Dever, and Kreuter 2018); trimming a final weight without re-raking changes the population the estimate describes.

Build a weight

Predict first

In the synthetic population below, one group is oversampled, and younger adults and adults without a degree respond less often. After raking to known margins, will trimming the largest weights move the estimate toward the population truth or away from it? What will happen to $\nKish$ ?

Building a survey weight, one stage at a time

2,000 adults sampled from the synthetic population, with group G selected at 4 times the rate of everyone else; younger adults and those without a degree respond less often. Outcome: the share uninsured.

Seed 11,079 of 2,000 responded (54%)

Trimming cap, as a multiple of the mean weight2.0×

91 weights capped at 37.1 people each.

Stage	Estimate	Change	SE, weights only	CV(w)	n_Kish	Largest ÷ mean
UnweightedEvery respondent counts once.	15.2%	–	1.09 pts	0.00	1,079	1.00×
Design weightsd = 1/π undoes the oversampling of G.	11.9%	−3.3 pts	1.06 pts	0.54	839	1.45×
Nonresponse-adjustedWithin each age × degree cell, respondents also carry the design weight of that cell’s nonrespondents.	13.0%	+1.1 pts	1.17 pts	0.60	795	2.19×
RakedScaled in turns until the weighted totals match the known age × degree and G totals.	12.9%	−0.1 pts	1.15 pts	0.58	807	2.04×
TrimmedWeights above the cap are cut and the excess spread over the others.	12.9%	0.0 pts	1.15 pts	0.58	811	2.00×
Final (re-raked)Raked again so the margins hold after trimming.	12.9%	0.0 pts	1.15 pts	0.58	807	2.04×

Raking converged in 5 iterations (largest margin error by iteration: 2.0×10⁻⁴, 2.1×10⁻⁶, 2.2×10⁻⁸, 2.3×10⁻¹⁰, 2.4×10⁻¹²). Trimming moved the margins by up to 1.76%; re-raking restored them in 5 iterations. Population total after raking: 20,000.

Three interpretations of the same final weights

Every reading gives the same mean, 12.9%. The standard errors differ because each reading implies a different repetition.

Frequency weights fᵢSE 0.24 ptsEach record stands for fᵢ identical records, so the software sees Σfᵢ = 20,000 observations and reports a far smaller SE.
Precision weights aᵢSE 1.02 ptsEach outcome is treated as an average with variance σ²/aᵢ; the weights are read as known precision, so the variance that unequal weights add is ignored.
Sampling weights wᵢSE 1.15 ptsEach record represents wᵢ people; this weights-only linearized SE counts the weight dispersion but treats the weights as fixed and ignores the strata and the raking.

The repeated-sampling SD reruns the whole pipeline (sampling, nonresponse, raking, trimming) on fresh samples. Only the sampling-weight reading aims at that repetition, and even it treats the weights as fixed: the design and the weight construction together determine the variance.

What is held fixed. The population, the sample design (2,000 adults, G oversampled 4×), the response model (age and degree), and the outcome. The stages change only the weights; the respondents stay the same. The truth is the simulated population value, not a benchmark.

The widget runs a simplified version of the BRFSS chain on a synthetic population with a known answer: design weights for an oversampled group, a nonresponse adjustment within age-by-degree cells, raking to known margins, trimming at a cap you set, and a final re-raking. It reports the estimate, $\CV(w)$ , and $\nKish$ at each stage; its three-interpretations panel shows one weighted mean with three standard errors; and a final button reruns the pipeline on fresh samples and reports the repeated-sampling spread of the final estimate, the quantity the sampling-weight standard error is trying to estimate.

In this population the answer to the prediction is: away from the truth, on average. At the default cap, trimming raises $\nKish$ in nearly every sample and pulls the estimate down. The largest weights belong to the young adults and adults without a degree whom nonresponse made scarce, so capping them moves the estimate back toward the respondents' composition. In any one sample the trimmed estimate can land nearer the truth by chance. Re-raking restores the margins and, on average, returns the estimate to about where raking left it.

How the widget's chain differs from CDC's

BRFSS truncates the design weight before raking; the widget trims after raking and then rakes again, the iteration described in the previous section. The cap is a multiple of the mean weight, and the excess is spread over the other weights so the total is unchanged.

Note

A new case

An analyst caps a survey's final weights and rescales them so that every state keeps its weighted total, but the capped weights no longer match the population's age distribution. What can the analyst still say about a state's weighted total, and what can no longer be assumed about an outcome that varies with age?

One answer

State totals, and anything that depends only on them, are unchanged. The age mix is not: the capped weights describe a population with the wrong age distribution, so an outcome that varies with age, such as frequent mental distress, is pulled toward the respondents' age mix. How far depends on how much the outcome differs across the misrepresented groups, which the capped weights alone cannot reveal. Re-raking to the margins restores the age distribution, and comparing the capped and re-raked estimates shows how much the lost margins mattered for that outcome.

What we learned about the question

An estimated 15.6% of adults in the 49 states and DC that released 2024 BRFSS data report frequent mental distress, among those who answered. It is far more common among young adults, 21.8% of those 18–24, than among adults 65 and older, 8.7%. Young adults are scarce among respondents, so an analysis that ignores the weights understates the prevalence. The BRFSS weight is mostly a nonresponse model with a design scaffold; the ATUS weight is mostly a design probability with an adjustment on top.

What the method changed

The represented population. The unweighted 13.9% is a fact about who answered the telephone; the weighted 15.6% is an estimate for adults. Moving from one to the other raised the estimate by 1.7 percentage points, most of it at the raking step, which moved weight from older and college-educated respondents to younger and less-educated adults. The design weights repair unequal chances of selection, which the survey controls and records. Raking repairs unequal chances of responding, which are unknown and are modeled by assuming that respondents resemble nonrespondents within the raked categories. The same weights cost precision: the Kish effective sample size is 86,231 of 442,292 respondents.

What this analysis does not establish

A national prevalence. Tennessee is absent from the 2024 file, and the territories are outside the target.
Freedom from nonresponse bias. Raking removes bias only to the extent that it runs through the raked characteristics; mental distress may differ between respondents and nonrespondents who share age, sex, race, education, and phone status. The estimate is as good as that assumption.
Record-level reconstruction. The intermediate stages are diagnostics, not alternative estimates, and _LLCPWT2 and _LLCPWT cannot be rebuilt from the public file.
The full uncertainty. The Taylor standard error treats the weights as fixed.
Anything about causes. The age and education contrasts are descriptive, and FMD counts self-reported days, not diagnoses.