Data, Code, and Reproducibility

You will need this appendix when you want to check a number rather than read it: to find which release and which file a chapter used, to see how a headline figure was validated and against what, to learn which tool versions produced it, or to find out how far a rebuild on your own machine can go today. It also records how the book's examples were chosen, for readers who want to know what was tried and set aside.

A reader who wants to check the NHIS smoking standard error in Chapter 8 can obtain the same public-use file from IPUMS, read the same build.py, and still not run it, because the script expects a derived current_smoker column that lives in the author's store. "Every number comes from a script" and "a reader can rerun the script" are different promises. Four need keeping apart.

• Public inputs are available. Every headline result uses public-use files, from the producers or through IPUMS extracts, which require a free account and agreement to IPUMS terms.
• Analysis code is available. Each chapter's build.py, and its verify.R where one exists, is published in the analysis/ folder of the site's repository.
• The full preparation pipeline is not yet available. The scripts that define the IPUMS extracts, download them, and clean them into the author's store, including derived outcome variables such as uninsured and current_smoker, live with the store, not in the repository.
• Independent execution from a fresh setup is not yet possible. The build scripts read the store's file paths and derived variables. A reader would need the same extracts, variable definitions, joins, and intermediate checks, and the booklet does not yet document all of them. Until it does, the tiers below describe the data, not a rebuild on a fresh machine.

Data sources and pinned releases

Each chapter's artifacts/manifest.json lists the files it read, with row counts. The list below is compiled from those manifests.

• American Community Survey, IPUMS USA (Ruggles et al. 2025). The 2024 1-year sample, with its 80 person replicate weights, carries Lab 3, which also checks its code against the Census Bureau's 2024 PUMS verification estimates. The same sample, restricted to wage and salary workers aged 25 to 64 who usually work at least 30 hours a week for 40 or more weeks a year, carries the work-at-home wage regressions of Chapter 4 and Chapter 5, again with its person replicate weights. The 1-year samples for 2008–2019, with person replicate weights for the design window, carry the Medicaid-expansion comparison of Chapter 6. The Chapter 8 guide uses the 2024 sample's design manifest and weight total. Design and replicate methods follow the Census Bureau's documentation (U.S. Census Bureau 2024).
• Current Population Survey, IPUMS CPS (Flood et al. 2025). Chapter 1 uses the November Voting and Registration supplements for 2008–2024, the December 2023 Food Security supplement, and the Annual Social and Economic Supplement (ASEC) for survey years 2020–2025 (income years 2019–2024), with its 160 replicate weights, for the comparison of poverty measures (Shrider and Bijou 2025). Chapter 2 compares the weights of the ASEC 2024, the December 2023 Food Security supplement, and the November 2020 voting supplement. The Chapter 8 guide declares the ASEC 2025.
• American Time Use Survey, IPUMS ATUS (Flood, Sayer, and Backman 2025). The 2023 file carries the work-time estimands of Chapter 1 and the design-weight example of Chapter 2; the 2024 file has a Chapter 8 guide. Published BLS estimates are the external benchmark (BLS 2024).
• National Health Interview Survey, IPUMS NHIS (Blewett et al. 2025). The 2023 sample adults carry the domain-estimation lesson of Chapter 3; the 2024 file has a Chapter 8 guide, which uses the 2023 file to count the PSUs the two years share.
• NHANES, National Center for Health Statistics (NCHS 2024). Chapter 3 uses the August 2021–August 2023 cycle, the 2017–March 2020 pre-pandemic file for the period comparison, and the 2017–2018 cycle for the comparison of blood-pressure instruments; the August 2021–August 2023 cycle also has a Chapter 8 guide. The official hypertension data brief is Chapter 3's external benchmark (Fryar et al. 2024).
• BRFSS, Centers for Disease Control and Prevention (CDC 2024). Chapter 2 rebuilds the weight chain from the raw 2024 public file, which omits Tennessee, checks its coding against CDC's 2024 codebook, and uses the harmonized 2014–2024 store for coverage by year. Chapter 3 uses the 2023 and 2024 files, and the 2024 file has a Chapter 8 guide.
• State Medicaid-expansion dates for Chapter 6: effective coverage dates for the 50 states and DC from KFF's tracker (KFF 2026), checked state by state against federal and state documents, with a source recorded for every state.
• Synthetic data. Chapter 0's example, the interactives, the six simulated designs of Chapter 7, and the numerical checks of Appendix B use seeded synthetic populations; no survey data enter them.

Data access tiers

Each result's inputs fall in one of three tiers. The tiers describe access to the data, not whether the analysis runs from the repository on a fresh machine.

• Tier 1, public inputs. The inputs are public-use files that anyone can obtain, either from the producers (Census Bureau, Bureau of Labor Statistics, National Center for Health Statistics, CDC) or through IPUMS extracts, which require a free account and agreement to IPUMS terms. Every headline result in the first edition is Tier 1.
• Tier 2, inputs after registration. Data whose producer requires an application or a data-use agreement beyond a free account. The PSID provenance box of Chapter 1 would be the only use in this edition, and the booklet does not reproduce its numbers.
• Tier 3, released aggregates only. None in the first edition.

The author's store and build scripts make the booklet reproducible for the author. For a reader, the four promises at the top of this appendix say what is and is not yet possible.

Validation layers

A number reaches the page only after passing the layers that apply to it.

1. Two implementations. Design-based estimates and standard errors are computed through svy's declared design and again with an independent hand-coded formula, and each chapter's manifest records the agreement.
2. R survey sentinel checks. Two or three headline design-based results per chapter are recomputed in R's survey package, the field's reference implementation, and the comparison is recorded in the chapter's manifest. Chapter 8's verify.R also sources the field guide's R declarations for NHIS, NHANES, BRFSS, ATUS, and CPS ASEC against the store, and confirms two documented failure modes: R stops on NHIS 2024 without nest = TRUE, and on the full BRFSS 2024 file without a lonely-PSU rule. The ACS declarations were checked only on a synthetic frame with the same column names, and the Stata declarations were not executed (Chapter 8).
3. Published benchmarks. Each chapter compares its headline numbers with published figures and records the comparison in its manifest: ATUS work time with BLS, turnout with the Census Bureau, food insecurity with USDA, poverty rates with the Census Bureau, BRFSS mental distress with CDC's codebook, the hypertension cascade with NCHS Data Brief 511, and ACS totals with the Census Bureau's PUMS verification estimates. The data pipeline also reproduces a set of published figures on its cleaned store (Figure C.1). All are external benchmarks, not truth: a published figure carries its own sampling and processing error.
4. Checks of mathematical claims. Each interactive's claims and each derivation in Appendix B are checked on seeded synthetic data. Chapter 7's simulations run in R's fixest; the standard errors of its opening draw are recomputed in pyfixest, and the comparison is in its manifest.
5. Determinism. Rebuilding a chapter from the same inputs produces byte-identical facts, ledgers, and figures; only the manifest's timestamp changes.

Published benchmarks the data pipeline reproduces

Re-run on the cleaned analysis store; each value compared with the producer's published figure

Table of external benchmarks: for each series, the published figure recorded in the validation log, the years re-computed, the clean-store values, and whether raw and clean stores agree.

Series	Published benchmark (as recorded)	Years	Clean-store values	Raw vs clean store
CPS turnout, citizens 18+ (Census convention), 2008-2024	63.6/45.5/61.8/41.9/61.4/53.4/66.8/52.2/65.3 (2008-2024)	2008–2024 (9 years)	2008: 63.6 / 2010: 45.5 / 2012: 61.8 / 2014: 41.9 / 2016: 61.4 / 2018: 53.4 / 2020: 66.8 / 2022: 52.2 / 2024: 65.3	identical
TUS current smoking, adults 18+ (TSRWT)	~16 (2010) declining to ~9 (2023)	2010–2023 (8 years)	2010: 16.1 / 2011: 15 / 2014: 14.3 / 2015: 13.3 / 2018: 12 / 2019: 11 / 2022: 9.3 / 2023: 8.7	differs where the clean store uses a derived variable (2018: 12.4 vs 12; 2019: 11.4 vs 11; 2022: 9.7 vs 9.3; 2023: 9 vs 8.7)
Volunteering rate, 16+ (VLSUPPWT)	2005=28.8, 2010=26.3, 2015=24.9 exact; VCL ~1.5pp above AmeriCorps	2005–2023 (7 years)	2005: 28.8 / 2010: 26.3 / 2015: 24.9 / 2017: 31.9 / 2019: 31.7 / 2021: 24.8 / 2023: 29.9	identical
ASEC uninsured all year (ASECWT)	CY2022=7.9 (svy 2023), CY2023=8.0 (svy 2024)	2020–2025 (6 years)	2020: 8.1 / 2021: 8.6 / 2022: 8.3 / 2023: 7.9 / 2024: 8 / 2025: 8	identical
ASEC median household income (ASECWTH, PERNUM=1)	CY2022=$74,580, CY2023=$80,610 (nominal)	2020–2025 (6 years)	2020: 69000 / 2021: 68000 / 2022: 70201 / 2023: 74282 / 2024: 80024 / 2025: 83602	identical
Food insecurity, households, December (FSSUPPWTH)	USDA: 14.9/12.7/10.5/10.2/12.8/13.5 exact (2011-2023)	2011–2024 (7 years)	2011: 14.9 / 2015: 12.7 / 2019: 10.5 / 2021: 10.2 / 2022: 12.8 / 2023: 13.5 / 2024: 13.7	identical
ACS national population by year (sum PERWT)	PERWT untouched -> must be identical; 2023 state totals matched Census exactly	2005–2024 (6 years)	2005: 288.4 / 2010: 309.3 / 2015: 321.4 / 2020: 329.5 / 2023: 334.9 / 2024: 340.1	identical
NHIS current smoking, adults 18+ (SAMPWEIGHT), 1997-2024	24.7/20.9/19.3/15.1/14.0/11.5 exact vs published	1997–2024 (7 years)	1997: 24.7 / 2005: 20.9 / 2010: 19.3 / 2015: 15.1 / 2019: 14 / 2022: 11.6 / 2024: 9.9	identical
ATUS minutes/day personal care + work (WT06)	~570-590 min/day; BLS_* untouched -> identical	2010–2023 (3 years)	2010: 568 / 2019: 577 / 2023: 590	identical

Two rows need a reading note. The TUS smoking series differs between raw and clean stores from 2018 on because the clean store uses a derived current-smoking variable that keeps the standard definition across a questionnaire change; and the NHIS row lists six published values against seven re-computed years, so the year alignment of the published list should be confirmed before the row is quoted.

How the author rebuilds the booklet

These steps run on the author's machine against the store; the four promises at the top of this appendix say which of them a reader can repeat today.

1. Data. IPUMS extracts are defined in the pipeline's definitions/ and downloaded and cleaned into the Box store's analysis/ layer; NHANES is downloaded from NCHS and assembled by the NHANES scripts; BRFSS is downloaded from CDC as SAS transport files and harmonized by the BRFSS scripts. Microdata stay in the author's store.
2. Artifacts. In each chapter folder, python build.py reads only the columns it needs, computes every number, and writes artifacts/: facts.json, ledger.json, figures/*.json, and manifest.json with inputs and validation results. Optional verify.R scripts run the R checks.
3. Export. The integrator's export.py copies the artifacts and the chapter's MDX into the site repository, where each page reads its numbers from those files and never from typed text.
4. Check. node scripts/check-mdx.mjs compiles each page with the site's own MDX pipeline and confirms that every fact, figure, ledger, and citation it references exists.
5. Deploy. The static site is built with Next.js and deployed to Vercel under the /weights path, marked as a draft and excluded from search indexing.

Figure C.2 lists the software versions the builds recorded.

Tool versions on the build machine

Captured by appendix-c-reproducibility/build.py when the artifacts were last built

Table of the software used to build the book, with the version installed on the build machine and each tool's role.

Tool	Version	Role in the book
Python	3.13.14	analysis language for every build.py
svy	0.28.0	design-based estimation (Taylor and replication); the book's computational spine
svy-rs	0.17.0	compiled engine behind svy
duckdb	1.5.5	scans and pre-aggregation over the parquet store
polars	1.43.0	data frames; the independent second path for point estimates
pandas	3.0.5	interchange with pyfixest
numpy	2.5.1	simulation and hand-coded formulas
scipy	1.18.0	t quantiles
pyarrow	25.0.0	parquet I/O
pyfixest	0.60.0	regressions with fixed effects and cluster-robust variance
R	4.6.1	independent oracle for design-based results
R survey	4.5	sentinel checks (svydesign, svrepdesign)
R arrow	25.0.1	reads the parquet store directly from R
R fixest	0.14.2	Chapter 7's simulated designs and their variance estimators
Node.js	24.19.0	site build and scripts/check-mdx.mjs
Next.js	16.3.4	static site framework
@mdx-js/mdx	3.1.1	MDX 3 compiler
KaTeX	0.16.47	math rendering
React	19.3.0	components and widgets

Terms of use

IPUMS data are used under the IPUMS terms of use: they are cited as each collection requests, and IPUMS microdata are never redistributed. The site publishes aggregates only: estimates, standard errors, intervals, counts, and the design facts in the field guide, each exported as a small JSON artifact. The same rule applies to NHANES and BRFSS, which are cited as their producers request. Microdata never leave the author's store, and nothing on the site allows a record to be reconstructed.

Development history: the auditions

This section records how the empirical spine changed during drafting; reproducing a result does not require it. The proposal required each application to pass an audition before a chapter was written: the substantive question had to be worth answering on its own, and a central technical correction had to have a visible payoff. The auditions changed the spine.

The audition table, chapter by chapter

Chapter	Planned application	Built on	Why
1	Housing-cost burden, ACS 2024	ATUS, "How much do Americans work?"; CPS turnout; CPS food security by household, person, and child; CPS ASEC official versus supplemental poverty	The ACS extract has no housing-cost variables. ATUS shows the whole weighting mechanism, weekend oversampling and person-day weights, in one small table. Housing burden returns as a second application if the extract is acquired. Census-convention turnout reproduces every published rate from 2008 to 2024, including 2022 and 2024, once supplement nonrespondents coded "Not in universe" are counted as nonvoters.
2	BRFSS weight construction	BRFSS 2024	Kept. The harmonized store holds only the final weight, so the raking chain is re-exported from the raw 2024 file for the waterfall figure.
3	NHANES hypertension cascade	NHANES August 2021–August 2023, plus the BRFSS–NHANES design contrast and an NHIS domain-estimation lesson	Kept and extended. The audition found that deleting rows instead of declaring a domain matters when strata lose PSUs, which became the chapter's domain lesson; the chapter's thinning experiment shows the understatement growing gradually and becoming serious only in very small domains.
4	Telework and earnings, CPS monthly	ACS work-at-home, with replicate weights	No CPS basic monthly or earnings file is in the store; the ACS has a pre-2020 series. The construct differs: the ACS records the usual means of commuting, the CPS any telework in the reference week.
5	Hidden weights, same CPS sample	the same ACS work-at-home sample	Kept in design; work-at-home prevalence and within-group wage contrasts vary widely across education groups, which is what the implicit weights need.
6	Medicaid expansion and uninsurance	ACS person files, 2011–2019: the January 2014 expansion cohort against states not expanded by 2019	Kept. The audition's split of states into expansion and comparison groups was approximate; Chapter 6 replaces it with a sourced panel of effective coverage dates and stated cohort rules. The minimum-wage and paid-leave backups were not needed.
7	Not in the proposal	Simulation only: six designs with a known zero effect (a classroom lottery, a state-level regressor, a state-policy panel with 50 and with 10 states, two non-nested dimensions, spatial shocks, common shocks), 1,0001,000 replicationsEstimandNumber of simulated data sets per designVarianceNone reportedSourceSimulation: sim.R with R fixest (this chapter); no survey dataNoteEach design is drawn from its own fixed seed draws each, in R fixest 0.14.2; the opening draw's standard errors cross-checked in pyfixest and recorded in the chapter's manifest	Added after the first draft. Chapter 6 works one study; the author wanted the clustering rule tested where the truth is known, across the designs economists use most. No survey data enter it, so the audition rule (a substantive question plus a visible technical payoff) applies in simulated form: each design's rejection rate is the payoff. Its code is in analysis/ch07-cluster-choice in the repository.
8	Four release guides	six guides: ACS, CPS ASEC, NHIS, ATUS, NHANES, BRFSS	All six are on disk; the machine-readable design fields exist for only a few datasets, so the guides mark each item's provenance.

One audition failed in the sense the proposal defined, and the failure is informative. In the ACS 2024 audition sample of full-time wage and salary workers, weighting barely moved the raw or the adjusted work-at-home wage contrast. The proposal's rule would have cut the application. The implementation plan recommends amending the rule so that an application also passes when a correction demonstrably does not change the answer and a reader would have assumed it would. That is Solon, Haider, and Wooldridge's own argument about when weighting matters (Solon, Haider, and Wooldridge 2015), so Chapter 4 reports the agreement and hands the interesting question, whose contrast the regression summarizes, to Chapter 5. The two historical examples, the 1968 PSID poverty rate and the Wolfers divorce-law regressions, remain boxes: provenance, not headline applications, and neither is yet reproduced.