Methods

This page defines the terms the figures rely on and the rules the analysis applies to every estimate. Each chapter's own methods note, at the end of the chapter, gives the sample, weights, years, estimator and interval method for that chapter's numbers.

Surveys and weights

Most chapters use surveys of individuals. A survey gives each respondent a survey weightSurvey weightsNumbers that scale each respondent so the sample matches the population on known traits such as age, race, education and region. A weighted average counts a respondent with a large weight for more than one with a small weight. so that the sample matches the population on known traits, and every survey estimate in the book is weighted unless its source line says otherwise. The source line under each figure names the survey, the years and the weight variable.

Intervals and sample sizes

A survey estimate carries a confidence intervalConfidence intervalA range around an estimate that would contain the true value in 95% of repeated samples if the sampling model held. The book reports 95% intervals unless it says otherwise. and the number of respondents behind it. Unless a note says otherwise, the interval is a 95% interval built from the effective sample sizeEffective sample sizeThe number of equally weighted respondents that would give the same precision as the weighted sample. It is smaller than the raw count when the weights vary a lot, and it sets how wide the confidence interval is.. That method treats the weighted sample as a simple random sample and leaves out the survey's design, which is the strata and clusters used to select respondents. Leaving out a design feature can change the uncertainty, and the book does not assume the direction: clustering tends to widen an interval and stratification tends to narrow it. A chapter reports a design-based comparison where it has run one, and the results differ. Chapter 1.1, Party and ideology finds larger design-based standard errors for its General Social Survey cells, Chapter 6.1, Confidence in thirteen institutions and Chapter 6.6, Tolerance of speech find ratios on both sides of one, and Chapter 2.1, Feelings toward the other party finds a smaller one for its 2024 estimate. Elsewhere the interval has no design correction, and the chapter says so.

The Cooperative Election Study and Nationscape are opt-in online panels. Their respondents chose to join, so they are not a probability sample and have no probability design to follow. An interval for them describes sampling variation among the panel's respondents and leaves out the error that comes from who joins the panel. That selection error is separate from the design effect of a probability survey, and no interval in the book measures it.

Regressions report HC3HC3 robust standard errorsA way of computing standard errors that stays valid when the spread of the outcome differs across respondents, with a correction for small samples. The book uses it for regressions that have no clusters. standard errors, or clustered standard errorsClustered standard errorsStandard errors that allow answers within the same group, such as the same county or the same week, to be related. They are wider than ordinary errors when members of a group resemble each other. where observations share a county, a week or another group. Where an interval comes from resampling, the resampling is seeded, so a rerun gives the same numbers.

Full counts

Election returns are full counts rather than samples. An estimate from a full count has no sampling interval, and the note under the figure says so. A number that comes from a model fitted to full counts, such as a regression across counties, carries the uncertainty the chapter reports for it.

Small cells

Cells with too few respondents are suppressed rather than plotted. The export that writes the data files also refuses any file whose rows look like individual respondents, so the book holds aggregate numbers only.

Survey mode breaks

A change in survey modeSurvey modeHow respondents were interviewed: in person, by phone, on paper or online. Mode can change answers, so a change in mode can look like a change in opinion. can move an estimate without any change in opinion. Where a series crosses a break, the chart marks it and, where the data allow, shows the estimate by mode. The start page names the three surveys that changed mode.

People and places

A pattern across counties, precincts or neighborhoods can differ from the pattern among the people who live in them. This is the problem of ecological inferenceEcological inferenceDrawing a conclusion about individuals from data on groups or places, such as counties. A pattern across counties can differ from the pattern among the people who live in them., and a chapter that works with places describes places. Where the same question is asked of individuals and of places, the chapters link to each other under "Same question at another grain".

Descriptive and causal language

Chapters use descriptive verbs such as "is higher among" and "moves with" unless the design supports a causal claim. A regression that holds other characteristics fixed, for example with fixed effectsFixed effectsRegression terms that give each group, such as each country or each year, its own baseline. The other coefficients then compare respondents within the same group., compares like with like on those characteristics and does not turn a description into a cause.

Where the numbers come from

No number in the prose is typed by hand. Each is printed from a facts file that the analysis writes, and hovering over it or focusing it shows its unit, population, period, source and, where it has them, its interval and sample size. A figure's title, caption and text alternative draw their numbers from the same file, and its "Show data" table lists the values behind every mark.

The data sources page lists each dataset with its years, weights and license. The glossary defines the terms of art that the text marks. The figure gallery lists every figure in the book.