Glossary
Terms of art, each defined once in plain language. In the chapters, a marked term shows its definition on hover or focus.
- AUC
- Area under the ROC curve: the chance that a randomly chosen positive record gets a higher score than a randomly chosen negative one. It measures ranking, not calibration.
- Calibration
- Whether predicted probabilities match observed frequencies: among items given a probability of 0.3, about three in ten should turn out positive.
- Checkpoint
- A point in an agent's work at which a controller chooses the next action.
- Constructed set
- Examples written for this booklet with known answers, used where sampled data cannot show a case clearly.
- Cross-fitting
- Splitting the data into folds, fitting on all but one fold and scoring the held-out fold, so that no item is scored by a rule fitted on it.
- Cutoff
- The probability above which a policy takes an action.
- Expected loss
- The average cost of an action, weighting each possible outcome's cost by its probability.
- Sample weight
- A number attached to each sampled item so that weighted results describe the population the sample was drawn from rather than the sample's own mix.
- Uplift
- The change in an outcome for a given person when an action is taken, compared with the same person without the action.