Part IV · Chapter 12

Targeting, Ranking, and Operating ML

Turning scores into audiences, rankings, and a monitored system that still works six months later.

This chapter focuses on the point where algorithmic ideas stop living in a notebook and start spending a media budget. It follows the bridge from a clustered segmentation to a targetable audience on an ad platform, reframes a lookalike audience as nearest-neighbour scoring run at platform scale, then turns scores into ranked lists through collaborative, content-based, and learned recommenders graded with precision@k and NDCG. It closes the lifecycle with the discipline that keeps a model alive — separating data drift from concept drift, designing a four-KPI monitoring dashboard, setting retraining cadences, and weaving everything into a single Bean & Basket Customer Intelligence loop. The recurring lesson: high conversion proves selection, not causation, and only an incrementality holdout separates real lift from customers who would have converted anyway.

Topics covered

lookalike audiences as nearest-neighbour scoringthe reach-vs-similarity dialretargeting funnels and incrementality holdoutscollaborative vs. content-based recommendersmarket-basket analysis (support, confidence, lift)precision@k and NDCG ranking metricscold start and feedback loopsdata drift vs. concept driftmonitoring KPIs and retraining cadences

In this chapter

  1. 12.1From Segments to Targeting: Ad Platforms and LookalikesShows how segments become ad-platform audiences, recasting lookalikes as nearest-neighbour scoring and setting the reach-vs-similarity dial per campaign goal.
  2. 12.2Recommenders and RankingSurveys collaborative, content-based, and learned recommenders, grading ranked lists with precision@k and NDCG while navigating cold-start and feedback-loop traps.
  3. 12.3Deployment, Monitoring, and DriftDistinguishes data from concept drift and lays out the four-KPI dashboard, retraining cadences, and human-in-the-loop policies that keep models healthy.
  4. 12.4Customer Intelligence StudioWeaves score, segment, target, act, and monitor into one Bean & Basket loop, with a one-page executive brief tracing every claim to an artefact.