§0.4

The Data-to-Decision Loop

The modern data system is a loop. Human and machine activity creates records. Records move into storage. Storage feeds transformations, metrics, charts, models, search indexes, and AI workflows. Those evidence assets inform decisions. Decisions change customer experience, operations, pricing, policy, product, staffing, or automation. Those changes create new data. The loop starts again.

The executive question: where are we in the data-to-decision loop?

Most analytics mistakes come from losing track of the loop. A team starts with a model but has not named the action. A dashboard monitors a metric but has no threshold. A causal analysis estimates an effect but does not connect to a decision cadence. An AI workflow answers questions but has no evaluation or escalation path. Each failure is a broken link between data and action.

The data-to-decision loop

Step 1
Human or machine activity
A customer acts, a process runs, a model responds
Step 2
Source record
A transaction, log, ticket, document, image, or prompt trace
Step 3
Storage and transformation
Operational DB, lake, warehouse, feature table, vector index
Step 4
Evidence asset
Metric, chart, causal estimate, prediction, retrieval result
Step 5
Decision and action
A manager changes a price, offer, process, policy, or workflow
Step 6
Feedback and monitoring
The action creates new data and the loop starts again
The loop is circular, not linear. Every decision changes the business, and that changed business generates the next round of data.
Figure 1. The data-to-decision loop. Every decision changes the business, and the changed business generates the next round of evidence.

Figure 1 is the front-door operating model for the book. Part I begins in the source record. Part II turns records into visual evidence. Part III asks whether an action caused an outcome. Part IV predicts and ranks future cases. Part V brings text, documents, images, embeddings, and language models into the loop. Part VI asks how the organization runs the loop repeatedly without losing ownership, quality, or governance.


Data-driven versus data-decorated

A decision is data-driven only when three things are named:

  1. The action. A specific lever someone can pull.
  2. The comparison. What would happen if the firm did not act or acted differently.
  3. The threshold. The signal, effect size, ROI, quality level, or risk standard required to act.

Anything missing one of these may still be useful description. It may be a good dashboard, a good analysis, or a promising model. But it is not yet a decision.

Table 1. Data-decorated work often has impressive evidence but a missing decision link.
Failure patternWhat it looks likeMissing link
Dashboard without actionWeekly KPI review shows a metric fallingNo owner, threshold, drill path, or response playbook
Causal claim without counterfactualCustomers who received an email spent moreWhat those same customers would have spent without the email
Prediction without workflowA churn score ranks customers every MondayWhich offer, threshold, queue, and follow-up action the score triggers
AI without evaluationA chatbot answers from company documentsGrounding tests, refusal rules, escalation, monitoring, and ownership

The decision ladder

The book's evidence languages climb a ladder of business questions. The lower rungs do not disappear when the upper rungs arrive. A prediction model still depends on a clean target. A causal design still depends on a well-defined unit, timing, and outcome. A language model workflow still needs source data, storage, evaluation, monitoring, and human review.

The decision ladder

IDescription
What happened?
IIVisual comparison
Where & for whom?
IIICausal designs
What caused it?
IIIRegression / elasticity
How much does X matter?
IVPrediction
What is likely next?
VAI workflows
What does the text/image say?
VISystem view
How do we operate this?
Figure 2. The decision ladder. Each rung asks a different business question and requires a different evidence language.

The ladder is a routing device:

  • What happened? Read the data and define the metric.
  • Where and for whom? Visualize, segment, compare, and diagnose.
  • What caused it? Construct a credible counterfactual.
  • How much does the lever matter? Estimate effects, elasticity, and heterogeneity.
  • What is likely next? Predict, rank, and evaluate.
  • What does the text, image, or document say? Use unstructured data and AI workflows.
  • How do we operate this? Monitor, govern, communicate, and learn.

Six evidence languages

Six evidence languages, one per Part

Decision questionEvidence languagePartStudio
What happened?Description, metricsIData Language Studio (§4.1)
What should the eye see first?Visual evidenceIIVisual Decision Brief (§8.2)
What caused it?Causal designsIIIPricing & Promotion (§13.4)
What is likely next?Prediction & segmentationIVCustomer Intelligence (§17.4)
What does the text or image say?AI workflowsVCustomer Voice Intelligence (§22.2)
How do we operate this?System viewVIFinal Integrative Case (§25.1)

Each Part teaches one evidence language and ends with a Studio that ships its capstone artefact.

Figure 3. The evidence languages. Each Part of the book adds one way of turning data into decision-relevant evidence.

The important point is not the numbering. It is the discipline of choosing the evidence language that fits the decision. A dashboard should not be asked to prove causality. A predictive model should not be treated as an intervention. A language model should not be trusted because it is fluent. A causal estimate should not ship if no one knows what action it changes.


The artifacts that survive the work

The book does not end each method with "and now you know the method." It ends with an artifact that a firm can reuse, audit, and refresh.

The artefact family — five one-page documents that survive the work

Decision Question Card
What action, on what unit, with what counterfactual?
§9.1
Predictive Task Contract
What target, for what unit, on what horizon, with what features?
§14.2
Model Card
What does this model do, where does it fail, who owns it?
§15.5
AI Workflow Card
What does this workflow do, what governs it, who responds?
§22.1
Decision Memo
What is the recommendation, what evidence supports it, what next?
§24.1

Each artefact extends the discipline of the one above. The card you write at §9.1 grows into the memo you sign at §24.1.

Figure 4. The artifact family. These one-page artifacts turn analysis into reusable decision infrastructure.

The artifact family matters because modern analytics is not a sequence of one-off clever analyses. It is infrastructure for repeated decisions. A metric card can be reused in a dashboard. A predictive task contract can be reused by a modeling team. A model card can be used by risk, legal, product, and operations. An AI workflow card can be audited when the workflow changes. A decision memo can show what evidence led to action and how the firm will learn afterward.


The cases

One through-line company, Bean & Basket Coffee, appears throughout the book. Standalone cases add real empirical grounding where a specific method needs a richer dataset.

The case portfolio

Bean & Basket CoffeeThe continuous through-line

A multi-store specialty coffee chain with reviews, tickets, transactions, panel data, campaigns, products, stores, and an internal knowledge base. Appears in every Part.

Standalone case studies
Progresso Soup
Pt II, Pt III
Visual evidence, fixed effects, elasticity
Milk Field Data
Pt III
Quasi-experiment, heterogeneous effects
Zillow Colorado
Pt III
Difference-in-differences, synthetic control
BAV Fast Food
Pt IV
PCA, perceptual maps
Airbnb (illustrative)
Pt IV
Numeric prediction, residuals
Yelp Reviews
Pt V
Sentiment, topics, GPT measurement
Goose Island Twitter
Pt V
Emotion vs. sentiment
Earnings Calls
Pt V
Evasiveness measurement
Job Postings
Pt V
Construct measurement

Standalone cases are appended outside chapter prose. They give the methods a second testing ground beyond the Bean & Basket through-line.

Figure 5. The case portfolio. Bean & Basket provides continuity; standalone cases give specific methods a second testing ground.

The purpose of the cases is not to decorate chapters. It is to make the loop concrete. Every case asks: what activity generated the data, where is it stored, what evidence language fits the decision, what artifact should survive, and how would the organization monitor what happens next?


Where Part I begins

Part I now begins after the full system is visible. We zoom in from the operating loop to the basic object inside it: a dataset. The first question becomes almost physical: what does one row mean? That question sounds small, but it controls everything that follows. Grain, structure, variable type, joins, reshaping, metrics, and data quality are the mechanics that make the larger loop trustworthy.

Concept check

Three questions about the loop and the evidence language it requires.

  1. 1.
    A dashboard shows that customers who received a marketing email spent more than customers who did not. The team wants to roll out the email nationally. What is missing?
  2. 2.
    A customer-facing AI assistant answers from policy documents. Which artifact is most natural?
  3. 3.
    Why does Part I begin with rows, columns, and grain after the broader Part 0 map?