Data Driven Decision Making

Part V · Chapter 16

LLMs, Workflows, and Governance

An LLM is a language interface for workflows — value lives in the wiring, the gates, and the governance, not the model.

This chapter reframes the language model as a programmable component in a workflow rather than a chatbot, then builds out the discipline that keeps it shippable. It opens with the language-shaped tasks an LLM does well and the six-slot prompt brief that specifies them — leaning on the GABRIEL finding that once a construct is clear, phrasing barely moves the answer. From there it forces machine-readable JSON behind a schema contract, wraps the model in tools and a human-approval gate to make an agent, and lays down a governance layer: an eight-dimension evaluation rubric, a risk-control map, and a one-page AI Workflow Card. The capstone wires every Part V method into the Bean & Basket Customer Voice Intelligence Studio as one monitored loop a sponsor can sign off on.

Start reading

Topics covered

the six-slot prompt briefthe GABRIEL phrasing-invariance findingreasoning vs. fast modelsJSON schema as a contractper-field confidence thresholdstool use and the control loopthe human-approval gatethe eight-dimension evaluation rubricthe risk-control map and AI Workflow Card

In this chapter

  1. 16.1LLM Capabilities and PromptingMaps the eight language tasks LLMs do well, where they fail, and the six-slot prompt brief that specifies any of them.
  2. 16.2Structured Outputs and ExtractionTurns messy text into validated JSON, treating the schema as a contract with confidence thresholds routing low-certainty records to humans.
  3. 16.3Agents and Tool UseDefines an agent as model plus tools plus a loop, with the human-approval gate as the central risk-control design choice.
  4. 16.4AI Evaluation, Risk, and GovernanceProvides the eight-dimension evaluation rubric, risk-control map, and one-page AI Workflow Card that make a workflow auditable.
  5. 16.5Customer Voice Intelligence StudioWires every Part V method into one monitored Bean & Basket customer-voice loop: classify, measure, cluster, retrieve, act, monitor.