AutoML, Explainability, and Model Cards

This article maps that shift. We look briefly at what an AutoML leaderboard does (and doesn't) tell a manager; we work through the two interpretability tools — feature importance and partial dependence — that survive into production; and we land on the model card, the one-page document that should accompany every shipped model.

The Model Card is the third entry in the artefact family introduced at §0.4. The Decision Question Card (§5.1) frames the original action; the Predictive Task Contract (§9.2) specifies the prediction; the Model Card describes the system that ships. Each one extends the discipline of the last. The chain continues into §16.4 (AI Workflow Card) and lands at the §17.2 Decision Memo.

The Executive Question

If algorithm selection is mostly automated, what is the manager's job in the model — and how do we make sure the model behaves the way it claims to?

The short answer: the manager defines the task, audits the features, sets the threshold and cost matrix, signs off on the limitations, and owns the monitoring. Everything in this article is in service of those five duties.

What AutoML Does and Doesn't Do

A typical AutoML run will:

1. Take a labelled dataset and a metric (AUC, RMSE, log-loss).
2. Iterate through a configurable space of preprocessing pipelines and model families.
3. Cross-validate each candidate.
4. Return a leaderboard ranked by the chosen metric and a chosen "winner" pipeline that the user can deploy.

This automates a lot of the historically valuable work, well. The leaderboard's quality, however, is only as good as the task definition that produced the labels. AutoML does not check:

• Whether the target represents what the firm actually wants to predict.
• Whether features could be observed at decision time.
• Whether the units are aligned with the action.
• Whether class imbalance has been handled or hidden.
• Whether the cross-validation reflects the deployment split.

In other words, AutoML automates the easy half. The hard half — the half this entire book is about — remains the manager's responsibility.

A practical guideline: treat the AutoML leaderboard as a menu of candidate models, not as a deployment recommendation. The model the team ships is the one whose threshold–profit curve looks best under the production-shaped evaluation from §10.2.

Feature Importance: Inspection, Not Causation

Once a model — single or ensemble — has been fit, the standard inspection move is to ask which features it relied on most.

Two flavours of importance dominate:

• Tree-based importance. For random forests and boosting, the standard summary is total impurity reduction across all splits that use a feature. Fast to compute, dependent on the specific tree structures, and biased toward high-cardinality features.
• Permutation importance. Shuffle a feature's values across rows and measure the drop in held-out performance. Slower, more reliable, model-agnostic. The honest first choice when you have the compute budget.

A standing rule, restated as often as needed: importance is not causation. A high importance on support_tickets_last_30d tells you the model uses tickets to rank customers, not that reducing tickets will reduce churn. A retention program that intervenes on a high-importance feature is doing causal inference, not prediction, and the Part III framework applies.

Partial dependence plots complement importance by showing, for a chosen feature, how the model's prediction changes as that feature varies (averaging over the others). A churn model's partial dependence on days_since_last_purchase may rise steeply between days 14 and 30 and plateau afterward — useful information for designing intervention timing, with the same causal caveat.

Local Explanations: SHAP and Friends

For per-customer explanations, modern practice leans on SHAP (Shapley additive explanations). For each prediction, SHAP attributes the model's deviation from the baseline to each feature, with attributions that sum to the actual prediction.

The strongest use for SHAP is debugging and regulatory accountability:

• A high-confidence prediction with a single-feature attribution dominating is a red flag for leakage.
• Local explanations are the kind of artefact a regulator or a customer-care team can actually act on ("why was this loan flagged?").

The weakest use is causal storytelling. A SHAP value of −0.12 for discount_share on a specific customer's churn probability is a model attribution, not a counterfactual. The same warning carries forward.

The Model Card

Every shipped model needs a one-page contract that captures what it is, what it is for, and what it can and cannot do.

The card should include, at minimum:

• Intended use. What decision the model supports, for which customers, on what cadence.
• Target and features. What is being predicted, what windows the features were computed over, and what the unit of prediction is.
• Training data. Dates, populations, exclusions.
• Held-out performance. Headline metrics (AUC, PR-AUC, calibration, lift at decile) on a production-shaped held-out set.
• Known failure modes. Subpopulations where performance is poor, edge cases, types of input the model should not be scored on.
• Fairness review. Whether the model was audited for disparate performance across protected groups; what was found.
• Refresh cadence. When the model retrains, what triggers an early refresh, who owns the schedule.
• Owner. A real human and a real team.

A model without a model card is not yet operational. It is a research artefact that has accidentally ended up in production.

Putting Chapter 10 Together

Across Chapter 10 we have built a single recipe:

1. Start with a logistic regression baseline (§10.1) on a thoughtful feature catalog from §9.4.
2. Grade it with the full classification evaluation toolkit (§10.2), ending with a threshold–profit curve and a marked operating point.
3. For numeric targets, swap classification metrics for numeric prediction evaluation (§10.3).
4. Move to trees and ensembles (§10.4) only when the baseline's bias is the binding constraint.
5. Ship the model with a model card and inspection artefacts — feature importance, partial dependence, local explanations — that survive contact with audit, compliance, and rotation of personnel.

That recipe is conservative on purpose. The chapters that follow — segmentation, targeting, ranking — will show how the score, once produced, becomes part of a larger customer system. The discipline of Chapter 10 is what makes those downstream systems trustworthy.