Skip to main content

Production Style Guide: the gate an AI clears before it runs unattended

The opinionated rules Cleon applies before an AI system runs on real traffic — the pre-ship gate as a binary checklist (cost ceiling, latency fallback, input guardrail, PII masked, a human on irreversible actions, the audit trail, a rollback ready, the eval gate passed), and the in-platform-versus-build-it matrix that says what Agentforce and the Einstein Trust Layer give you by construction versus what you assemble off-platform, dimension by dimension. The discipline document that turns the production gotchas into rules and the production-readiness principles into a checklist: an unmet row blocks the ship. And because this is the last page of the AI Engineering catalog, it ties the five subcategories together — agents, grounding, prompting, evaluation, production — into the single arc the whole discipline traces.

Decision framework·Last updated 2026-06-08·Drafted by Lira · Edited by German Medina

This is the page where Cleon stops describing what production readiness is and says what has to be true before an AI system runs unattended on real traffic. The reference pages lay out the parts — what production readiness is and the six dimensions it demands, the cost and latency levers in order of force, the guardrails and safety layer on each side of the model, PII and governance, human-in-the-loop and accountability, and deploying to production. The gotchas lay out the ten ways a demo lies about being ready. This Style Guide is the gate that decides whether a system is allowed near a customer — and the rule that decides it is binary.

The rules are short on purpose. When a rule needs an explanation, the explanation lives in the page it links to. This is the operational form of the AI Engineering principles — each row below is one of those principles with its sleeves rolled up, and we cite the number so you can trace the rule back to the reasoning. It is the run-it-safely companion to the agent Style Guide, the grounding Style Guide, the prompting Style Guide, and the evaluation Style Guide: the first three end a sentence with "eval every change," evaluation is the page that sentence points to, and this is the page that says what else has to be true before the evaluated change ships to real traffic.

The pre-ship gate: what must be true before this AI runs in production

This is the page that gives the gate its home. Before an AI system serves real traffic — its first launch, or any change to a prompt, model, agent, retriever, or tool that ships into production — every row below is true, or it isn't ready. Each row closes a gotcha that turned a working demo into a system nobody could run, audit, or stop. The "where it's covered" column points to the page that builds the row in full.

The checkWhy it blocks the shipWhere it's covered
A cost ceiling and an alert are setA retry storm, a recursive loop, or an unbounded context can 10x the bill overnight, and the first signal is the invoice — a hard cap per request and per session plus an alert before the cap turns runaway spend into a number you steer. (Principle 6.)Cost and latency · Production gotchas
A latency budget and a fallback existA correct answer that lands too slowly is a wrong answer to the user — set a target p95, not just a median, and give the system a fallback when it blows the budget: a faster model, a cached answer, a shorter context, or a graceful timeout. (Principle 6.)Cost and latency · Production gotchas
Input guardrails are on (injection screened)Every input is an attack surface, and the indirect case — adversarial instructions hiding in a retrieved document or a tool result — is the one teams miss; screen the input before the main model and treat retrieved content as data, never instructions. (Principle 5.)Guardrails and safety · Production gotchas
PII is masked and retention is handledReal customer fields walking to a third-party model with no masking and no retention floor is a regulatory finding, not a bug — mask or tokenize sensitive fields before the prompt leaves your boundary, and know the provider's retention terms per feature. (Principle 9.)PII and governance · Production gotchas
A human is in the loop on irreversible actionsA non-deterministic system will make a bad call eventually, and on a send, a delete, a payment, or a permission change there is no undo — reversible actions the agent takes autonomously, irreversible ones wait for a human to approve. (Principles 8, 9.)Human-in-the-loop and accountability · Production gotchas
The audit trail is onWhen the system does something wrong, the first question is "what did it see and what did it do," and if nothing recorded it you are guessing at a non-deterministic system after the fact — log inputs, retrieved context, prompt, output, and every action, enough to replay the decision. (Principle 11.)Human-in-the-loop and accountability · PII and governance
A rollback is readyA one-word prompt edit can shift behavior across the whole input distribution, and a bad change shipped to everyone at once leaves you authoring a fix-forward while production is broken — keep the previous version one switch away so rollback is instant. (Principle 1.)Deploying to production · Production gotchas
The eval gate passed"It worked in three tries" is the long-tail trap wearing a deployment hat — the change cleared a scored regression set sized so noise can't swing it before it shipped, because if you can't evaluate it you can't ship it. (Principle 3.)Evaluation Style Guide · What is evaluation

If any row is unmet, the change isn't ready. The two most expensive omissions: a system ships with no kill switch — no fast, out-of-band way to disable it — so when it misbehaves the damage runs for the length of a deploy; and an irreversible action ships with no approval gate, so the agent acts and you find out after. Both belong on the list above and neither has a row to itself by accident: the kill switch is the operational floor under "a rollback is ready," and the approval gate is the whole of "a human is in the loop."

In-platform versus build-it: the same dimensions, decided by where the system runs

The gate above is constant. What changes is how much of each row you get for free versus build yourself — and that is decided by where the system lives, not by which toolkit you pledged loyalty to. Like grounding, prompting, and evaluation before it, the surfaces compose: complementary instruments picked by where the system runs, never rival products to choose between (principle 7). On Salesforce, Agentforce wrapped in the Einstein Trust Layer hands you several dimensions as governance by construction; off-platform, on the Claude API plus your own infrastructure, every dimension is yours to assemble. The matrix is the no-"vs" spine made operational — read each row as "this is given here, and built there," not "this beats that."

DimensionAgentforce + Einstein Trust Layer (given by construction)Off-platform Claude API (you build it)
CostAgent runs are metered and visible on the platform; model selection and context are managed for youYou set the token budget and the model tier, cache the static prefix, batch the work that can wait, and meter spend yourself
LatencyThe platform manages model choice and context to keep responses in budgetYou pick the model, cap output with max_tokens, stream for perceived speed, and own the p95 budget and fallback
GuardrailsThe Trust Layer screens output for toxicity and attaches a score on every response; the platform's guardrails review applies to topics and actionsYou build the input harmlessness screen and the output classifier, treat retrieved content as data, and gate on the score
PII / governanceData masking before the prompt leaves the platform, a zero-retention agreement with the model provider, and secure data retrieval that honors the running user's permissionsYou mask or tokenize sensitive fields, confirm retention terms per feature, and redact the log yourself
Human-in-the-loopHuman escalation and approval are first-class moves in the flow; the high-stakes step routes to a person by constructionYou build the confirmation gate into the tool layer — the agent proposes, a human disposes, the irreversible call waits
AuditThe Trust Layer logs the interaction for audit; Agentforce traces turns, LLM calls, and actionsYou build the trace — inputs, context, prompt, output, actions — with PII redacted, the same stream your online eval scores
Deploy / rollbackChanges deploy through Agentforce DX with the platform's release tooling and sandboxes; agents can be deactivated through admin controlsYou version the prompt and model config, ship behind a canary, and wire the rollback path and the kill switch

Read across a row and the shape is always the same: the Trust Layer governs the language side — what the model is sent and what it says — but it does not govern what your Actions do, so the blast radius of an Action that can write or delete is yours to bound on either surface (principle 5). And you do not pick one surface and stop. A real system frequently runs both — an Agentforce agent owning the governed, on-platform actions on customer data, an off-platform agent owning a step it can't, with a clean handoff where accountability gets a seam (principle 9). The skill is composing the surfaces to the system you actually built.

The "ship it / don't ship it" rule

The gate is binary, and the rule is one line: every row is green, or you don't ship. There is no weighted average, no "ship now, harden later" on a row that bounds a real harm, no partial credit. A system with no cost ceiling ships fine and bankrupts a budget on the first traffic spike. A system with no approval gate on an irreversible action ships fine and deletes the wrong record on the first ambiguous input. The rows that feel skippable in the demo are exactly the ones that take the system down in production, because a demo never exercises the spike, the adversary, the long tail, or the un-sendable email. So you hold the line: an unmet row blocks the ship the same way a failing test does, and "we'll add it after launch" is the sentence that precedes the incident with no owner.

The one judgment the rule leaves you is how much of each row a given system needs — a read-only internal assistant carries a lighter audit and HITL burden than an agent that issues refunds — but the row itself is never skipped, only sized to the blast radius. Size each control to the cost of the thing going wrong, then confirm it's present before the system meets real traffic.

Smells: the anti-patterns this gate exists to refuse

  • "It worked in the demo, so it's ready." A demo is a sample of size three from the easy end of the distribution; production is the long tail. Make the eval the gate, not the demo. (Gotcha 9 · principle 1.)
  • No cost ceiling. A system with no spend cap and no alert can 10x its bill on a spike or a prompt regression, and the first signal is the invoice. Cap per request and per session, alert before the cap. (Gotcha 1 · principle 6.)
  • No latency budget. Tuning the median while the p95 makes the system unusable — set a p95 target and a fallback, or the user waits ten seconds and leaves. (Gotcha 2 · principle 6.)
  • Trusting retrieved content as instructions. An indirect injection rides in a fetched page, an inbound email, or a tool result; if you didn't screen it and didn't treat it as data, "ignore your instructions" goes straight in. (Gotcha 3 · principle 5.)
  • Real PII on a handshake. Sensitive fields leaving your boundary with no masking and no retention floor is a regulatory finding waiting for an audit, not a bug to fix later. (Gotcha 4 · principle 9.)
  • The agent acts, then you find out. No approval gate on a send, a delete, a payment, or a permission change means a non-deterministic system takes an irreversible action you can't undo. (Gotcha 5 · principles 8, 9.)
  • Logging nothing. When the system does something wrong and nothing recorded what it saw or did, you are guessing at a non-deterministic system after the fact. (Gotcha 6 · principle 11.)
  • Deploy with no way back. A prompt or model change shipped to everyone at once with no rollback leaves you fixing forward while production is broken. (Gotcha 7 · principle 1.)
  • No kill switch. If the only way to turn a misbehaving agent off is a code change and a deploy, the damage runs for the length of that deploy. (Gotcha 10 · principle 9.)

Closing: the whole arc, in five subcategories

None of these rows is hard to set up before launch; every one of them is expensive to discover live, because a missing control looks exactly like a present one right up until the spike, the adversary, or the irreversible action finds it. The throughline is the one the whole AI Engineering catalog circles back to: the model was never the hard part — the engineering around it is, and production is where that engineering is cashed out, capped on cost, budgeted on latency, screened on input, masked on data, gated on irreversible actions, logged for audit, deployable with a rollback, and killable in one move.

This is also the last page of the catalog, so it's worth saying the arc out loud. The five AI Engineering subcategories are one discipline read in order: agents is what you ship — the system wired to tools that act; grounding is what it knows — the retrieval the answers stand on; prompting is how you steer it — the context and instructions that make a capable model dependable; evaluation is how you measure it — the gate that proves a change is safe before it ships; and production is how you run it safely — everything above, the bar between a measured change and an AI a customer can trust. Each subcategory has a Style Guide, each Style Guide ends by pointing at the next link in the chain, and the chain closes here. The discipline is the whole arc, not any one link: a system grounded but ungoverned, or evaluated but unkillable, is a demo with extra steps. Ground it, steer it, measure it, run it safely — or it doesn't ship.

If you spot a row missing — or one of these rules being violated in our public work — write to hello@wearecleon.com. We add it, or we fix it and we say so.

Related

Reference: