Agent Production Readiness

Tell a prototype from a system you can defend.

Talk to us
Runs against
One workflow, end to end
Assessed against
Eight layers, sixteen questions
You get
A maturity rating and a ranked plan

The question production asks

A demo proves a model can produce a plausible answer. Production asks a harder question: can this system complete a valuable task reliably, at an acceptable cost, with the right data boundaries and enough evidence for a human or an auditor to trust it?

Most agent systems that reach a demo have never been asked it. They work along the path someone rehearsed, and the parts that decide whether they can be deployed — what happens after a failed tool call, which credentials the run actually holds, whether anyone can reconstruct what it did last Tuesday — were never built, because nothing in the demo required them.

So the useful thing to establish is not whether it works. It is which of those parts exist, which do not, and which to build first.

The maturity ladder, and the evidence each rung requires Four rungs read from the bottom up: prototype, pilot, production and portfolio. An arrow rises through each gap, and beside it the evidence a team has to produce before it can climb. Leaving prototype needs scenario tests, traces and an acceptable review time. Leaving pilot needs a stable task-success rate, a cost per completed task and an incident runbook. Leaving production needs cross-workflow governance, budget controls and drift monitoring. The gate, not the rung, is what the assessment measures. The star marks the top rung: a portfolio run under one governance, which you keep. Autonomy is earned, not enabled Portfolio Production Pilot Prototype Each rung is a gate, not a date. cross-workflow governance, budgets, drift monitoring stable success rate, cost per task, a runbook scenario tests, traces, acceptable review time The maturity ladder, and the evidence each rung requires Four rungs read from the bottom up: prototype, pilot, production and portfolio. An arrow rises through each gap, and beside it the evidence a team has to produce before it can climb. Leaving prototype needs scenario tests, traces and an acceptable review time. Leaving pilot needs a stable task-success rate, a cost per completed task and an incident runbook. Leaving production needs cross-workflow governance, budget controls and drift monitoring. The gate, not the rung, is what the assessment measures. The star marks the top rung: a portfolio run under one governance, which you keep. Autonomy is earned, not enabled Portfolio Production Pilot Prototype Each rung is a gate, not a date. cross-workflow governance, budgets, drift monitoring stable success rate, cost per task, a runbook scenario tests, traces, acceptable review time
A team earns more delegated authority by demonstrating reliability, not by turning on a more permissive agent mode. The assessment says which rung a system is on, and names the evidence missing from the next one.

What the assessment covers

We assess one workflow against the eight layers a production stack needs. Each is argued in full in The Production AI Agent Stack in 2026; the labels below link to the layer they came from.

  • Outcome Whether the system starts from a measurable business outcome rather than a model choice, and whether that outcome can be scored.
  • Orchestration and durable state Whether a run can resume from a checkpoint after a rate limit, a failed tool or a context-window limit, rather than restarting.
  • Model routing Whether work is matched to the right model by task, cost, latency and data residency, rather than sent to one vendor by default.
  • Context, memory and tools Whether tool access is typed, versioned and permissioned, and whether retrieval is a defined contract rather than an unbounded store.
  • Execution sandbox Whether the environment an agent acts in is the smallest one that can do the work, and whether a kill switch can revoke a running agent’s capabilities.
  • Identity, policy and approval Whether the agent inherits no more authority than the user, and whether irreversible actions are gated before the side effect rather than after it.
  • Observability and evaluation Whether you can explain what happened on any given run, and whether evaluations cover realistic failures and adversarial inputs.
  • The interface Whether the people responsible can see the objective, the current phase, what is blocked and what needs approval.

The layers are coupled, but they are not interchangeable. A sandbox limits execution; it does not tell you whether the result was correct. A better model reduces errors; it does not create an operating policy.

How the assessment runs

Four stages, each ending in something you keep. There are no weeks attached to them, on purpose: the method rates a system on the evidence it can produce, and putting a calendar on that would contradict it.

  1. Stage 1 Scope One workflow, followed end to end from request to accepted outcome — not the architecture diagram. We write the task contract with the people who own the process. You get: a task contract naming the acceptance test, the risk tier and the escalation path.
  2. Stage 2 Evidence Traces, tool definitions, policies, credentials and configuration read as they actually are, and a cohort of real runs observed — including the ones that failed, because that is where the remediation order comes from. You get: the run record the rating is built on, in your own telemetry.
  3. Stage 3 Rating The eight layers rated, and the sixteen checklist questions answered with a yes, a no, or the evidence missing to answer either way. That third answer is the common one, and the most useful. You get: a maturity rating per layer, and the rung it puts the system on.
  4. Stage 4 Readout Presented to engineering and to the sponsor together, so the gaps and the money to close them are discussed in the same room. You get: a ranked plan, each gap with what closes it and what has to come first.

What you leave with

Where the answer to several questions is no, we say so plainly: the system is still a prototype, however impressive the demo.

  • The task contract — yours to reuse on the next workflow, and the thing the rating is measured against.
  • The checklist, answered — sixteen questions with a yes, a no, or the evidence you would need to answer them.
  • A maturity rating per layer — and the rung it puts you on, with the evidence the next rung requires.
  • A ranked plan — each gap with the layer it sits in, what closes it, and what has to come first.

The method, published

We argued this in public before we sold it. The same article sets out the maturity model this assessment rates against, the sixteen-question checklist it answers, and the five things not to do that most systems have already done.

The distinction the whole assessment turns on is between deciding and acting: the model takes part in the control plane, but it is not the security boundary. Permissions, network access and approval rules have to be enforced outside the model-generated text, and a great deal of what we find is a system where they are not.

Send us one workflow you already run, and we will tell you which of the eight layers it already has.

Talk to us

Related services

AI and Agent Engineering

Agent Evaluation and Cost per Task

Instrument what your agents actually complete, and what each completed task costs once retries and review are counted.

Tell me more
AI and Agent Engineering

AI Governance and Data Boundaries

Model registry, routing policy and a hard data boundary, so regulated workloads can use AI and prove how.

Tell me more
AI and Agent Engineering

LLM Gateway and Model Routing

One vendor-neutral gateway in front of every model, so routing, cost and failover are yours to change.

Tell me more