Agent Sandboxing and Observability

Bound what it can do. Record what it did.

Talk to us
Runs against
One agent workflow and its real runs
Two builds
A sandbox and an evidence layer
You get
A blast-radius statement and replayable runs

An agent can return HTTP 200 and still be wrong

Two questions decide whether an autonomous system is safe to run: what is the worst thing it can do, and can you explain what it actually did? A sandbox answers the first. Traces answer the second. Neither answers the other.

Teams usually have half of one. There is a container, so the reach feels bounded; there are logs, so the behaviour feels recorded. Then something goes wrong and nobody can say which credentials the run held, which documents it read before it decided, or whether the retry that fixed it also sent the message twice.

So we build both halves, and we build them against the same run identifier — because a boundary you cannot audit and a record you cannot act on are each half a control.

The sandbox boundary, and the trace every crossing leaves An agent loop sits inside an execution sandbox that bounds its filesystem, network, credentials and time. Two actions leave the sandbox toward the systems it changes. One passes straight through. The other is held at an approval gate before it can take effect. Both crossings, whatever their outcome, are written to a trace holding the goal, the model and prompt versions, the tool calls, the cost and the approvals. The boundary stops an action; the trace explains it. The star marks the trace: runs you can replay, which you keep. Bound the reach. Record the crossings. Execution sandbox Agent loop Systems and artifacts Trace A sandbox stops it. A trace explains it. filesystem, network, credentials, time plan, act, check approval goal, versions, tool calls, cost, approvals The sandbox boundary, and the trace every crossing leaves An agent loop sits inside an execution sandbox that bounds its filesystem, network, credentials and time. Two actions leave the sandbox toward the systems it changes. One passes straight through. The other is held at an approval gate before it can take effect. Both crossings, whatever their outcome, are written to a trace holding the goal, the model and prompt versions, the tool calls, the cost and the approvals. The boundary stops an action; the trace explains it. The star marks the trace: runs you can replay, which you keep. Bound the reach. Record the crossings. Execution sandbox Agent loop Systems and artifacts Trace A sandbox stops it. A trace explains it. filesystem, network, credentials, time plan, act, check approval goal, versions, tool calls, cost, approvals
The gate and the trace are different controls. One decides whether an action happens; the other decides whether anyone can say afterwards what did. A system with only the first cannot be improved; a system with only the second cannot be trusted.

What the work covers

  • Bounded execution The smallest environment that can do the work — which files, which domains, which binaries, and what CPU, memory and time the run may consume.
  • Scoped, short-lived credentials Available for the run rather than to it, and revocable while it is still going.
  • Reversibility and approval gates Previews and idempotency keys on the actions that change something, and irreversible ones gated before the side effect rather than reported after it.
  • A kill switch One that revokes a running agent’s capabilities, not merely its process — those are different things, and only one of them stops an in-flight write.
  • Run traces under one identifier Goal and policy context, model and prompt versions, retrieved documents, tool calls, state transitions and retries, cost, approvals and final artifacts, joined per task.
  • The path, not just the answer Wrong tool calls, unnecessary loops, unsafe actions, failed retries and review burden — the measures that move before the success rate does.

A trace does not have to retain private reasoning to be useful. We store what the product, security and compliance teams can justify keeping, and write down why for each field.

How the engagement runs

Four stages, each ending in something you keep. No weeks are attached: the length is set by how much the agent can currently reach, which is exactly what the first stage exists to find out.

  1. Stage 1 Blast radius A written statement of the worst thing this agent can do today, which systems it would touch doing it, and who would find out. It is a short document and it is regularly the most uncomfortable one. You get: a blast-radius statement, and the list of irreversible actions inside it.
  2. Stage 2 Sandbox The smallest environment that can still do the work, credentials scoped and short-lived, egress on an allowlist, and the irreversible actions put behind gates. You get: the sandbox running, and a reach you can state in one sentence.
  3. Stage 3 Evidence A trace schema, artifacts and evaluator scores joined under one task identifier, with retention decided deliberately rather than by whatever the logging library does. You get: replayable runs, and a written retention decision per field.
  4. Stage 4 Drill A real failed run replayed and explained end to end, and the kill switch exercised on a live run — because a control nobody has pulled is a control nobody knows works. You get: one failure explained from the trace, and a kill switch you have seen fire.

What you leave with

  • The blast-radius statement — the document that makes the rest of the engagement arguable.
  • The sandbox — scoped filesystem, network, credentials and time, with the gates on what cannot be undone.
  • The trace schema — in your telemetry, joined to the same identifier the evaluation work uses.
  • A rehearsed kill switch — and the record of it revoking a live run’s capabilities.

The method, published

Layer 5 sets out the eight things a sandbox has to define and the controls that sit above it — previews, idempotency keys, transaction boundaries, an egress allowlist and a kill switch. The rule it lands on is the one this engagement implements: give the agent the smallest environment in which it can complete the task.

Layer 7 is the other half — what a trace captures, the nine metrics worth watching, and the four levels of evaluation from component tests to adversarial ones. What not to do names the failure this page exists to prevent: measuring only the final answer, when the path is where the problem is.

Both halves are one row of the plane model: the execution plane asks whether every action can be limited, observed and reversed, and the evidence plane asks whether the team can prove what happened and improve the workflow.

Tell us the worst thing one of your agents could do today. If that takes more than a sentence to answer, stage one is the engagement.

Talk to us

Related services

AI and Agent Engineering

Agent Production Readiness

Assess an agent system against the eight layers a production stack needs, and get a ranked plan to close the gaps.

Tell me more
AI and Agent Engineering

Agent Evaluation and Cost per Task

Instrument what your agents actually complete, and what each completed task costs once retries and review are counted.

Tell me more
AI and Agent Engineering

AI Governance and Data Boundaries

Model registry, routing policy and a hard data boundary, so regulated workloads can use AI and prove how.

Tell me more