Code as
Agent Harness
Toward executable, verifiable and stateful agent systems.
A beginner-friendly guide to the paper’s core idea, system taxonomy and practical build path.
The agent gets better when its work becomes executable.
Instead of asking a model to remember everything and declare success in prose, give it a governed workspace where it can write plans, run tools, inspect state, test changes and recover from failure.
Executable
Intent becomes a program, tool call, test or workflow that can actually run.
Inspectable
Plans, diffs, traces, logs and test results expose what happened between prompt and outcome.
Stateful
The system can preserve useful state across steps, sessions, agents and context resets.
Long-horizon autonomy is mostly a systems problem.
A stateless model improvises
It may lose the thread, hallucinate an API, repeat a failed action, or stop because the answer sounds plausible.
A controlled system learns from state
It can decompose work, use tools, persist evidence, run checks, route failures and ask for approval at the right boundary.
Judgement
Reason, interpret, propose.
Control surface
Expose tools, memory, permissions and feedback.
Evidence
Return observable state, outcomes and constraints.
One idea, many agentic worlds.
The paper uses code as the common interface across environments that look very different on the surface.
Software engineering
Repository state, diffs, tests and CI become the agent’s working world.
Example: resolve an issue, run tests, produce a reviewable patch.
GUI and OS automation
DOM state, accessibility trees, screenshots and executable checks ground actions.
Example: update a record, verify the UI changed as intended.
Scientific discovery
Hypotheses, simulations, lab protocols and analysis pipelines form a program world.
Example: propose an experiment, run the pipeline, inspect results.
Embodied agents
Policies, skills, simulators and sensor feedback connect intent to physical action.
Example: compose a skill, simulate it, then execute under constraints.
Personalisation
Preference state and feedback loops turn recommendations into editable policies.
Example: respect explicit constraints, remember corrections, show rationale.
DevOps and enterprise
Workflows, permissions, approvals and audit logs govern consequential changes.
Example: investigate an incident, propose a change, gate deployment.
Code sits in the loop between intent and evidence.
Deep dive
From interface, to mechanisms, to a shared substrate for multi-agent systems.
The paper’s map has three connected layers.
Harness interface
How code connects the model to reasoning, action and environment state.
Harness mechanisms
How planning, memory, tools, control and optimisation sustain execution.
Scaling the harness
How agents coordinate through shared code, state, feedback and verification.
The unifying question
What must be executable, inspectable and persistent so the system can keep moving without losing control?
Three roles for code at the interface.
Code for reasoning
Externalise computation into programs, traces, symbolic solvers or proof scripts that can be checked.
Useful when arithmetic, logic, search or intermediate state needs stronger grounding than prose.
Code for acting
Translate intent into API calls, GUI actions, robot policies, behaviour trees or reusable skills.
Useful when actions need affordance checks, constraints or feedback-driven retry.
Code for environment
Represent repositories, simulators, tests, logs and state transitions as queryable program worlds.
Useful when the agent must inspect what changed and predict what happens next.
Reliability comes from mechanisms, not one clever prompt.
Planning
Decompose, search, structure and orchestrate.
Memory
Manage working state, evidence, experience and shared context.
Tool use
Expose governed APIs, shells, repos, sandboxes and validators.
Control
Turn feedback into bounded state transitions through PEV.
Optimisation
Use telemetry and replay to improve the harness itself.
Design implication
Every mechanism should answer: what state does it read, what state does it change, and what evidence says the change worked?
Failure implication
If the system cannot replay a failure, locate its cause or roll back the change, it is difficult to improve safely.
Use Plan → Execute → Verify as the system’s heartbeat.
Make intent explicit
Write the change contract: files, invariants, validation commands, risk tier, rollback point and acceptance criteria.
Ask before acting
Apply a bounded transition
Run the planned actions inside a sandbox with explicit permissions, resource limits and trace capture.
Keep the action observable
Let evidence decide
Use tests, compilers, linters, static analysis, runtime traces, fuzzers and human review as sensors.
Choose the next state
Memory is not a bigger context window. It is state management.
Compaction
Keep decision-relevant evidence active while preserving a full-fidelity handle to logs, traces and diffs.
Governance
Store less, but store better: provenance, validation status, freshness and who or what approved it.
Once you can observe the harness, you can improve the harness.
The paper’s important caveat: harness evolution is a code change to a safety-critical runtime. Every mutation needs a contract, replay, regression checks and a rollback path.
Scaling the harness
Multi-agent coordination is a shared-state problem before it is a topology problem.
Agents coordinate better when they share artifacts and oracles.
Specialised roles
Manager · planner · coder · reviewer · tester · executor. Each role owns a slice of the workflow, but the substrate keeps them aligned.
The tester should be as independent as possible from the coder to reduce circular validation.
Artifact-mediated interaction
Files, diffs, tests, logs, schemas, blackboards and workflow state are more durable than conversational hand-offs.
The key question is which artefact is authoritative when channels disagree.
Plan
Translate intent into obligations.
Build
Create or revise the programme.
Challenge
Test, fuzz, review or debate.
Converge
Accept, revise, merge or escalate.
Four ways to represent the shared harness.
Implicit / file-only
Simple and common. But agents reconstruct state from the latest context, so belief can silently diverge from reality.
Repository-based
Navigable files, dependencies, call graphs and version history. Stronger structure, still weaker on runtime behaviour.
Execution-based
What matters is what the programme does: tests, traces, security findings, performance and simulation results.
Blackboard / shared state
A persistent, queryable store that all roles can read and update. Closest to a formal harness substrate.
Executable feedback helps. It does not make the oracle true.
Weak verification
A green test can still miss the real specification, unsafe intermediate actions or scientific validity.
Regression-prone evolution
A harness change can improve a benchmark while weakening safety, cost or rare-case behaviour.
Semantic conflicts
Agents can share files while disagreeing about assumptions, freshness or user intent.
Safety as durable state
Permissions, approvals, denials and corrections should become auditable state, not one-off interruptions.
Multimodal grounding
Screenshots, sensor readings and physical states need persistent evidence, action grounding and calibrated uncertainty.
Harness-level evaluation
Measure trajectory efficiency, recovery, state consistency, safety compliance and replayability, not only final success.
Build it
A practical, governed path from one reliable loop to a reusable harness.
Start with one verifiable workflow.
Choose a bounded task
Pick a workflow with observable inputs, reversible actions and a clear definition of done.
Define the state model
Name the source of truth: files, database rows, issue state, test results, traces or a shared blackboard.
Write the PEV contract
Specify the plan schema, allowed tools, validation checks, stop conditions and escalation rules.
Put execution in a sandbox
Separate read-only inspection, sandbox edits and full-access actions. Capture every transition.
Add independent verification
Prefer deterministic tests, static checks, traces or domain validators. Document what each oracle cannot see.
Replay before you optimise
Build a small golden set, replay failures, and change one harness component at a time.
Keep the first harness small enough to inspect.
# a minimal code-centric harness state/ task.json # goal + risk tier plan.md # change contract evidence/ # retrieved facts traces/ # full-fidelity logs approvals.jsonl # durable gates workspace/ repo/ # sandboxed worktree validators/ tests/ static_checks/ policies/ run.py # P → E → V loop
Minimal loop
plan = agent.propose(state)
check_plan(plan)
result = sandbox.execute(
plan.actions,
permissions=plan.risk_tier
)
evidence = validators.run(result)
state.record(plan, result, evidence)
if evidence.accepted:
promote(state)
elif evidence.recoverable:
revise(state, evidence)
else:
escalate(state, evidence)The point is not this exact folder layout. The point is that plans, actions, evidence, permissions and approvals are first-class artefacts.
Permission tiers are part of the agent state.
Observe
Browse files, inspect logs, retrieve documentation, run static analysis.
No external side effects.
Experiment
Patch a disposable worktree, run tests, install temporary dependencies under limits.
Reversible and isolated.
Affect reality
Network access, credentials, deployment, publishing, destructive changes or physical action.
Human approval, audit trail and rollback required.
Approval record
What was proposed? What evidence was shown? What risk was surfaced? Who approved it? What changed afterwards?
Context-sensitive policy
A command’s risk depends on its arguments, environment, data sensitivity and expected side effects, not only its tool name.
Translate the idea into your own agent work.
Shared project memory as a substrate
Make plans, decisions, artefacts, provenance and validation state queryable. Let agents work through durable files and explicit hand-offs rather than hidden chat history.
First experiment: a task ledger with read/write sets, evidence links and a PEV status.
Governed workflow agents
Treat product, data and engineering workflows as executable state machines with role-based permissions, evaluation gates and replayable traces.
First experiment: one low-risk workflow with a golden set and a human approval checkpoint.
Tool use with observable outcomes
Keep tool schemas typed, persist tool outputs, and attach deterministic checks to the agent’s generated action plan.
First experiment: record the plan, tool call, result, validator and repair attempt as one trace.
Provenance as a safety surface
Represent evidence, confidence, missing data and review status as explicit state. Keep high-stakes actions behind escalation gates.
First experiment: require an evidence bundle before an answer can be promoted.
Measure the harness, not just the answer.
Outcome
Did the task satisfy the actual specification, not only the visible proxy?
Trajectory
How many tokens, tool calls, edits, retries, executions and minutes were needed?
Recovery
Can the system diagnose and repair an injected failure?
State
Do memory, repository, traces and agent beliefs remain synchronised?
Safety
Were permissions, sandboxes, approvals and data boundaries respected?
Replayability
Can another person reconstruct what happened and why the system stopped?
Build agents as closed-loop systems.
Make important reasoning executable
Turn fragile prose into code, tools, tests, structured plans or formal contracts.
Make every transition inspectable
Persist the plan, action, evidence, decision and permission boundary.
Make state explicit
Decide what is authoritative, what is derived and how stale state is detected.
Make verification plural
Tests are powerful, but no single oracle sees the whole specification.
Make improvement reversible
Evolve the harness through telemetry, held-out replay, regression checks and human governance.
Read the paper as a systems map.
Primary source
Ning, X. et al. Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems. arXiv:2605.18747v1, 18 May 2026.
Paper-linked reading map
The authors maintain a curated collection of work on code as agent harness, including planning, memory, tool use, execution feedback and multi-agent coordination.
Deck note: claims about the paper are paraphrased and page-linked. Implementation patterns and project examples are clearly labelled as synthesis or recommendation.
Built as a self-contained interactive HTML deck. Use the overview button to jump to any slide, or the full-screen button for presentation mode.