The last AI built by humans?
Toward genuine recursive self-improvement: a beginner-friendly tour of the autonomy ladder, the evidence, and how to build a credible improvement loop.
RSI is more than an AI fixing its last answer.
The key test is whether a validated change survives into the next round and changes how future improvements are found, judged, or retained.
Think of the difference between a developer fixing one bug and a team improving its coding process, test suite, memory, release gate, and ability to find the next bug.
The bottleneck is moving from model output to model development.
As systems become more agentic, improvement requires more experiments, more interaction data, more verification, and more human coordination.
Expensive foundations
Data, architecture, optimisation, evaluation, hardware, and failure recovery all scale together.
Learning signals are costly
Synthetic data, reinforcement learning, self-play, and practice still need reliable generation and retention.
Real work keeps changing
New documents, tools, workflows, and environment states create recurring failures after launch.
Interactive capabilities still have more headroom.
The paper normalises heterogeneous benchmark results with its Headroom-Closed Index, or HCI: 0 is the entry-year frontier and 100 is a perfect score.
Headroom is a hypothesis about opportunity, not a forecast.
The source uses an illustrative post-2026 extension to show how verified repeated experience, regression testing, and retained updates might focus effort on weaker deployment workflows.
What a useful RSI loop could buy you.
Better research cycles
Retain hypotheses, failed branches, executable methods, and evidence so the next experiment starts smarter.
Safer practice
Let current capability shape the next simulated task, while keeping physical trials gated and reversible.
Self-improving harnesses
Turn recurring repository failures into tested skills, tools, context rules, or harness changes.
Governed adaptation
Accumulate validated clinical knowledge and workflow safeguards without unrestricted trial and error.
The practical benefit is not “no humans”. It is shifting human effort towards mission, evidence quality, protected evaluation, and consequential approval.
How does recursive improvement earn its name?
Start with the loop. Then ask what changes, what persists, and who still controls the decision.
Every credible loop has a carrier of change.
Ask three questions
Where does the loop close? Does the change return to the AI system?
What is updated and inherited? Weights, memory, tools, harness, evaluator, or policy?
Which decisions stay external? Objective, budget, safety boundary, acceptance, or release?
An accepted change becomes an improvement only when it enters the next system state. The next system is the successor.
Not every automated improvement process is RSI.
| Capability | Continual learning | AutoML | Agentic AI / ML | RSI |
|---|---|---|---|---|
| Persistent retention | ✓ core | ◦ sometimes | ◦ sometimes | ✓ core |
| System self-modification | ✓ possible | ◦ boundary | × not required | ✓ core |
| Candidate proposal | × | ✓ | ✓ | ✓ |
| Successor re-entry | ✓ | × | × | ✓ |
| Mechanism revision | × | ◦ boundary | × | ✓ |
| Mechanism reuse | × | × | ◦ sometimes | ✓ |
The distinction is functional, not branding: does a persistent improvement loop form around the system itself?
Five levels of responsibility, plus the non-RSI baseline.
Output change without persistent system change.
Examples include self-refinement, reflections, and tree search that end with the task.
AI runs the improvement recipe. Humans still write the recipe.
human-defined
for next task
Filtering, labelling, synthetic supervision, and quality pipelines at scale.
Profiling, code changes, deployment adaptation, and runbook-guided recovery.
When the procedure meets an unanticipated condition, the system can’t independently redesign the procedure.
AI chooses which improvement to try next.
failure
Prompt → agent harness → workflow graph → model configuration → training rule → kernel implementation.
Still external: the objective, task boundary, evaluation criterion, and acceptance rule.
AI decides what it should learn from next.
The evolving learner changes the next learning agenda. That is the crucial feedback connection.
Generate near the frontier
Use current errors, uncertainty, or solver consistency to propose tasks that are difficult but learnable.
Proposer meets solver
A challenger or task generator adapts as the learner changes.
Explore the environment
The agent chooses a useful next practice goal and turns success into a reusable skill.
Live interaction becomes a source of persistent adaptation.
Pagination failure → reusable skill
Observed: stops after page 1 Evidence: next_page = 2 Change: loop until next_page is null Gate: sandbox + paired task tests Retain: “collect every page” skill Reuse: later API tasks
What can change
Memory, skills, tools, code, harness, or weights.
What stays governed
Objective, access boundaries, protected evaluation, release authority.
L4 is about deciding which consequences of experience persist in deployment. It isn’t proof that the adaptation remains useful under distribution shift.
The improvement mechanism becomes the improvement target.
Agent adapts; process stays fixed.
It learns a new skill or updates its harness, but the method that diagnoses and accepts updates remains unchanged.
Process adapts and is inherited.
A revised improver, evaluator, search policy, or research policy governs later successor rounds.
Structural vs effective L5
Structural: a change persists and is invoked in a later improvement round.
Effective: the revised mechanism produces or selects better successors under matched budgets and independent assessment.
Measure the trajectory, not just the latest score.
Adaptivity
Gain, improvement trajectory, and time to target.
Retention
Replay loss, forgotten capabilities, regressions, and recovery.
Transfer
Held-out gains within and across domains.
Efficiency
Tokens, time, total cost, review effort, and rework per validated gain.
Stability
Harmful updates and the largest temporary decline.
Meta-recursion
Mechanism reuse, successor quality, goal choice, and stopping decisions.
The same loop behaves differently in each world.
The feedback may be executable, simulated, delayed, physical, clinical, or fundamentally uncertain. That changes what “safe inheritance” can mean.
Four domains, four different verification problems.
Evidence is sparse and open-ended.
Failed experiments are hard to attribute. Provenance, assumptions, uncertainty, and reproducibility must travel with the result.
The policy chooses what it gets to see.
Experience is endogenous, components are coupled, and physical trials can’t always be reset or reversed.
Tests are powerful, but conditional.
Executable feedback is fast, yet passing tests depends on the specification, coverage, repository, and environment.
Learning cannot be unrestricted trial and error.
Outcomes are delayed and confounded. Updates may not transfer across populations or institutions.
Industry is already building pieces of the loop.
Environment → data → model → environment
The paper reports workspace pilots where reconstructed environments improved rubric scores by 18.65-39.67 percentage points across five fixed model-harness pairings. The full sustained cycle remains to be established.
Delivery feedback → quality-system update
Reported internal comparison: key-defect packages fell from 9.0% to 3.7%, while average handling time fell from 48 to 27 minutes, after four feedback-driven updates.
Executable experience bank
Past code, logs, scores, and insights are recombined to guide parallel proposals. The evaluator can also be strengthened when loopholes appear.
Research artefacts as inheritance substrate
The paper reports 93.7% vs 72.4% QA over prior work and 64.4% vs 57.4% reproduction success for agent-native vs conventional artefacts.
All figures on this slide are reported by the preprint. They aren’t independently verified here, and some are company-reported or preliminary.
Persistence turns small mistakes into system behaviour.
Wrong diagnosis. The fix targets the symptom, not the component that caused the failure.
Experience corruption. A weak difficulty proxy, judge, or memory entry steers later learning.
Library drift. More skills can degrade retrieval and stall progress.
Evaluator exploitation. The system learns to score well, not to become more capable.
Capability displacement. A new behaviour improves one task and quietly breaks old ones.
Keep an independent anchor.
Protected evaluation, matched budgets, version histories, held-out tasks, and rollback provide the boundary between “the score changed” and “the system improved”.
Human authority remains explicit
Mission, safety constraints, access permissions, resource limits, and consequential releases should remain governed.
Build the smallest credible loop first.
Start with a bounded, reversible target and prove that an update helps later tasks before increasing autonomy.
Choose one target and define the state that can persist.
Pick a bounded target
Examples: a retrieval policy, coding-agent harness, skill library, research workflow, or evaluator wrapper.
Define the successor state
Versioned files, memory entries, tools, prompts, or model weights that the next round will actually load.
{
"target": "agent_harness",
"state": "git_commit + skill_registry",
"objective": "fewer incomplete repository changes",
"protected_eval": "hidden_regression_suite",
"human_gate": "required_before_release"
}Capture evidence so a future agent can tell what happened.
Input
Task, environment, model, tools, constraints, and initial state.
Trace
Actions, intermediate outputs, errors, tool results, and human corrections.
Outcome
Tests, scores, costs, latency, side effects, and uncertainty.
Lineage
Parent version, proposed diff, decision, and later reuse.
parent_state proposed_change motivating_evidence evaluation_config acceptance_decision supporting_trace applicability_conditions rollback_pointer
The paper argues that future successors need failed branches and operating conditions, not only polished final answers.
Turn recurring failures into testable intervention hypotheses.
“Make the agent smarter.”
Too broad to attribute, test, or roll back.
“Add a pagination check to the API-retrieval skill because 7 of 12 failures stopped before next_page was null.”
Specific cause, evidence, change, and predicted effect.
Validate before you let the change persist.
same tasks same base model same tools same budget control: current state treatment: candidate state compare: - target outcome - old capabilities - cost / latency - side effects
Fast gate: unit tests, compilation, schema checks, sandbox, safety filters.
Regression gate: fixed protected tasks and previously solved capabilities.
Transfer gate: fresh tasks, held-out environments, or a different repository.
Human gate: required for consequential changes or uncertain evidence.
Retain selectively, reuse explicitly, and make rollback cheap.
Write to the approved state
Attach evidence, scope, dependencies, and the validation result.
Make reuse observable
Log whether the next agent retrieved and followed the skill or harness change.
Manage drift
Cap the library, track contribution, and revalidate or retire stale artefacts.
Rollback pointer
Every accepted update should identify its parent and provide a one-action revert path.
Release log
Record who or what approved it, what it changed, and what later evidence says.
Only graduate to higher autonomy when the evidence earns it.
Three safe places to experiment with RSI thinking.
Repository-grounded agent improvement
Cluster recurring implementation failures, propose one harness or skill change, run the golden test suite, and retain only a versioned, reversible improvement.
Evolutionary narrative systems
Use playtest traces to evolve crisis-generation rules or NPC memory, while keeping the game’s design constraints and protected scenarios fixed.
Agent-native research inheritance
Persist hypotheses, failed branches, provenance, executable analyses, and validation status so each research cycle inherits evidence rather than only prose.
The paper uses software engineering, science, and research infrastructure as examples where feedback can be executable or traceable, but it doesn’t evaluate these three projects.
These are proposed applications of the framework to your projects, not findings reported by the paper. Keep them bounded, offline or sandboxed, and reversible.
Genuine RSI is a governance and evidence problem as much as a capability problem.
Persistence is the first boundary.
Fixing an output is not the same as changing the system that produces future outputs.
Autonomy has layers.
Execution, strategy, experience, deployment, and meta-improvement should be evaluated separately.
Verification must survive recursion.
If the evaluator changes, the comparison may change too. Protect anchors and preserve lineage.
Open questions: long-horizon accumulation, cross-domain transfer, cost-aware stopping, independent evaluation, and safe adaptation in science, robotics, software, and healthcare.
Read the preprint with the right level of confidence.
The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement
Yi Duan, Ying Liu, Zirui Tang et al. arXiv:2609.11873v1, 10 September 2026. Supplied PDF, 75 pages.
Blue source notes identify where the paper supports a claim. Slides marked “recommendation / implementation inference” extend the framework into a practical design pattern.
Company-reported and preliminary results are labelled as such. No claims in this deck have been independently reproduced.
This is an educational explainer, not a deployment, safety, medical, or investment recommendation.