WikiSkill
How agents turn messy execution experience into persistent knowledge, then use it to evolve reusable skills.
gets better
Based on Liyan Tang et al., “WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution”, Google Research, 27 August 2026. Read the paper ↗
An agent should remember what happened, not just rewrite its instructions.
Run real tasks
Save the agent’s actions, tool calls, feedback and final answers as raw traces.
Make patterns durable
Turn repeated successes and failures into a searchable wiki with evidence and history.
Promote only winners
Propose a small skill change, test it on validation tasks, and roll back if it gets worse.
Paper summary and Methodology, Sections 1–3. “Wiki” means a persistent knowledge layer, not a public website.
Most skill evolution has a short memory.
The agent may analyse traces and edit a skill, but the reasoning behind those edits can stay scattered across optimisation history. Useful lessons become hard to retrieve, compare or reuse.
WikiSkill inserts a durable knowledge layer between “what the agent experienced” and “what the skill tells it to do”.
Paraphrase of the paper’s motivation in Introduction, Section 1.
Three layers, three different jobs.
Click a layer to see what it protects, what it contains and who can read it.
Keep the evidence intact: the agent’s reasoning, tool calls, observations and final answers. Later components can inspect it, but the history itself is not rewritten.
Methodology, Section 3.1 and Figure 2.
A four-part loop turns experience into a better skill.
Inference agent
Runs training tasks with the active skill and writes traces to raw/.
Wiki maintainer
Performs root-cause analysis and consolidates useful patterns into wiki/.
Skill proposer
Reads the wiki and selected traces, then proposes one atomic skill update.
Gating + rollback
Tests the candidate on validation tasks. Keep it only if the score improves.
Methodology, Section 3.2 and Algorithm 1.
The learner sees the skill. The skill-maker sees the wiki.
The Inference Agent uses active skills, but is restricted from reading the wiki during training.
The Wiki Maintainer and Skill Proposer inspect traces plus accumulated wiki knowledge.
The paper’s ablation found that giving the Inference Agent wiki access during training reduced average performance from 63.7% to 60.9% in the tested Gemini configuration.
Make the traces diagnostically useful.
If the agent can solve tasks by directly consulting the wiki, its traces may reveal less about what the executable skill itself is missing. The authors describe this as a hypothesis, supported by the ablation rather than proven as a universal law.
The wiki learns from every attempt. The skill earns its place.
A candidate skill update is evaluated on a validation split against the best score so far.
Candidate becomes the new active skill and the best score moves up.
Skill reverts to the last winner. The wiki still records what was tried.
skill
improves
degrades
Methodology, Section 3.2.4 and Algorithm 1. The paper uses a strict “must improve validation score” rule.
WikiSkill was the strongest method on average across the five tested models.
Important: this is a visual summary of selected values, not a re-analysis. The underlying table reports averages over five benchmarks and three independent evolution runs; the paper reports paired bootstrap testing at p < 0.05.
Source: Table 1. Bars are scaled to 68.1%, the highest WikiSkill average in the table. Values shown are test accuracy percentages.
Model capability and procedural knowledge are complementary.
WikiSkill gains grew with model scale.
Average improvement over no-skill baselines was reported as +12.3 points for Qwen-3.5-4B, +17.5 for Qwen-3.5-9B and +23.9 for Qwen-3.6-27B.
Skills can compensate for model size.
Qwen-3.5-9B with WikiSkill reached 47.4% average accuracy, versus 39.4% for Qwen-3.6-27B without skills.
Results and interpretation from Section 4.2.1. Percentages are percentage-point differences in the paper’s wording.
A skill can travel. It can also carry the wrong assumptions.
Useful transfer +26.2 pts
Qwen-3.6-27B Spreadsheet skills lifted Qwen-3.5-9B from 24.3% without skills to 50.5%.
Negative transfer −32.4 pts
Qwen-3.5-4B Spreadsheet skills reduced Gemini-3.5-Flash from 50.5% without skills to 18.1%.
Source: Table 2 and cross-model analysis in Section 4.2.2. Differences calculated from the reported values.
The wiki turns a rejected idea into a concrete rule.
“Goal-directed action” is too abstract and fails to improve validation performance.
“Break repetition loop” adds a concrete rule: don’t return an item to its origin location.
New evidence leads to a sharper rule: perform each operation type once per item.
Source: Figure 3 and Case Study, Section 5.3. File contents in the paper’s diagram are simplified for clarity.
This is a memory architecture for improving procedures.
Make improvement inspectable.
Keep the trace, the diagnosis, the proposal, the score and the decision linked. “The agent got better” should have an audit trail.
Use two speeds of knowledge.
Raw traces are append-only evidence. Wiki patterns are curated understanding. Skills are the small runtime surface.
Test changes, not vibes.
Require a validation decision for each atomic update, while preserving rejected attempts as future context.
These are practical interpretations of the paper’s architecture and findings, not additional experimental claims.
Promising does not mean production-complete.
Retrieval is not tested
Active skills were directly injected into prompts. Skill discovery and triggering at scale remain open problems.
Strict gating can be myopic
Neutral changes are rejected even if they could enable a later improvement.
The wiki can grow without bound
The paper does not yet automate pruning or consolidation over long runs.
Long horizons remain open
The benchmarks do not cover tasks spanning hundreds of actions or multiple hours.
Limitations, Section 7. Treat this slide as the boundary around the paper’s evidence.
Build a tiny WikiSkill loop around one agent task.
Choose a narrow task family
For example: spreadsheet edits, document retrieval or a bounded tool workflow. Define train, validation and test splits.
Log complete traces
Capture inputs, observations, tool calls, outputs, errors, final answer and score. Never overwrite the original trace.
Ask a maintainer for patterns
Sample both failures and successes. Require root cause, evidence and a concrete workaround.
Ask a proposer for one change
Give it the wiki index, prior impact log and selected traces. Keep each proposal atomic.
Gate on validation
Evaluate the candidate with the same harness. Accept only if your chosen rule says it is better.
Record the decision
Store the diff, score and accepted/rejected outcome in the wiki so future proposals do not repeat dead ends.
Implementation sequence adapted from Algorithm 1 and Sections 3.1–3.2. The guide adds practical engineering detail.
Keep evidence, knowledge and runtime instructions visibly separate.
├── raw/
│ └── traces/iter-0001/task-004.jsonl
├── wiki/
│ ├── patterns/
│ │ └── search-wrong-file.md
│ ├── index.md
│ ├── logs.md
│ └── skill-impact.md
└── skills/
└── document-search/
├── SKILL.md
└── PURPOSE.md
raw/ is your replayable evidence. Append new traces; do not “clean” history.
wiki/ is the durable diagnosis: patterns, evolution log, and what each proposal changed.
skills/ is the compact runtime contract. Each skill should point back to the wiki patterns that motivated it.
Directory names and file responsibilities follow the paper’s Three-Layer Knowledge Architecture.
The core can fit in one page of pseudo-code.
best_score = evaluate(empty_skill, validation)
skill = empty_skill
wiki = empty_wiki
for iteration in range(max_iterations):
traces = rollout(agent, train_tasks, skill)
sampled = sample_failures_and_successes(traces)
wiki = maintainer.update(wiki, sampled)
proposal = proposer.create(wiki, traces, skill)
candidate = apply(skill, proposal)
candidate_score = evaluate(candidate, validation)
accepted = candidate_score > best_score
skill = candidate if accepted else skill
best_score = max(best_score, candidate_score)
wiki = log_decision(wiki, proposal, candidate_score, accepted)Adapted from Algorithm 1. In the paper, the proposer can read selected traces on demand and the wiki is retained whether the skill update is accepted or rejected.
Before you call it self-improving, check the loop.
0 of 7 checked
Checklist synthesises the paper’s architecture, ablation, transfer findings and limitations into a practical pre-flight check.
Don’t make the skill the only place the agent remembers.
Let the wiki accumulate the messy learning history. Let the skill stay concise, executable and earned by evaluation.
Paper: Tang et al., WikiSkill, Google Research, arXiv:2608.27454v1. Source paper ↗