Text-Space Optimization
Treat a natural-language artifact — a skill document, a prompt, a playbook — as the trainable state of a frozen model, and optimize it with the machinery of weight-space training. The model's parameters never move. What moves is markdown. SkillOpt (Microsoft Research) is the worked instance: a separate optimizer model reads scored rollouts of a target agent and emits bounded add / delete / replace edits to a single skill document, under an explicit budget the repo calls a textual learning rate, organized into epochs with a (mini-)batch size of sampled tasks. The vocabulary is not decoration — it names the specific disciplines borrowed (bounded step size, held-out validation, epoch-wise slow/meta updates, a rejected-edit buffer that functions as negative memory) and the specific one that cannot be borrowed: there is no analytic gradient, so a language model computes the update direction from trajectories instead.
The property that makes this a distinct kind of artifact rather than
a training curiosity: the optimized output is a compact document
that runs against the unchanged target model with zero extra
inference-time model calls. All the search cost is paid
offline; the deployed thing is best_skill.md. That is the
inverse of most agentic quality moves (LLM-as-Judge, evaluator-optimizer in Agentic Workflow Patterns,
reflection loops), which buy quality by spending inference-time
calls on every request. Where those pay per-request forever, text-space
optimization pays once and ships a file. The cost is moved, not removed
— the search still consumes rollouts, judges, and tokens — but it moves
to a place where it can be budgeted, gated, and amortized.
This is the mechanized end of a spectrum the graph already holds. Self-Improving System and Skill-Driven Loop Development improve skills through a human-driven loop; the skill gets better because a person notices and edits. Text-space optimization asks what happens when the noticing and the editing are themselves an optimizer, with Validation-Gated Update standing in for the human's judgment.
Claims
- The skill document is the trainable state of a frozen agent; optimizing it is a legitimate optimization problem, not just prompt-fiddling. principle — durable and the source's thesis. Once you name the artifact as state and the edits as steps, the whole apparatus of optimization (step size, validation, overfitting, transfer) applies to markdown. This is the reframe that the rest of the page depends on.
- The failure mode of ad-hoc skill evolution is that it does not reliably improve over its starting point. observation — the repo's stated motivation: hand-crafted, one-shot-generated, and loosely self-revised skills are all "none of which reliably improves over its starting point under feedback." The word doing the work is reliably — the complaint is variance, not incapacity.
- Bound the size of each edit (a "textual learning rate") and keep a buffer of rejected edits. best practice — context: an automated optimizer editing a document across many epochs, where an unbounded rewrite can destroy a working skill in one step and where a rejected edit is information, not garbage. The analogy to a numeric learning rate is asserted by the source, not derived; whether a "textual" step size behaves like a numeric one under composition is exactly the kind of thing a paper's ablations, not a README, would settle.
- Optimizing offline into a static artifact keeps inference-time cost at zero — unlike reflection or judge loops, which pay per request. principle — durable and the structural advantage of the approach. A file that has been optimized costs no more to load than a file that has not.
- A separate optimizer model — not the target agent — proposes the edits. observation — the role split matters: the agent being improved does not grade or rewrite itself, which is the self-assessment problem LLM-as-Judge also routes around by separating actor from judge.
Check-worthy source claims (attributed, not adjudicated — a later grounding pass can verify):
- SkillOpt reports best or tied-best on all 52 evaluated (model, benchmark, harness) cells, across six benchmarks, seven target models, and three execution harnesses (direct chat, Codex CLI, Claude Code CLI). observation — the repo's headline result.
- On GPT-5.5 it reports lifting average no-skill accuracy by +23.5 points in direct chat, +24.8 in the Codex agentic loop, and +19.1 inside Claude Code. observation — named model, named numbers; verify against the paper (arXiv 2605.23904), not this page.
- The deployed
best_skill.mdis "typically 300–2,000 tokens." observation
Related
- Validation-Gated Update — the accept/reject rule that makes the optimization monotone; the half of this concept that supplies the judgment a human otherwise supplies.
- Skill Artifact Transfer — what the optimized artifact turns out to be worth beyond the run it was trained on.
- Offline Consolidation Cycle — the same optimizer applied to a live agent's own past sessions rather than to a benchmark's task set.
- Self-Improving System — the human-driven form of the same ambition; the tension between them (is a validation gate a substitute for the operator's judgment?) is the live question.
- Minimal Skill Surface — the authoring discipline arrives at a small skill by taste; this arrives at one by optimization. Both land on "the deployed surface is small."
- Skill Pruning — an optimizer that emits delete edits gated on held-out score is the deletion test run mechanically, at scale.
- Agentic Workflow Patterns — the evaluator-optimizer pattern is this loop run at inference time on a single output; text-space optimization runs it at training time on the instructions.
- LLM-as-Judge — how a text-space rollout gets scored when the benchmark has no programmatic answer key.
- Eval-Driven Development — "don't trust vibes; measure" is the precondition: text-space optimization is only possible where a score exists.
- Execution Commoditization — if the method, not the model, is the differentiator, then a way to train the method is a way to manufacture differentiation.
- Distillate: microsoft/SkillOpt — training skills like weights, without touching weights