TL;DR
Harness² (google-research/harness2) is a research framework from Google Cloud AI Research that improves an AI agent without touching the model. It edits the harness instead: the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins that wrap a model and turn it into an agent. For each new task it proposes harness edits, runs them on that same task, compares what changed, and composes a better harness. After each task it saves the adopted harness and the lessons behind it, so the next task starts from experience.
Facts as of October 26, 2026:
- 19,951 GitHub stars, 1,958 forks. The repository was created on October 6, 2026.
- 28 open issues (issues only, not pull requests).
- License: Apache-2.0. Benchmarks and code under
third_party/keep their upstream licenses. - No tagged releases. The package in
pyproject.tomlis version 0.1.0; the last commit landed October 14, 2026. - Paper: arXiv:2610.14224, submitted October 9, 2026.
Not run by us: our hands-on lab (a throwaway GitHub Actions container) could not start today because GitHub refused the job over a billing limit on the account. Everything below comes from the README, the setup and experiment guides, the source code and the paper. We did not install Harness² or run any of its commands.
Our verdict: Harness² is the most concrete public answer so far to the question “can an agent improve its own scaffolding on tasks nobody wrote tests for?” The method is clear, the code is tidy and honest about its sharp edges, and the paper’s gains are large on weaker executors. It is a research release, not a product: the provided configuration assumes Vertex AI, every gain is paid for with extra executions, and the benchmark numbers come from the authors alone. Read it to steal ideas for your own agent setup; run it if you study agent harnesses for a living.
What “harnessing” means here
The paper’s framing is Agent = Model + Harness. Keep the model weights frozen and everything else is fair game. Harness² exposes eight editable components, each mounted as a file in the agent’s runtime:
| Group | Component | What it controls |
|---|---|---|
| Context | System prompt, guardrails, memory, skills | What the model reads before and during work |
| Tool interface | Tool guidance, scripts | How existing tools are used; new deterministic helpers |
| Orchestration | Subagents | Delegated workers with their own context window |
| Middleware | Plugin | Code in the tool loop that enforces behaviour instead of asking for it |
If you use Claude Code, Codex or OpenCode, the list will look familiar: roughly an AGENTS.md, a skills folder, hook scripts and subagent definitions. Harness² treats all of that as something a second model can rewrite.
How it works: two levels of recursion
Task-level recursion improves the harness for the task in front of it, using a step the authors call Propose–Probe–Compose:
- Propose. Run the current harness once as a reference. Then an editor model writes two candidate harnesses: an invariant one aimed at general agent weaknesses (planning, tool use, context management, delegation) and a domain one aimed at the knowledge and procedures of this task’s field.
- Probe. Run each candidate on the same task under the same model and environment. Compare trajectories and deliverables with the reference. The paper is explicit that these contrasts are evidence about behaviour, not proof of correctness — there is no ground-truth grader at this point.
- Compose. A composer reads the reference, the candidates, their diffs and the execution evidence, then merges supported changes, adopts one candidate, or keeps the current harness. Each decision must point to an observation. The composed harness runs once more to produce the final deliverable.
You can scale this two ways with a budget k. Parallel recursion probes k invariant and k domain candidates and produces k compositions. Sequential recursion chains k Propose–Probe–Compose steps, carrying the refined harness forward.
Harnessing-level recursion is the part that makes it more than test-time search. After each task an “experience reflector” stores the adopted harness plus an EXPERIENCE.md recording when the change applies, what was learned and what evidence supports it. Libraries are kept per domain; tasks in a domain run in order and only read earlier entries. Future proposals can reuse or adapt those components — but the editor prompt tells it to re-check task-specific facts against the current task.
One detail from the published prompts is worth copying into your own agent setup: the editor is told that “verified facts about this workspace are the most valuable thing you can encode” — the exact join between two files, which entries have no counterpart, a field’s units — because those are what earlier runs paid to discover.
Installing it
The README asks for Linux, Python 3.12 and uv. The package declares Python 3.12–3.13 and POSIX only, so macOS is not a listed target and Windows is out.
git clone https://github.com/google-research/harness2
cd harness2
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2harness2-demo is the one thing you can run without credentials. Reading harness2/demo.py, it builds a tiny “reconcile a ledger” task, a base harness whose scripts/summarize.py only sums positive amounts (the bug), and a deterministic proposer instead of a model. It shows the directory layout and the decision ledger, but nothing about how well a real model edits a harness.
For model-backed runs you also need Node.js 20+, the Google Cloud CLI, and per-benchmark extras: Pandoc and Poppler for LAB, Docker for WorkBuddy-Bench. Configuration lives in .env:
| Setting | Meaning |
|---|---|
HARNESS2_SOLVER_MODEL | Executor model in OpenCode’s provider/model format |
HARNESS2_IMPROVER_MODEL | Model that edits the harness; defaults to the executor |
HARNESS2_JUDGE_MODEL | Judge model ID (bare, the adapter adds the Vertex prefix) |
HARNESS2_RUNS_DIR | Output root, default runs/ |
Then gcloud auth application-default login. The setup guide says the provided configuration uses Vertex Gemini executors and judges and supports Vertex Claude or Gemini as the improver. JobBench can point its judge at another OpenAI-compatible endpoint, and the Codex executor can talk to any Responses API endpoint, but WorkBuddy setup refuses to run unless the solver is a google-vertex/gemini... model. In practice: plan on a Google Cloud project.
Running a benchmark
Each benchmark installs separately and follows the same five commands. The README’s smallest real example is one LAB legal task:
harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
--only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallelUseful switches from the experiment guide:
--dry-runonharness2previews task selection with no model calls;harness2-setup <bench> --dry-runpreviews setup.--max-taskscaps tasks per domain, not per run — an easy way to launch more than you meant. The full LAB stream is 1,143 tasks.--substrate cxswaps the default OpenCode executor for Codex. For Gemini on Vertex,harness2-bridgestarts a local LiteLLM server that speaks the Responses API.--seed Nshuffles the task stream reproducibly but does not control model sampling.- Re-running the same command resumes. Judging and reporting must use the same benchmark, executor, subset,
k, mode and seed as the run.
harness2-judge --bases also scores the reference executions, which is what you want if you are measuring the gain over your own base harness rather than trusting the paper’s.
What the paper reports
Every number in this section is the authors’ own, from the paper. Executors were Gemini 3.5 Flash and GLM-5 under off-the-shelf OpenCode and Codex harnesses; the default harness editor was Claude Opus 4.8 running under OpenCode. Main tables use one improvement step (k=1); ”+ Recursion” means parallel recursion at k=5. The library starts empty for each benchmark stream.
Headline gains from a single step: up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench and 11.8 percentage points on LAB’s partial pass rate. A few rows from the WorkBuddy-Bench table (average of the code, web and office domains, OpenCode executor):
| Executor | Base harness | Meta-Harness† | Harness² (k=1) |
|---|---|---|---|
| Gemini 3.5 Flash | 72.4 | 76.4 | 77.5 |
| GLM-5 | 64.2 | 69.7 | 75.7 |
† Meta-Harness searches harness code offline against a training split; Harness² and the other baselines never see that split.
Other findings the authors report:
- Weaker models gain most. GLM-5 jumps 11.5 points on WorkBuddy; Gemini 3.5 Flash gains 5.1. With the same model as both executor and editor at
k=2, all five models tested — GLM-5 up to Claude Opus 5 — improved on every benchmark; Opus 5 gained 3.3, 5.5 and 3.0 points, GLM-5 gained 13.7 on LAB. - More budget, more gain. For Gemini 3.5 Flash under OpenCode, raising
kfrom 1 to 5 lifts the gain over base from 4.5 to 7.5 points on JobBench and from 5.1 to 8.4 on WorkBuddy. The paper places both Harness² settings on its compute–performance Pareto frontier and says Meta-Harness used more compute for lower scores. - Every part matters. Removing the library, the invariant/domain split or the composition step each cost between 1.6 and 3.7 points in the ablation.
- Lessons transfer a little. A library built on JobBench improved one-step refinement on WorkBuddy by 0.6, 0.5 and 2.9 points across code, web and office.
LAB results are on a 100-task held-out set judged by Claude Sonnet 4.6. That is a model grading a model; the authors report standard deviations over three runs, which helps, but some deltas in the table sit within one standard deviation.
What the numbers do not tell you
- Cost in dollars. The paper counts tokens on its Pareto plot but does not translate a run into a bill. A
k=5parallel run means five reference executions, ten probed candidates, five compositions and more editor sessions — per task. On long legal or office tasks that adds up quickly. - Latency. The paper notes parallel sessions cut wall time but not compute. An agent that runs a task six or more times before answering is not an interactive assistant.
- Your workload. The three benchmarks are professional, document-heavy tasks with rubrics. Nothing here measures software engineering on a large private codebase, which is where most people’s harnesses live.
- Independent replication. As of today the only results are the authors’.
A warning the docs put in writing
The experiment guide is unusually candid: all three benchmarks are public, JobBench publishes each task’s RUBRICS.json on Hugging Face, and “in a validation run of this release, a Codex rollout on JobBench downloaded its task’s rubric without being instructed to.” Executor network access is unrestricted and Codex runs with its sandbox disabled. If you want numbers you can believe, block huggingface.co, hf.co, *.hf.co, github.com and raw.githubusercontent.com for the executor at the network level after setup, and report the restrictions with your results.
What people are saying
The release went wide fast: about 20,000 stars in under three weeks, coverage on MarkTechPost, a Product Hunt listing and a Hacker News thread. The skeptical questions on HN are the right ones: whether this differs in kind from what agent projects such as Hermes already do with self-written skills and memory, and why a company that keeps its own production harnesses closed would open the method for improving them. The fair answer to the first is that Harness² adds the controlled part — candidates probed against a reference on the same task, with a recorded reason for each adopted change — which most self-editing agents skip.
If you have been following the agent-harness wave, it sits next to tools like Ruflo — but those are harnesses you run. Harness² is a method for improving whichever harness you already run.
Who it is for
- Agent researchers comparing harness-optimization methods. The code separates method, executor and benchmark cleanly;
demo.pyis a complete minimal adapter. - Teams with a Google Cloud project and a rubric-graded task stream where a few points of quality are worth several times the inference spend.
- Anyone maintaining an
AGENTS.mdby hand. The Propose–Probe–Compose discipline — change one thing, run it on the same task, keep it only if the trajectory shows why — beats editing prompts on instinct.
It is not for people who want a coding agent, a hosted product or a quick win on a laptop.
Honest limitations
- Vertex-first. The shipped configuration expects Vertex AI and
gcloudcredentials; other providers need extra wiring, and WorkBuddy insists on Gemini. - Linux only. POSIX, Python 3.12–3.13. Docker for WorkBuddy.
- No releases, no support. Version 0.1.0, no tags, and the README states it is not an officially supported Google product. The last commit was twelve days ago.
- Expensive by design. The paper’s own limitations section says more recursion means more inference cost, and that execution-based feedback stays “sensitive to execution variability and shared model biases” and does not guarantee correctness or transfer.
- Benchmark licensing. WorkBuddy-Bench ships under a custom Tencent license with a territorial limitation.
- The library grows without pruning. The authors list consolidating old entries as future work.
FAQ
What is Harness²?
An open-source framework from Google Cloud AI Research for “recursive agent harnessing”: it improves an LLM agent by editing its prompts, memory, skills, tools, subagents and plugins for each task, and saves what worked for later tasks. The model’s weights never change.
Does Harness² need ground-truth answers or tests?
No. Its Propose–Probe–Compose step compares the behaviour of candidate harnesses against a reference run on the same task. Graders are only used afterwards, by harness2-judge, to score results.
Can I try Harness² without an API key?
Only the demo. harness2-demo --output runs/demo --k 2 runs both recursion modes with a deterministic, non-model editor on a toy ledger task. Benchmark runs need model credentials (Vertex AI in the provided configuration) and incur API costs.
Which agents can it improve?
The release supports OpenCode (default) and Codex (--substrate cx) as executors. The harness components map closely to what Claude Code and similar agents use, but there is no adapter for them in this release.
How does it compare with Meta-Harness?
Meta-Harness searches for one best harness offline on a training split, then deploys it. Harness² adapts per task at test time and builds a library across tasks. In the authors’ tables Harness² at k=1 beats Meta-Harness on WorkBuddy for both executors shown above, and the paper says Meta-Harness spent more compute.
Verdict
Harness² turns “tune your agent’s scaffolding” from craft into a procedure with evidence attached, and it is open about where that procedure can fool itself — down to documenting an agent stealing its own rubric. The gains it reports are real enough to take seriously, especially for cheaper executors, and they are bought with compute. Start by reading demo.py and the editor prompts in the paper’s appendix; reach for the full pipeline only if you have a Vertex project, a rubric-graded task stream and a budget to burn on repeated runs.