# Same model, different harness

2026-10-23

Why the app a coding agent runs inside changes the work you get back, even when the model stays the same.

Most of the "which AI coding tool should I use?" talk is really about models. Opus or GPT, this week's leaderboard, who shipped what. I get it. I used to think that way too. But the longer I work with agents, the more I think the model is only half of it.

The other half is the harness. That's the app the agent actually runs inside: the loop that keeps calling the model, the tools it's handed, what goes into its context and what gets remembered, what it's allowed to do, and the screen where you watch it work. [Empryo](https://empryo.com)'s own blog puts it about as plainly as anyone: "An AI agent harness is the software around the model that runs that process. It gives the model context, offers it tools, runs the actions it's allowed to take, records the results, and decides when to call the model again."

Pick the wrong one and a great model will still do dumb things in your repo. I've watched it happen.

## The worktree that went nowhere

For a long time [Claude Code](https://code.claude.com/docs/en/overview) was the de facto default. If you were using Anthropic models, the assumption was that Anthropic's own tool would get the most out of them. Made sense to me.

Then I started running into stuff like this. I'd give Opus a task in Claude Code, and it would spin up a brand-new Git worktree on its own. Then it wouldn't follow through on the work. It would commit, then try to push things around according to its own idea of a Git flow. My setup assumes `dev` is the branch you're always working from. Claude Code skipped that entirely and forked off `master`. Every time.

That wasn't the model being dumb. It was the harness. Claude Code's system prompt has its own opinions about how a repo should work, and those opinions didn't match my local setup. Same model, wrong room.

Empryo and [oh-my-pi (omp)](https://github.com/can1357/oh-my-pi) don't do that to me. The bigger thing I've noticed is that I get consistent results out of both of them no matter which model I pick. With Claude Code I'm tied to Anthropic's Claude models, and even then the behavior isn't always the same from one codebase to the next. Empryo's opinionated setup gives me the most predictable output of anything I've used. omp is a very close second.

## It's not just me

The public numbers back this up, at least roughly. When [Terminal-Bench](https://www.tbench.ai/news/terminal-bench-2-1)'s maintainers re-ran top agents on the fixed 2.1 version of their test (May 2026), GPT-5.4 solved 77.3% of tasks inside OpenAI's [Codex CLI](https://github.com/openai/codex) but only 54.8% inside the neutral Terminus 2 harness. Same model, a 22.5-point gap.

That's not new, either. On SWE-bench Lite, the same model (GPT-4 Turbo) fixed 18% of issues with [SWE-agent](https://arxiv.org/abs/2405.15793)'s purpose-built interface, versus 11% when it only had a plain Linux shell (Yang et al., NeurIPS 2024, Table 1). Anthropic said it outright back in 2024 when it [wrote up its own SWE-bench work](https://www.anthropic.com/engineering/swe-bench-sonnet): "The performance of an agent on SWE-bench can vary significantly based on this scaffolding, even when using the same underlying AI model."

I don't read benchmarks as gospel. But when my own experience and the people running the tests point the same way, I pay attention.

## What I actually do all day in Empryo

My work sessions now are mostly just Empryo, with tabs named for the agents in them.

Chief is the project manager. It plans and hands work out. It doesn't always use Empryo's built-in [orchestrator](https://empryo.com/docs/tools/side-agents#orchestrator), which is fine; sometimes I just want it to tell the other tabs what to do. Grunt is the builder. As a project grows I add more tabs: SEO, a reviewer, testing, scouts. Each tab keeps its own model and mode. The agents can message each other and hand work off with Empryo's [link tool](https://empryo.com/docs/agents/child-tabs#the-link-actions), and a tab can even open a child tab that stays attached to it.

When I'm still figuring out what I even want, I open [Duck](https://empryo.com/docs/tools/side-agents#duck). It's one of Empryo's side agents, a rubber duck to talk a problem through with. A duck tab can't see my repo at all, which is kind of the point. It's great for brainstorming before anything touches code.

Then there are [morphs](https://empryo.com/morphs). A morph is a small add-on, one JSON file, that can add a panel to the app or give the agent rules to follow. You can share them, too. I've got three: an agents dashboard, a [GitButler](https://gitbutler.com) view, and a [GH Dash](https://github.com/dlvhdr/gh-dash) view. They let me see everything in one place, hit a few quick actions, and pop the terminal UI open in [Kitty](https://sw.kovidgoyal.net/kitty/) when I want it.

The part I'd miss most is the code map. Empryo keeps a ranked map of your files and their dependencies, which it calls the Genome, and puts a slice of it into every prompt. In practice that means the agent already knows where things live and which exported functions and classes each file offers, so it doesn't reinvent the wheel every single time. On a big codebase that's the difference between sustainable and exhausting.

omp has its own opinions on this. For omp and anything else I run, I add [CodeGraph](https://github.com/colbymchenry/codegraph), which parses the code with tree-sitter into a local graph of functions, classes and calls that the agent can query. Empryo has that kind of thing built in, so CodeGraph is really just for the other harnesses.

A quick word on the room the harness runs in, because people mix these up. Kitty, [Ghostty](https://ghostty.org), [Tern](https://stencil.so/tern) and [Herdr](https://herdr.dev) aren't harnesses. They're where the harness lives. I use both Tern and Kitty. Kitty's my steady pick for stability, and Tern made omp noticeably better for me. Older versions of Tern were heavy on RAM with their background processes, but the latest release has caught up to Kitty on resource use.

## Where I part ways with Miessler

Daniel Miessler has been writing about harnesses all year, and he's mostly right. His point in "[Good and Bad Harness Engineering](https://danielmiessler.com/blog/good-and-bad-harness-engineering)" is that step-by-step instructions age badly: "the smarter AI gets the more antiquated your instructions will become." What holds up is context about you: "no matter how smart an AI becomes it will still be better at getting you great results if it understands who you are and what you like."

I agree, to a point. If you treat the agent like a junior developer and tell it clearly what you want, it'll do the work. I've seen that over and over.

But I still want guardrails. Not so much around system tools; those are the harness's job. Empryo's blog nails that part too: "A prompt asking the model not to run a command is a weaker boundary than a harness that refuses to run it." I mean guardrails around the codebase itself.

For me that's [GAP](https://github.com/edheltzel/GAP), my global agent policies. GAP tells every agent what I expect: code quality, how the code is structured, how work gets produced and checked into a repo. Docs, type checks, linting, and the overall architecture down to the infrastructure. Whatever harness I'm in, GAP comes with me.

People sometimes tell me a sandbox covers this. It doesn't. A sandbox protects your production environment. It doesn't make the code right. Code that passes in a sandbox and falls over in production is just "it works on my machine" with extra steps. I've lived through enough of that already.

## Taking my tools with me

I'm also building GAP to keep the other harness tools I've made in sync across every harness I use. Recall comes first, and Echo follows once it has its first release. That's [the plan](https://github.com/edheltzel/GAP/blob/master/docs/plans/2026-10-20-gap-env-provisioner-plan.md), anyway. The idea behind GAP is simple. I write the rules once in one `AGENTS.md`, and `gap sync` copies them into the file each agent already reads, whether that's Claude Code, Pi, omp, Codex or Cursor. It installs the skills I've picked, too, and tells me when one has drifted.

[Echo](https://github.com/edheltzel/Echo) gives my agents a voice. Coding agents finish in silence, and you only find out they're waiting when you look back at the terminal. Echo is a small local daemon on my Mac that speaks the completion line when a turn ends, and on hosts that support it, it also speaks up when an agent can't continue without me. Claude Code, Pi, omp, Codex and Empryo can all talk through it.

[Recall](https://github.com/edheltzel/Recall) is memory. Every coding agent forgets when a session ends. Recall captures past sessions into one local SQLite database, makes them searchable, and on harnesses that support it, puts the best of it in front of the agent when a new session starts. So my agent doesn't start from scratch every time.

## I'm the bottleneck now

Here's the honest part. With the right harness, a code map, CodeGraph and GAP all working together, the agents aren't my problem anymore. I am.

I'm the human in the loop. I still review every PR and decide how it gets merged. I want to get to where I can trust auto-PR and auto-merge, I really do. But I'm not there, and I don't think the tooling is either. Before I hand that over, I need better observability, a better eval process, and tooling that lets me check objectively that the agents didn't reinvent the wheel, didn't add regressions, and didn't just lie about fixing a bug.

That's the next problem for me, and I think it's mostly a harness problem too. Picking the right one got me consistent work. Picking one I can actually trust to merge is the part I'm still working on.

## Sources

- Empryo, "What is an AI agent harness? Follow one code change" (Sep 15, 2026): https://empryo.com/blog/what-is-an-ai-agent-harness
- Empryo docs, "How it works": https://empryo.com/docs/concepts/architecture
- Empryo docs, "Tabs": https://empryo.com/docs/agents/cross-tab
- Empryo docs, "Modes & the prompt": https://empryo.com/docs/context/prompt-system
- Empryo docs, "Duck, advisor & council" (side agents): https://empryo.com/docs/tools/side-agents#duck
- Empryo docs, "Child tabs" (the link actions): https://empryo.com/docs/agents/child-tabs#the-link-actions
- Empryo, Morphs: https://empryo.com/morphs
- Empryo: https://empryo.com
- Terminal-Bench, "Terminal-Bench 2.1" (May 6, 2026): https://www.tbench.ai/news/terminal-bench-2-1
- Yang et al., "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering" (NeurIPS 2024): https://arxiv.org/abs/2405.15793
- Anthropic, "Raising the bar on SWE-bench Verified with Claude 3.5 Sonnet" (Oct 30, 2024): https://www.anthropic.com/engineering/swe-bench-sonnet
- Daniel Miessler, "Good and Bad Harness Engineering" (Apr 14, 2026): https://danielmiessler.com/blog/good-and-bad-harness-engineering
- oh-my-pi (omp): https://github.com/can1357/oh-my-pi
- Claude Code overview: https://code.claude.com/docs/en/overview
- OpenAI Codex CLI: https://github.com/openai/codex
- CodeGraph: https://github.com/colbymchenry/codegraph
- GitButler: https://gitbutler.com
- GH Dash: https://github.com/dlvhdr/gh-dash
- Kitty: https://sw.kovidgoyal.net/kitty/
- Ghostty: https://ghostty.org
- Tern (Stencil): https://stencil.so/tern
- Herdr: https://herdr.dev
- GAP (Global Agent Policies): https://github.com/edheltzel/GAP
- GAP tool-sync plan: https://github.com/edheltzel/GAP/blob/master/docs/plans/2026-10-20-gap-env-provisioner-plan.md
- Echo: https://github.com/edheltzel/Echo
- Recall: https://github.com/edheltzel/Recall