A lot of AI discussion is about which model is best. I care about that question too, but the longer I work with AI coding agents on real projects, the more convinced I am that it is only half of the question.
I have seen the same underlying model produce materially different results depending on the coding harness wrapped around it. Same model, same kind of task, noticeably different work. That pushed me toward a distinction I now use constantly:
Model quality and agent quality are not the same thing.
What a harness actually is
When you use a coding agent such as Claude Code or OpenCode, the model is not looking at your repository directly. The harness shapes almost everything about the environment the model works in:
- what the model sees, and in what order;
- which tools it has, and how those tools are described;
- what comes back when a tool call fails;
- when, or whether, it is pushed to verify its work;
- what it remembers between steps;
- how work is delegated to sub-agents, and what context reaches them;
- how permissions are communicated and enforced.
The model is the reasoning engine. The harness is the workshop, the instructions, the measuring tools, and the inspection step. A capable machinist in a shop with no measuring tools will still turn out parts that do not fit.
Where the differences come from
I compare harnesses on real projects rather than isolated benchmark questions, because a real project has everything benchmarks tend to strip out: an existing architecture, documentation, many files, tests, browser behavior, incomplete state, and ambiguous feedback. The differences I see trace back to a few places.
Context construction
Which files get read, how the repository is indexed, which project instructions survive into the prompt, and what gets dropped when the context fills up. Two harnesses can hand the same model very different pictures of the same repository.
Tools and tool feedback
A tool is only as useful as its description and its error messages. If an edit fails without a clear error, the model keeps working from a false belief. Good tool feedback is how a model finds out that something did not happen.
Verification loops
This is the one I watch most closely. A harness that runs tests, reads the console, or looks at the rendered page pushes the model into an observe, change, test loop. A harness that does not lets the model decide it is finished because it wrote some files.
Sub-agents and memory
Delegation helps when the sub-agent gets the context it needs and its failures come back to the parent in a usable form. It hurts when the sub-agent works from a thin summary and the parent accepts its report without checking.
The test bed: a local model and an arcade game
To study this with fewer variables, I run experiments on my own hardware. The workflow I have been validating splits the job between two models. A frontier cloud model, Claude Opus, does the research and writes a detailed specification. A locally hosted coding model, qwen3-coder running on an NVIDIA DGX Spark, does the implementation. The specification is the contract between them.
The test case is deliberately demanding: an arcade-accurate Galaga product requirements document, built as a browser game with HTML5 Canvas, vanilla JavaScript, Web Audio, sprite systems, a fixed-timestep game loop, and input handling.
The goal is not really the game. A game is a good test bed because its feedback is immediate and hard to argue with. The model can say the implementation is complete, and the browser can show a black screen.
What I am actually studying is how the local model fails:
- Does it mistake file creation for completion?
- Does it inspect runtime behavior at all?
- Does it notice a black screen?
- Does it verify that the JavaScript actually executes?
- Does it trust its own assumptions too easily?
- Can a better harness force it into a tighter observe, think, change, test loop?
- How much intervention does it need before it starts behaving like a stronger coding agent?
The last two questions are the interesting ones. If structure, feedback, and verification can close a meaningful part of the gap between a smaller local model and a frontier model, that changes both the economics and the privacy options for a lot of real work. That is the open question I am testing, not a conclusion I am claiming.
Documentation is part of the agent’s environment
One side effect of this work is that I take project documentation more seriously than ever. Documentation used to be written mainly for the next person. Now it is also part of the runtime context for AI systems. Clear architecture notes, written conventions, and a real specification give an agent something to check itself against. A vague request gives it room to decide that whatever it built is what you meant.
That is why the two-model workflow puts so much weight on the specification. The frontier model is good at research and at writing down precisely what “done” means. My working bet is that a smaller model does its best work when that definition is explicit and checkable.
What business teams should take from this
If your team is adopting AI coding tools, or AI agents for any operational work, a few practical points follow.
- Don’t choose on the model name alone. Evaluate the whole agent on your own work: your repository, your conventions, your definition of done.
- Invest in the environment. Project instructions, documentation, tests, and a way for the agent to see the running result can matter as much as a model upgrade.
- Require independent verification. The agent’s summary is not evidence that the work is correct. Tests, a browser, and human review are.
- Treat permissions as part of the design. Decide what the agent may change, how it shows what it changed, and what stays under human control.
- Keep local models on the table. How far good scaffolding can carry a model running on hardware you control is exactly what I am testing, and it matters when your data cannot leave the building.
The shift I would encourage is moving from “we gave an AI access to our code” toward “we designed an environment where an AI agent can work, verify, document, and fail safely.” That design work is most of what I do in AI agents and automation and in private, self-hosted AI.
I expect this to stay one of my main research interests for a while. The models will keep improving. A lot of the remaining difference lives in the machinery around them, and that is the part you actually control.