Agentic thinking · 8 min read
Field note 011Agentic thinking
The model is not the whole system
Every AI tool you use is a model plus a harness. Learn what an AI agent harness is, why it matters more than the model, and how to map one in ten minutes.
01What is an AI agent harness?
An AI agent harness is everything wrapped around a language model that turns it into a usable tool: the permissions, the data connections, the memory, the approval steps, and the undo paths.
The model writes the sentence. The harness decides which documents the model could read before writing it, whether the result gets sent or merely drafted, and whether a human looked at it in between.
Two products built on the same model can behave completely differently, because the harness is different. That is why the harness, not the model, is the thing to evaluate before you let a tool into your work.
02Why did "self-improving agents" make the news?
In early August, an AI research company called Prime Intellect launched Prime Agent, describing it as a "self-improving" coding agent (announcement).
Strip away the branding, and the interesting part is what the company says it actually built: a harness. Their pitch is that the scaffolding around the model — what it sees, what it can change, how its work is checked, how changes are rolled back — is what makes the agent useful at all.
The "self-improving" claim is a vendor claim, and nobody outside the company has benchmarked it yet. That is fine. The lesson does not depend on the claim being true.
The lesson is that even the frontier labs selling agents are telling you, in their own marketing, that the harness is the product.
03Why the harness matters more than the model
Think about the last time an AI tool at work did something wrong. The wrong file quoted. An email that went out before anyone reviewed it. A confident answer built on data it should never have seen.
None of those are model failures. The model did exactly what models do: it generated plausible output from whatever it was given.
The failures were harness failures. The tool saw a folder it should not have. It acted where it should have drafted. Nobody was placed between the output and the send button.
When a tool works well, people credit the model. When it goes wrong, the harness did it. If you can read a tool's harness, you can judge it in ten minutes. If you can only read its marketing, you are guessing.
04A worked example: your meeting summariser
Take a tool thousands of Singapore professionals switched on this year: an AI meeting summariser.
- The model part
is the same everywhere: audio becomes transcript, transcript becomes summary.
- The harness part
is where the products differ. Does it join every meeting automatically, or only when invited? Does the summary go to all attendees, or to you for review first? Is the recording kept forever, on whose servers, and can you delete it?
- The risk part.
The model affects output quality. The surrounding system determines what data it can access, which actions it can take and how its work is checked. A summariser that auto-sends to all attendees has a very different blast radius than one that drafts to your inbox — even if the summaries are word-for-word identical.
Now apply the same three-step read to any tool on your desk: the drafting assistant, the research chat, the spreadsheet copilot. The model tells you how good the output will be on a good day. The harness tells you what happens on a bad one.
05The Agent Harness Map: six boxes
Here is the ten-minute exercise. Pick one AI tool you actually use — your own workflow, not a client system — and fill in six boxes:
- Sees
- What information can it read? Your prompt only? Your files? Your whole inbox?
- Does
- What can it change in the world? Draft only? Send? Book? Pay?
- Remembers
- What does it keep after the session ends, and where?
- Approves
- Which of its actions pass through a human, and which do not?
- Checks
- How would you know its output was wrong before the wrongness mattered?
- Undoes
- If it did the wrong thing an hour ago, what exactly is the undo?
Write real answers, not vibes. "I think it only reads what I paste" is not an answer; it is a hypothesis you have never tested.
Six questions that profile any AI tool.
What information can it read? Your prompt only? Your files? Your whole inbox?
What can it change in the world? Draft only? Send? Book? Pay?
What does it keep after the session ends, and where?
Which of its actions pass through a human, and which do not?
How would you know its output was wrong before the wrongness mattered?
If it did the wrong thing an hour ago, what exactly is the undo?
06What a blank box tells you
If you cannot fill a box, that is not a failed exercise. That is the finding.
A blank box is where your risk lives, because it means the tool has a capability — or a data path, or a memory — that you are using without understanding. The vendor's documentation or settings page is where you go next, and "I could not find the answer" is itself an answer worth escalating.
In our experience the box that comes back blank most often is Undoes. Almost nobody can say, precisely, how to reverse an AI action after it has left the building. If that is your blank box, you have just learned the most important thing about your tool.
07Is this only for engineers?
No — and that is the point.
The harness questions are not technical questions. "Who approves this before it goes out?" is a management question. "What does it remember about our clients?" is a compliance question. "What is the undo?" is an operations question.
Engineers build harnesses. But the people who decide whether a tool belongs in a workflow — team leads, operations managers, compliance officers, business owners — are the ones who need to read them. The vocabulary is six plain words, and the exercise takes ten minutes.
Judge AI tools the way an engineer judges them: by the harness, not the model. "Which model is it?" tells you almost nothing about whether a tool belongs in your workflow. "What can it see, do, and undo?" tells you everything.
08Frequently asked questions
A model generates output when asked: text in, text out. An agent is a model placed inside a harness that lets it take actions — reading files, calling tools, sending messages — often across multiple steps without a human prompting each one. The model affects output quality. The surrounding system determines what data it can access, which actions it can take and how its work is checked.
It is everything around the model: what the AI is allowed to see, what it is allowed to do, what it remembers, who approves its actions, how its work is checked, and how its actions are undone. If the model is the engine, the harness is the rest of the car — steering, brakes, seatbelts and all.
Because most workplace failures are permission and process failures, not intelligence failures. The tool read data it should not have, acted where it should have drafted, or skipped the human review that mattered. Those are all harness properties, which is why two tools on the same model can have completely different safety records.
Run the six-box Agent Harness Map on it: Sees, Does, Remembers, Approves, Checks, Undoes. Fill every box with a verified answer from the vendor's documentation or settings, not an assumption. Any box you cannot fill is a question for the vendor before rollout, not after an incident.
Treat "self-improving" as an engineering claim that requires evidence, not as a feature. The practical questions are the same harness questions as always, with one addition: what exactly is the agent allowed to change about itself, who reviews those changes, and what is the rollback? A self-improving agent with no rollback story is a marketing page, not a tool.
It applies to everything. A plain chat assistant has a minimal harness — it mostly just sees what you paste — but the moment you connect it to your files, your email, or your calendar, you have built a harness, and the six questions apply in full.
