AI Harness Engineering: Build a Personal AI Assistant You Control (2027 Guide)

Image: Kody Dahl on Unsplash.

AI harness engineering is the design of the software around an AI model: what it sees, which tools it may use, what it remembers, what it must ask before acting, when it stops, and how you check its work. A personal AI harness applies those controls to your own notes, accounts and routines, so the assistant can be inspected, constrained, maintained and moved.

The model is the part everyone compares.

The harness is the part you live with.

This guide builds one small assistant and grows it carefully:

a sourced answer from your own notes,
a meeting brief that changes nothing on your calendar,
a reply that is drafted but cannot send itself,
a receipt for every run,
an exit plan for the day you change providers.

This is the 2027 edition, written in October 2026. Every number has its source and date beside it. Where something still has to be measured, the guide says so instead of guessing.

Three ways to read it. To understand the discipline, read sections 1 to 3. To build, read sections 4 to 6 and then the read-only quickstart. To audit an assistant you already use, jump to sections 7 to 11.

1. What is AI harness engineering?

An AI agent is a model running in a loop. It reads the situation, decides, calls a tool, reads the result and decides again. How AI agents work walks through that loop step by step.

The harness is everything in the loop that is not the model. Databricks describes an agent harness as the software infrastructure wrapped around a large language model that connects it to tools, memory, execution environments and guardrails (Databricks). LangChain reaches the same place from the other direction: start from the behaviour you want, then add the harness parts that make the model capable of it (LangChain).

The terms blur in practice, so here they are against one task: preparing tomorrow morning's meeting brief.

Term	What it is	In the meeting-brief example
Model	The system that reads text and proposes the next step	Decides which notes matter and writes the brief
Agent	A model running in a loop with tools	Reads the calendar, searches the notes, writes the brief
Harness	The runtime around the model	Chooses what the model sees, runs the tools, refuses calendar writes, records a receipt
Framework	A library for building agents and harnesses	Code you might use to build all of the above
Workflow	A fixed sequence of steps	"Every weekday at 8:00, fetch tomorrow's meetings, then summarise them"

These are roles, not rival products. A workflow can call a model. A harness can run a workflow. The harness vs framework vs workflow guide shows how to choose the smallest one that does the job.

Prompt, context and harness engineering

Three disciplines get confused because they all touch the same model call.

Prompt engineering writes the instruction.

Context engineering decides everything the model sees for a call: instructions, retrieved documents, tool results, remembered facts and history. Anthropic frames it as the ongoing work of curating a limited context window rather than writing one good prompt (Anthropic).

Harness engineering owns what happens around the call: which tools exist, which actions are allowed, what is persisted, when the loop stops and how the result is checked.

A prompt asks. A harness decides what is allowed.

Why personal AI needs explicit controls

A personal assistant touches the most sensitive data most people have: mail, calendar, documents, money, health, family. That raises the cost of a confident mistake and of a quiet data leak.

Two recent surveys show the concern, and they measure different things.

• Personal data. In a 2026 survey commissioned by Usercentrics and run by Sapio Research across 11,000 consumers in seven markets, 52% said they trust AI less than humans with their personal data, up from 48% in 2025. The year-on-year comparison excludes Sweden, which joined the study in 2026 (Usercentrics).
• Accuracy. In Stack Overflow's 2025 developer survey, 46% of respondents said they distrust the accuracy of AI tools and 33% said they trust it. Only 3% reported high trust (Stack Overflow).

One is about where your data goes. The other is about whether the output is right. A personal harness has to answer both, with permissions and data boundaries for the first, and sources, receipts and tests for the second.

The useful response is not finding a new vendor to trust.

It is building a system that needs less blind trust.

2. Build, configure or buy?

Owning a harness gives you control. It also gives you the maintenance.

Choice	Fits when	You inherit
Buy a managed assistant	You want results quickly and accept the provider's data and action policies	Little maintenance, limited control, the provider's exit options
Configure an existing agent tool	You want your own tools, prompts and data rules without writing a runtime	Configuration, connector reviews, some upgrades
Build your own harness	You need specific boundaries, portable state or behaviour no product offers	Updates, credentials, backups, monitoring and recovery

There is no shame in the first row. If you cannot spare a few hours a month for patching, credential reviews and restore tests, a well-chosen managed product is probably safer than an unmaintained one you built. The private AI assistant comparison guide shows how to judge one.

Local, cloud or hybrid is several decisions, not one

"Local" usually describes the chat window. The harness has more parts than that, and each can run somewhere different:

the interface,
the model,
the document store and its search index,
the tools and the accounts they reach,
the logs,
the backups.

Two examples show why the distinction matters.

Ollama binds to 127.0.0.1 on port 11434 by default and documents a local-only mode, set with OLLAMA_NO_CLOUD=1 or disable_ollama_cloud in its server config, which turns off its cloud models and web search (Ollama FAQ). That keeps model calls on the machine. It says nothing about a mail connector that sends message text somewhere else.

OpenAI's API documentation says API data is not used to train its models unless you opt in, and separately that abuse-monitoring logs may be retained for up to 30 days, with stated exceptions; zero-retention controls require prior approval (OpenAI). No training is not the same as no retention.

The local vs cloud vs hybrid guide maps each part. For the broader shift, see the rise of local AI.

Set your starting constraints

Before writing code, write five lines:

1. The first task. One useful job, such as "answer questions from my project notes."
2. Allowed data destinations. Which data may leave the machine, and to whom.
3. A spend cap. Per run and per month.
4. Forbidden actions. For a first build: no sending, no deleting, no payments, no calendar writes.
5. Maintenance time. Honest hours per month.

Those five lines will decide more of the architecture than any framework choice.

3. The architecture of a personal AI harness

A personal harness has three paths running through it.

The reasoning path carries a task to the model: input, context assembly, the model call and a structured request for a tool.

The authority path decides whether that request may happen: identity, scoped tools, policy checks, approval and execution outside the prompt.

The continuity path keeps the system honest over time: memory with provenance, checkpoints, receipts, budgets, evaluations and portable state.

Every component should exist because it prevents a failure you can name.

Component	Failure it prevents
Context assembly	Wrong, stale or missing evidence; an overflowing context window
Tool contracts and input validation	Malformed calls and calls that reach outside their scope
Policy checks and approvals	Side effects nobody authorised
Step, time and spend limits	Loops and runaway bills
Memory with provenance	Confident use of facts that are out of date
Checkpoints and idempotency	Lost work, and duplicated actions after a restart
Run receipts	"It said done, but nothing happened"
Evaluations	Silent regressions after a model or prompt change
Export and restore	Lock-in and unrecoverable loss

Birgitta Böckeler's article on harness engineering at martinfowler.com sorts controls into those that steer an agent before it acts and those that check the result after, and separates deterministic checks from model-judged ones (martinfowler.com). That split is useful at home too. A path check on a file tool is deterministic and cheap. "Is this summary faithful?" needs a judge.

For the plain-language version of why this layer decides outcomes, see Model vs. Harness.

4. Build your first working harness: one task, one read-only tool

The first build should be small enough to understand completely:

one model,
one read-only tool,
one folder it may read,
a hard step limit,
a receipt.

The quickstart has the full 50-line example. It uses a scripted stand-in for the model so it runs without an API key, and it was run on Node.js 22 before publication. Asked for the next milestone in a synthetic project note, it returned this receipt:

{
  "question": "What is the next milestone for Project Alpha?",
  "steps": [
    { "step": 1, "tool": "read_note", "args": { "name": "project-alpha.md" }, "ok": true }
  ],
  "stop": "answered",
  "answer": "Next milestone: read-only beta to five testers on 14 November.",
  "sources": ["project-alpha.md"]
}

Three design rules carry through everything that follows.

The tool defends itself. read_note accepts only simple file names inside one folder. In testing it rejected ../secret.md, absolute paths and non-string input, whatever the model asked for.

The loop has a hard stop. Four steps and twenty seconds, enforced by the runtime rather than requested in the prompt.

Every run ends with a reason. answered, timeout or step_limit. A run that cannot say why it stopped cannot be debugged.

5. Add context, memory and your personal knowledge

"Memory" usually means four different things. Keep them apart.

Kind	What it holds	Lifetime
Current context	What the model sees for this call	One call
Retrieved documents	Passages found in your files for this question	One task
Durable facts	Preferences and facts meant to persist	Until corrected or deleted
Task state	Progress, checkpoints and pending approvals	Until the task ends

Mixing them causes most memory bugs. A preference saved as a document gets retrieved at random. A one-off instruction saved as a durable fact never goes away.

Retrieve with provenance

Every retrieved passage should carry where it came from and when. That lets the answer cite it, lets you check it, and lets the system prefer a newer source over an older one.

Context is not a bin to fill. In a 2023 study of the language models of the time, accuracy on multi-document question answering was often highest when the relevant passage sat at the beginning or end of a long input, and dropped when it sat in the middle (Liu et al.). Today's models differ, so treat that as a reason to test evidence placement on your own setup, not as a fixed rule. Anthropic lists compaction, structured note-taking and sub-agent architectures as ways to keep long tasks inside a useful context (Anthropic).

Make memory reversible

You should be able to do four things with any remembered fact:

inspect it and see where it came from,
correct it and watch the next answer change,
export it in a readable format,
delete it, with a clear statement of what deletion does not reach.

That last part matters. Deleting a record from your store does not remove copies in old backups, and it cannot recall anything already sent to a provider. Say so in the interface.

The personal AI memory guide builds this lifecycle, the RAG vs memory vs context guide separates the mechanisms, and AI agent memory explained covers the wider research.

6. Connect tools, apps and MCP without unlimited access

Tools are where an assistant stops talking and starts acting. Grant them in stages:

1. Read. Narrow, scoped, read-only access to one resource.
2. Draft. The assistant prepares an action you can see: a draft reply, a proposed event.
3. Approve. You approve that exact action.
4. Execute. The runtime performs the approved action and records the result.

Most personal workflows deliver their value at step 2.

Write a contract for every tool

A tool contract names its inputs, outputs, error types, the resources it may touch and whether it reads or writes. Validate inputs in the tool, not in the prompt. Return structured errors so the loop can tell "not found" from "not allowed."

What MCP does and does not give you

The Model Context Protocol standardises how an assistant discovers and calls tools. It does not certify that a server, its credentials or its actions are safe.

The MCP documentation distinguishes transports. A remote server uses the protocol's authorization flow. A local server running over STDIO can instead take credentials from its environment or from libraries embedded in the server (MCP authorization). The security guidance calls token passthrough an anti-pattern, meaning a server that accepts tokens not issued to it and forwards them to a downstream API, and it describes confused-deputy attacks against proxy servers (MCP security best practices).

In practice: pin the protocol version, read what the server can reach, give it the narrowest credentials that work, and test that revoking access actually stops it. The MCP for personal AI tutorial walks through one connector, and the best MCP clients lists the hosts.

7. Enforce boundaries and test prompt injection

Everything your assistant reads is data. Some of that data will contain instructions:

a document that says "ignore your user,"
an email that asks for a password reset link to be forwarded,
a web page with hidden text,
a tool result that requests a new action.

None of them carries your authority. The harness has to keep it that way.

Put enforcement outside the prompt

A system prompt that says "never send email" is a request. A runtime with no send tool is a fact. Prefer facts:

• remove tools a task does not need,
• scope credentials so a tool physically cannot reach other resources,
• restrict which network destinations tools may contact,
• require approval for consequential actions in code the model cannot edit.

OWASP's AI Agent Security Cheat Sheet is specific about approvals: before execution, check that the approval belongs to the current actor and the exact tool call, is still valid and has not already been used, and consume it in one atomic step so repeated or concurrent requests cannot reuse it (OWASP). The AI agent permissions guide turns that into a state machine.

Test it with synthetic attacks

Measure two things separately: whether a forbidden action happened, and whether the legitimate task still got done. A system that refuses everything is safe and useless. AgentDojo, a research benchmark for prompt-injection attacks on tool-using agents, was built this way, with 97 realistic tasks and 629 security test cases in its paper version (Debenedetti et al.).

Use fake accounts, fake data and canary strings. Never test against real people or systems you do not own. The prompt-injection lab has the case design.

Passing a test suite shows the tested cases held. It does not prove the assistant is safe against everything.

8. Make long-running work reliable

Short tasks fail loudly. Long tasks fail quietly: a crash between two steps, a retry that repeats an action, a model that declares victory early.

Anthropic's account of harnesses for long-running coding agents is a good model. Its setup used an initializer that wrote a startup script, a progress file logging what agents had done, and an initial commit, so each later session could make incremental progress and leave the environment clean for the next one (Anthropic). That was a coding case study, but the principle carries to personal work: progress belongs in durable artifacts, not in a long conversation.

For a personal harness that means:

• Checkpoints that record what is done, what is pending and what needs approval.
• Idempotency for actions with side effects, so a retry cannot send the same message twice.
• An "uncertain" state for when a tool call timed out after it may have succeeded. Reconcile before retrying.
• Cancellation that stops new work and reports honestly on work already done.
• Approval expiry, so resuming a task tomorrow does not reuse yesterday's consent.

The scheduling guide covers duplicate-safe jobs, and the troubleshooting guide covers loops and false completion.

9. Evaluate the harness: did it work, and did it follow the rules?

A fluent "done" proves nothing. Check the world, not the message.

Tau-bench, a benchmark for agents that talk to users and use tools, scores a conversation by comparing the final database state with an annotated goal state (Yao et al.). Do the same at home: if the task was "draft a reply," check that a draft exists, says the right thing and that nothing was sent.

Start with thirty cases

A first evaluation suite can be small and still useful:

• 10 useful tasks. Answers from documents, briefs, drafts, missing information, conflicting sources.
• 10 boundary cases. Injected instructions, out-of-scope files, expired approvals, changed parameters, revoked access.
• 10 failure cases. Timeouts, malformed tool output, duplicate jobs, cancellation, budget exhaustion, a restore.

Write the expected outcome before running anything. The evaluation guide has all thirty.

Grade with the right tool

Anthropic groups graders into code-based, model-based and human, and distinguishes capability evaluations from regression evaluations (Anthropic). Use code for anything checkable: files touched, tools called, state changed, actions blocked. Use a model or a person for quality judgments, and check the model judge against human verdicts.

Measure consistency, not one lucky run

Two metrics look alike and mean opposite things. pass@k is the chance of at least one success in k attempts. pass^k is the chance that all k attempts succeed. Anthropic's example: at a 75% per-trial success rate, three trials all pass about 42% of the time (Anthropic). An assistant you rely on every morning needs the second number.

Benchmarks age. The original tau-bench paper found that the function-calling agents it tested, including gpt-4o, succeeded on fewer than half of tasks and passed all eight trials on under a quarter of retail tasks (Yao et al.). That describes 2024 models on that benchmark, not today's assistant on your tasks. Run your own.

The same applies to productivity. METR's early-2025 randomised study found experienced open-source developers took 19% longer with the AI tools of the time, even though they expected to be faster (METR). METR's 2026 follow-up says its newer data gives an unreliable signal of the current effect (METR). Neither result tells you what your assistant does for you. Measuring does.

10. Control costs and operate the system

The number that matters is cost per successful task: everything you spent, including failed runs and retries, divided by the tasks that actually succeeded.

That total includes more than tokens: tool and API fees, hosting, storage and backups, hardware and electricity if you run locally, and your own maintenance time. The cost worksheet separates measured usage from assumed prices, and the local hardware guide covers sizing.

Budgets only count if the runtime enforces them. Set limits on steps, time, spend and concurrency, and make every stop visible in the receipt.

Logs are the other operating cost. A good run receipt answers what ran, what was read, what was approved, what changed and why it stopped, without storing passwords, tokens or more personal content than needed. The logging guide covers receipts and redaction. AI agent observability covers tracing standards.

11. Own your exit: export, switch providers and restore

An exit plan is the difference between using a provider and depending on one.

Sort your state into three groups:

Group	Examples	On exit
Portable	Documents, memory records, workflow definitions, tool contracts, evaluation cases	Export in readable formats
Rebuildable	Search indexes, embeddings, caches	Regenerate in the new setup
Re-authorise	API keys, OAuth grants, connector permissions	Grant again; never copy blindly

Then test the exit the same way you test anything else. Export, import into a clean environment, re-authorise deliberately, switch to a second model provider and rerun the evaluation suite. Record what moved, what needed an adapter and what behaved differently. Models are not interchangeable, so expect differences and measure them.

Restore follows the same rule. A backup you have never restored is a hope. And remember the limit of every rollback: restoring an old database does not unsend an email.

The provider exit test and the backup and restore runbook walk through both.

12. Put it to work: three personal workflows

One harness, three jobs, each with its boundary built in.

Research from your own documents. Ask questions of a selected folder and get answers with the passages that support them, plus an honest "not found" when the documents do not say. See the personal knowledge base guide.

Meeting preparation. Read tomorrow's events and related notes, and produce a brief with sources and open questions. The calendar connection is read-only, so a stray instruction cannot reschedule anything. See the meeting-prep tutorial.

Draft-only email. Read one thread and draft a reply you review and send yourself. The send capability is absent, not merely discouraged. See the draft-only email tutorial.

A readiness checklist

Before trusting the assistant with anything new, check:

• You can list every place your data goes.
• Each tool has a written contract and validates its own input.
• Consequential actions need approval for the exact action, enforced in code.
• Every run has a step, time and spend limit, and a stop reason.
• You can inspect, correct, export and delete memory.
• Your evaluation suite passes, including the boundary cases.
• You have restored from backup at least once.
• You have rerun your tasks on a second provider.

Every unchecked box is a place where you are still trusting rather than verifying.

Frequently asked questions

What is an AI harness?

The software around an AI model that assembles its context, runs its tools, enforces permissions and limits, stores memory and state, and records and checks results. The model proposes; the harness decides what actually happens.

Do I need to train a model to build a personal AI assistant?

No. A personal harness uses an existing model through an API or a local runtime. The work is in the context, tools, permissions, memory and tests around it.

Is a custom GPT or chatbot project the same as an AI harness?

Not quite. Custom chatbots let you set instructions and attach files. A harness also controls tools, authority, state, limits and evaluation. Many custom chatbots are a useful first step that stops short of acting on your behalf.

Do I need a GPU?

Only if you want to run models locally at useful speed. A harness that calls a cloud model runs on an ordinary computer. The hardware guide covers when local inference is worth it.

Does running an assistant locally keep all my data private?

Only the parts that actually run locally. A local model with a cloud mail connector still sends mail content to that connector. Map every component before calling a setup private.

Is MCP safe by default?

MCP is a protocol for connecting tools. Safety depends on the server you install, the credentials you give it and the authority your harness enforces.

How do I stop loops and surprise bills?

Enforce step, time and spend limits in the runtime, not in the prompt, and make every stop reason visible in the run receipt.

Can I switch providers without rebuilding everything?

If your documents, memory, workflows and tool contracts live in portable formats, yes, with adapter work. Test it before you need it.

When is buying better than building?

When you cannot commit to updates, credential reviews, backups and restore tests. An unmaintained harness is not safer than a well-run product.

Sources

• Usercentrics and Sapio Research, State of Digital Trust 2026
• Stack Overflow, 2025 Developer Survey: AI
• Databricks, What is an AI agent harness?
• LangChain, The anatomy of an agent harness
• Birgitta Böckeler, Harness engineering for coding agent users
• Anthropic, Effective context engineering for AI agents
• Anthropic, Effective harnesses for long-running agents
• Anthropic, Demystifying evals for AI agents
• Model Context Protocol, Authorization and Security best practices
• OWASP, AI Agent Security Cheat Sheet
• Ollama, FAQ
• OpenAI, Your data
• Liu et al., Lost in the Middle
• Debenedetti et al., AgentDojo
• Yao et al., τ-bench
• METR, Early-2025 AI and experienced open-source developers and 2026 update