How AI Agents Work: The Loop, the Tools and the Harness Around Them

Image: Franck V. on Unsplash.

An AI agent is less mysterious than the marketing suggests. Anthropic, which builds agents for a living, puts it in one sentence: agents "are typically just LLMs using tools based on environmental feedback in a loop" (Anthropic, 2024).

That sentence has three parts, and each one is where agents succeed or fail: the model that decides, the tools that act, and the loop that feeds results back until the job is done. Around all three sits a fourth part most explanations skip: the harness, the software that runs the loop, holds the keys, stops it when it should stop and writes down what happened.

This guide walks through each part in the order a real run touches them. If you want the one-paragraph definition instead, see What is an AI agent?.

The short version

1. You give the agent a goal, not a script.
2. The model reads the goal and everything in its context, then decides on the next step.
3. If the step needs the outside world, the model emits a structured tool call. The harness runs it.
4. The result goes back into the model's context.
5. Repeat until the model says it is finished, a stopping rule fires, or a human gate pauses it.

Everything else is detail about doing those five steps reliably.

1. The model decides; it does not act

A language model only produces text. When an agent "sends an email", what actually happens is that the model outputs a structured request, something like send_email(to=..., subject=..., body=...), matching a schema it was given. Code outside the model reads that request and does the work. That mechanism is called function calling or tool calling.

This split matters for two reasons.

• The model cannot do anything it was not given a tool for. An agent's reach is exactly its tool list. That makes the tool list a security decision, not a feature list.
• Every action passes through code you control. That code can log the call, check permissions, ask a human, or refuse. The model proposes; the harness disposes.

OpenAI's agent guide describes the same anatomy in three components: a model for reasoning, tools for acting, and instructions that define behaviour and guardrails (OpenAI, *A practical guide to building agents*). It also draws the line that separates agents from ordinary AI features: applications that use a model "but don't use them to control workflow execution — think simple chatbots, single-turn LLMs, or sentiment classifiers — are not agents."

2. The loop: reason, act, observe

The loop has a research lineage. In 2022, Yao and colleagues published ReAct, which had a model interleave written reasoning ("I need the population of X, so I will search for it") with actions ("search[X population]"), and fed each action's result back in before the next thought. Grounding reasoning in real observations reduced the hallucination and error propagation seen in pure chain-of-thought, and on two interactive benchmarks ReAct beat imitation and reinforcement learning baselines by an absolute 34% and 10% success rate using only one or two examples (Yao et al., ReAct).

Modern agents keep the same shape. A single cycle looks like this:

Step	What happens	Who does it
Read	Goal, instructions, prior results and relevant memory are assembled into context	Harness
Decide	Choose the next action, or declare the task complete	Model
Act	Execute the tool call with the model's arguments	Harness
Observe	Append the tool's output (or its error) to the context	Harness
Check	Stop rule? Approval required? Budget left?	Harness

Anthropic stresses the observe step: at each step an agent needs "ground truth" from the environment, such as tool results or code execution, to judge its progress (Anthropic, 2024). An agent that cannot see the result of its last action is guessing.

3. Tools: where most of the engineering goes

It is tempting to think the prompt is the product. Practitioners report the opposite. Building its SWE-bench coding agent, Anthropic "actually spent more time optimizing our tools than the overall prompt." One example: the model kept making mistakes with relative file paths after it changed directory, so the team changed the tool to require absolute paths, "and we found that the model used this method flawlessly" (Anthropic, 2024).

Anthropic calls this the agent-computer interface and suggests investing as much in it as you would in a human interface. In practice that means:

• Clear names and descriptions. Write the tool description the way you would write a docstring for a new colleague.
• Hard-to-misuse arguments. Anthropic's word is poka-yoke: design the inputs so the wrong call is hard to make.
• Few overlapping tools. OpenAI notes that some systems manage more than 15 well-defined, distinct tools while others struggle with fewer than 10 that overlap (OpenAI guide).

OpenAI groups tools into three kinds, which is a useful checklist for any agent you design:

• Data tools read context: query a CRM, read a PDF, search the web.
• Action tools change something: send a message, update a record.
• Orchestration tools are other agents called as tools.

The Model Context Protocol standardises how tools are exposed, so one MCP server works with any compatible client instead of needing custom glue per model.

4. Context and memory: what the model can see

The model only knows what is in its context window on this call. Everything an agent "remembers" is either in that window or retrieved into it. Two consequences follow.

First, more context is not free. Chroma tested 18 models and found performance "consistently degrades with increasing input length," even on deliberately simple tasks. On a conversational memory benchmark, models did significantly better with a focused prompt of about 300 tokens than with the full ~113,000-token history containing the same answer (Chroma, *Context Rot*).

Second, long-lived agents need memory outside the window: stored facts, preferences and past results that are fetched when relevant rather than replayed every time. We cover the types and trade-offs in AI agent memory, explained.

5. Stopping: the part demos leave out

A loop that never ends is a bill that never ends. OpenAI's guide lists the usual exit conditions: a final-output tool is called, the model replies without calling any tool, an error occurs, or a maximum number of turns is reached (OpenAI guide). Anthropic recommends stopping conditions "such as a maximum number of iterations to maintain control" (Anthropic, 2024).

Good agents also stop for a human. OpenAI names two triggers for human intervention: exceeding failure thresholds, such as repeated retries, and high-risk actions, such as cancelling orders, large refunds or payments.

6. The harness: everything that is not the model

Put the pieces together and the model turns out to be one component among several. The rest is the harness:

• Tool execution with the credentials the model never sees
• Permissions: which tools this agent may call at all
• Approval gates in front of consequential actions
• Budgets and turn limits so a confused agent stops
• Memory read and written between runs
• Tracing: a record of every prompt, tool call, result and cost

That last item is how you find out why an agent did something. LangChain's 2026 survey of 1,340 practitioners found 89% of teams had implemented observability for their agents, well ahead of evals at 52% (LangChain). See AI agent observability for what a good trace contains.

We make the longer argument for why this layer decides outcomes in Model vs. Harness.

One task, traced end to end

Goal: "Every Monday, summarise last week's new support tickets and post the three biggest themes to Slack."

1. Trigger. A schedule fires Monday at 9:00. No model involved yet.
2. Read. The harness loads the goal, the agent's instructions and a stored preference ("keep summaries under 150 words").
3. Decide → act. The model calls search_tickets(since="7 days ago"). The harness runs it with the helpdesk credentials and returns 212 tickets.
4. Decide. The model groups the tickets into themes. That is pure reasoning, so no tool is called.
5. Check. Posting to a public channel is marked as needing approval, so the run pauses and a person approves the draft.
6. Act. post_message(channel="#support", text=...) runs.
7. Stop. The model replies with no further tool call. The harness writes a receipt: inputs, the two tool calls, their outputs, tokens and cost.

The only step the model owned outright was deciding what the themes were. Everything else was the harness doing its job.

When you do not need an agent

Anthropic's advice is to find "the simplest solution possible, and only increasing complexity when needed," noting that agentic systems "often trade latency and cost for better task performance" (Anthropic, 2024). If you can draw the steps in advance, a fixed workflow is cheaper and more predictable. Use an agent when the next step genuinely depends on what the last one found.

How this maps to AGNT

Disclosure: AGNT is our product. In AGNT, an agent is a stored object with an explicit tool list, its own model and a credit limit. It carries typed memory between runs and writes a receipt for every execution, covering the prompt, each tool call with its input and output, and tokens and cost per step. Deterministic steps live in workflows; open-ended outcomes run as goals that plan, execute, evaluate and replan. It runs locally, and Community Core is free.

FAQ

What are the main components of an AI agent?

A model that decides, tools that act, instructions that set behaviour, and a harness that runs the loop, holds credentials, enforces limits and records what happened.

How is an AI agent different from a chatbot?

A chatbot returns text. An agent uses a model to control a sequence of actions through tools, observes the results and keeps going until the goal is met or a stop rule fires.

Do AI agents learn while they work?

The model's weights do not change during a run. Agents improve across runs by writing to external memory, such as corrections, preferences and patterns that worked, and retrieving it later.

Why do AI agents make mistakes?

The usual causes are unclear tool definitions, missing or overloaded context, and no feedback from the environment. Errors also compound: a wrong early step shapes every later one, which is why stop rules and human gates matter.

Sources

• Anthropic, Building effective agents (December 2024)
• OpenAI, A practical guide to building agents
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (ICLR 2023)
• Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance (July 2025)
• LangChain, State of Agent Engineering (survey fielded Nov–Dec 2025)