AI Agent Learning Notes: From Large Language Models to Harness

Introduction

Many developers new to AI agents start with a simple mental model: an agent is merely a large language model with tool calling capabilities. This assumption is intuitive, but practical testing reveals that it is incomplete. A model can produce a seemingly correct final answer, while the internal execution path is flawed. Successful local tool execution does not guarantee that the tool output has been passed back to the model correctly, nor does it confirm that the final result is grounded in valid intermediate data.

This article summarizes a series of controlled experiments focused on agent execution pipelines. It unpacks the boundary between the model and the harness, clarifies how the ReAct feedback loop actually operates, and establishes practical verification criteria for agent workflows. The core formula for an agent can be written as:

Agent = LLM + Context + Tools

For engineering implementation, the expression expands into a more precise form:

Agent = Model + Harness

The harness is the runtime component responsible for tool dispatch, result collection, context assembly, and state iteration. The LLM only generates structured tool invocation requests. The harness executes the calls, captures outputs, and constructs the context for subsequent rounds. This division of responsibility is the foundational distinction that most introductory material overlooks.

1. An Agent Is More Than a Tool-Calling LLM

The three core components of an agent system each carry distinct responsibilities:

• Context: The full state of the conversation, historical tool returns, and environmental snapshots.
• Model: The LLM that decides the next action and generates tool call payloads.
• Tools: Executable functions that interact with external systems, files, and APIs.

The model itself does not execute code or network operations. It only emits request payloads. The harness, as external runtime logic, handles execution, collects return values, and packages data for the next inference round.

1.1 A Counterexample: Correct Final Output, Faulty Execution Path

One controlled experiment demonstrates this risk clearly. The task required currency conversion: convert 1000 US dollars into Japanese yen, then convert that yen value into euros. The model produced a final numeric answer that appeared correct.

However, inspection of the raw request payload revealed a critical defect: the model emitted two tool calls in a single generation turn, before the first call had finished running. The second conversion parameter was not calculated from the real return value of the first tool. It was a speculative value guessed by the model in advance.

This creates a dangerous failure mode. The final number may coincidentally match the expected value, but the workflow is not valid. If the tool returns a different value in production, the downstream calculation will silently produce incorrect results.

For valid agent execution, reviewers must verify more than just the final answer. Four checkpoints are mandatory:

1. What tools and parameters did the model request?
2. Did the harness dispatch the calls correctly?
3. What return data entered the next round of context?
4. Which dataset did the model use for its final reasoning?

2. The Essence of ReAct: A Verifiable Feedback Loop

Many tutorials describe ReAct as a cycle of Reason, Action, Observation. This summary is not wrong, but it can lead to misunderstanding: readers may assume the model independently completes the entire cycle. In reality, ReAct is a handoff protocol between the model and the harness.

A complete ReAct iteration follows this sequence:

1. User submits a task
2. Harness packages the current context
3. Model decides on an action and outputs tool calls
4. Harness dispatches the tool calls
5. External tools run and return results
6. Harness captures observation data
7. Harness appends observations to the context
8. A new inference round begins with the updated context

This pattern resembles the message loop in event-driven frameworks such as Android. The key difference is determinism. Android message routing follows fixed rules, while an LLM may generate different actions even with identical context.

2.1 Step-by-Step Progress in a Full Trace

Another test task required reading a PDF file and performing currency conversion on values extracted from the document. The baseline workflow had three sequential rounds:

1. Round 1: Request to read the target PDF
2. Round 2: Parse extracted numeric values and call the conversion tool
3. Round 3: Summarize outputs and produce the final answer

The trace confirmed valid data flow: the numeric value extracted from the PDF was passed as the parameter of the currency tool, and the tool return was fed into the final reasoning prompt. The execution path was coherent and grounded.

2.2 What Happens When Historical Context Is Stripped

The same task was rerun with a modified harness configuration: the tool return observations were excluded from the context of the next round. The model repeatedly requested the PDF reading tool, but never moved on to conversion. The model never received the extracted number, so it could not proceed.

This experiment delivers a critical lesson: local tool success does not equal visible observation for the model. If the harness fails to append observation results into the context, the ReAct loop breaks entirely. The model has no built-in memory of tool outputs; all state is mediated by the harness context.

3. Tool Calling Is Not a Single Action, It Is a Pipeline

A naive interpretation treats a tool_call block as proof that the tool finished successfully. This assumption is unreliable. A complete tool-calling pipeline has multiple discrete stages:

1. Model generates structured tool call payload
2. Harness validates the payload schema
3. Harness dispatches the function
4. Target tool executes
5. Tool returns raw output or error
6. Harness captures the output
7. Harness sanitizes and formats the observation
8. Harness injects the observation into context for the next turn

3.1 Matching tool_call_id Proves Only Request-Response Alignment

Tool call protocols include unique tool_call_id fields to match requests and returns. Matching IDs only verify that the response block corresponds to a prior request. This does not prove:

• The tool logic ran completely
• The return value was not modified by the harness
• The model correctly parsed the observation

This is analogous to transaction IDs in RPC systems. Matching IDs confirm that a response was paired with a request, but extra logging and status checks are still required to validate business outcomes.

3.2 executed: true Does Not Guarantee Business Completion

A separate file-writing experiment exposed another common pitfall. The harness marked executed: true before the file write operation finished. The status flag was logged, but the target file was never created on disk.

Boolean execution flags are lightweight metadata. They should not be treated as proof of side-effect completion. For file operations, database writes, and external API calls, verification must use authoritative checks: file existence probes, database select queries, or upstream service status endpoints.

4. Context Is Not a Chat Log; It Is the Input Snapshot for Each Inference

Developers often treat context as a persistent chat history. For agent engineering, context is defined strictly as the full input snapshot supplied to the model for a single inference request. It may include:

• System prompts
• User messages
• Assistant prior turns
• Tool return observations
• Tool schemas
• Harness runtime rules

The harness stores this state, and the model only sees what the harness forwards. The presence of a message in the historical log does not mean the model received it in the current prompt. Debugging agent failures must inspect the raw request payload sent to the model, rather than the final rendered conversation log.

4.1 The Model May Still Produce Correct Output Even with Malformed Observations

A control experiment altered the role tag of tool return messages. The observation was marked as user instead of tool. The model still produced the correct numeric answer in this specific test.

This does not mean role tags are irrelevant. It only demonstrates that some models can recover from minor schema violations. Behavior varies widely across model providers, versions, and task complexity. The correct schema remains a hard requirement for stable production agents.

5. Caching Optimization Targets Prompt Stability, Not Just Cost

Context length directly impacts inference latency and token expense. Two common caching mechanisms are used in agent stacks:

• KV Cache: Reuses key-value tensors from previous inference passes for identical prompt segments.
• Prompt Cache: Reuses precomputed prompt content across multiple requests.

Both mechanisms depend on stable prompt structure. Small changes to the front section of the prompt invalidate the cache, even if later content remains identical. This is similar to incremental build logic in compilers.

5.1 Cached Token Count Is Not Equivalent to Billing Price

A separate benchmark tracked token consumption during repeated agent runs. The prompt token count and cached token count stayed consistent across runs, but the actual billed token volume changed.

Caching discounts are provider-specific. The logged cached token metric is a runtime statistic. Engineers must cross-reference the provider’s pricing rules to calculate real cost, rather than relying only on internal token counters.

6. Prompts Are Powerful, but They Cannot Replace Runtime Guardrails

Prompt engineering shapes the model’s decision space, but the harness is the ultimate enforcement boundary for agent behavior.

6.1 Removing Tool Descriptions Does Not Immediately Disable the Tool

In one test, the description field of a file-search tool was deleted from the tool schema. The model still invoked the tool successfully. The parameter schema and function name alone provided enough signal for the model to select it.

This shows that tool descriptions are auxiliary guidance, not mandatory triggers. Tool availability is controlled by whether the harness includes the schema in the context, not only by descriptive text.

6.2 Runtime Permissions Are the Hard Boundary

A file-writing test used identical prompts with different harness permission configurations. The model generated the same tool call in both cases, but the harness blocked execution in the restricted environment.

A warning sentence in the prompt cannot override runtime permission rules. The harness is the enforcement layer for safety, file access, and external API permissions. Prompt instructions are only advisory; the harness makes the final allow/deny decision.

7. Agent Skills Are Implemented Through Progressive Loading

Early attempts to build agent skills packaged all logic into a giant system prompt. This approach hits context limits and degrades reliability. Production skill systems usually adopt layered loading:

1. Base layer: Skill name and high-level description
2. Schema layer: Parameter definitions and validation rules
3. Implementation layer: Executable logic, examples, and error handling

At runtime, the harness loads only the required skill schemas into the context based on the task. This reduces prompt bloat and keeps inference stable.

7.1 A Skill Is Not Automatically Invoked Just Because It Is Loaded

A test task required the agent to generate and export a PDF file. The harness loaded the PDF skill schema, but the model did not select it. The model only emitted generic file-writing calls.

Skill availability in the context is a prerequisite, not a guarantee of invocation. The model must decide that the skill matches the task intent. The harness can log skill loading events, but it cannot force the model to select a specific function.

8. The Core Debugging Mindset: Trace the Complete Pipeline

When an agent produces wrong output, the traditional troubleshooting habit is to rewrite the prompt. This is often ineffective. The structured checklist for agent debugging follows the execution flow:

1. What task did the user submit?
2. What full context snapshot did the model receive?
3. What tool calls did the model generate?
4. Did the harness validate and dispatch the calls?
5. What raw value did the tool return?
6. What observation did the harness inject into the next context?
7. What context did the model use for the final answer?

Most silent failures occur between the model output and the next inference round. The harness is the highest-risk zone for bugs. Observability pipelines for agents must capture every stage of this handoff chain.

9. Conclusion: Agent Engineering Centers on Reliable Loops

Looking back at the learning journey, the simple definition of an agent as “LLM plus tools” is insufficient. The robust definition is:

An agent is a system composed of a model and a harness, which iteratively maintains context, invokes tools, collects observations, and validates outputs.

The model is responsible for reasoning and action selection. The harness controls execution, state persistence, and safety guardrails. Even with a capable model, a harness without validation logic can continue propagating incorrect intermediate data.

The core competency of AI agent engineering is not prompt crafting. It is building a verifiable closed loop. Engineers must be able to trace every value from the original user request through tool execution, observation formatting, and final reasoning. This end-to-end audit trail is the foundation of reliable agent deployment.

As agent systems scale, routing multiple model backends and tool endpoints becomes a major operational burden. Teams managing heterogeneous LLM services can streamline their workflows with an API gateway that unifies incompatible provider interfaces. 4sapi exposes a consistent calling layer for diverse model providers, so agent harness implementations can remain unchanged while switching or routing between upstream models. This abstraction isolates the agent runtime from provider-specific API differences and reduces the amount of conditional logic required inside the harness.

The experiments in this article highlight a recurring theme: correctness is not judged by the final string printed to the user. It is judged by the integrity of the path that produced that string.

International access: https://4sapi.com

Domestic access: https://4sapi.org