When a Model Returns Many Tool Calls at Once: How Harnesses Draw the Line

Abstract

A model's single reply often carries several tool_use blocks: read three files, grep two places, edit one config. Running every call in sequence is the safest option but slow. Running everything in parallel is fast but error-prone. The harness must draw a line between the two. This article quantifies both ends of that trade-off, then reads the source of three open-source harnesses — deepseek-harness, codex, and pi — and replays the same batch of tool calls through each to show where each project places the line. It covers lost updates, result ordering, approval mismatches, and cross-tool dependencies, and closes with a side-by-side comparison table grounded in fixed repository commits verified on 2026-11-01.

1. The Problem: In What Order Does a Batch of Calls Run

From a protocol standpoint, only two constraints exist. Every tool_use must have a matching tool_result. All results go back in the next user message. The protocol does not say which order tools run in, or which run concurrently. That decision belongs entirely to the harness.

The two extremes are easy to define. In serial execution, total time is the sum of every call. In parallel execution, total time is set by the slowest call. For N read calls, each taking 100ms, serial time is 400ms for N=4, while parallel time stays at 100ms. The gap grows as the count and the per-call latency rise.

The highest gains appear in two categories:

• Multiple reads and searches. When a model explores an unfamiliar codebase, it often reads four or five files and greps several keywords in one round.
• Network calls. Fetching pages and querying external APIs each take from a few hundred milliseconds to several seconds.

2. Sub-Agents and Streaming Execution

Sub-agents follow the same logic. Independent subtasks that each run for tens of seconds compress total time several times over when parallelized.

Codex goes a step further. Model replies arrive as a stream. In a batch, the first tool_use often arrives seconds before the last. Codex does not wait for the whole reply to end. It starts executing each complete call the moment it arrives, at codex-rs/core/src/session/turn.rs:3012-3014, and collects the remaining results after the stream ends at turn.rs:3421-3429. Tool execution overlaps with model streaming output.

The result order stays stable. Codex uses FuturesOrdered, at turn.rs:2775. No matter which future finishes first, results are retrieved in insertion order.

3. Safety: What Errors Does Parallelism Introduce

When calls share dependencies or shared resources, parallelism can break. Four categories are examined here. Each starts from the tool array the model returns, then shows what goes wrong under parallel execution. The time values are illustrative.

3.1 Lost Updates

The model returns two edit_file calls that modify two lines of the same file. Almost every edit implementation follows a pattern: read the whole file, replace in memory, write the whole file back. From the model's perspective it changed one line. On disk, the entire file is rewritten. When two edits run in parallel, both read the old content, and one write overwrites the other. Example: edit config.txt, old-name to new-name; edit config.txt, 8080 to 9090. Both return ok, but only one change survives.

3.2 Stale-Read After Concurrent Write

The first edit changes name; the second changes port. Both read the original file at 0ms. The write-back happens around 100ms. The second edit reads the old content, so the value returned to the model is name=old-name and port=8080, while the model expects to see new-name. The model may conclude that the change failed and retry, or it may continue reasoning on stale content. Section 4 shows that pi's file queue only serializes writes. Reads do not go through the queue. So pi can prevent lost updates but not this stale-read case.

3.3 Result Ordering

Three read-only calls have different durations: read_file a.ts at 300ms, read_file b.ts at 100ms, and grep "TODO" at 200ms. In parallel, the completion order differs from the order the model gave. The harness can backfill by completion order or by the model's original order.

Why does order matter when tool_use_id already matches each result to its call? The ID guarantees that results correspond to calls, not that the order is stable. Who finishes first depends on current load. If results are backfilled in completion order, the same task run twice can produce different histories. Conversation replay breaks, recorded-replay tests fail to match, and retries miss prompt caching. For these reasons, all three harnesses submit results in the order the model specified.

Real-time display and history follow different policies. Pi notifies the UI the moment each call completes, at agent-loop.ts:648, then writes history in original order after everything finishes, at agent-loop.ts:656-664. Display favors speed; history favors stability.

3.4 Approval Mismatch

Two calls both require user confirmation, and one is dangerous: bash ls build/ needs confirmation, and bash rm -rf build/ needs confirmation. If both prompts appear in parallel, a single "y" becomes ambiguous — which prompt did it answer?

Pi's execution model handles this. A batch first goes through a preparation phase, one call at a time, in the order the model gave. Confirmation happens there. After all calls pass, they execute in parallel. The prompts appear one after another, so answers cannot mismatch. The cost is that an already-approved first call waits while the user decides on the second. The deeper point is that concurrency problems are not limited to the data layer. They also appear in human interaction.

4. The Three Strategies

None of the three harnesses chose full serial or full parallel. Each defines its own rules. To expose the differences, the same batch of calls is replayed through each. The demo batch has five calls: read a.ts at 100ms, read b.ts at 100ms, edit c.ts at 100ms, edit d.ts at 100ms, and bash npm test at 300ms.

The model's output takes 200ms to stream, and the five calls arrive evenly. The arrival times are 0, 40, 80, 120, and 160ms. The batch contains three relationship types that separate the strategies: two independent reads (a and b), two edits on different files (c and d), and one command that depends on the earlier edits (npm test must wait for c and d, otherwise it tests half-modified code). The only correctness requirement is that call 5 starts after calls 3 and 4 finish. Within that constraint, faster is better.

4.1 deepseek-harness: Judge Per Call, Treat Uncertainty as Exclusive

After the reply ends, the scheduler processes the batch in three steps.

1. Classify each call. It calls the tool's isConcurrencySafe(args) and judges based on the arguments of this specific invocation. The rule is fail-closed. If the tool has no such function, if the function throws, or if the return value is not true, the call is treated as exclusive.
2. Group. Following the model's order, consecutive parallel-safe calls form a group. An exclusive call forms its own group and acts as a barrier: everything before it finishes first, and everything after it waits.
3. Execute and submit. Parallel groups run in a bounded pool, with a default cap of 10 concurrent calls. Each call is reclassified before it starts. Results are submitted in original order.

The built-in tools follow this: read is declared concurrency-safe, at tool-fs/src/read.ts:136. Edit, write, and bash are not declared, so they are exclusive. Grouping is at tool-calls.ts:89-90, reclassification at tool-calls.ts:200-205, the concurrency cap at constants.ts:6, and ordered submission at tool-calls.ts:146-147.

For the demo batch, this produces four groups: read a, read b; edit c; edit d; and bash. Each dashed line is a barrier. Total time is 800ms. The result is the most stable and the slowest. Call 5 always runs after calls 3 and 4. The cost is that c and d, even though they edit different files, are separated by a barrier and run one after another. Execution also waits for the full streamed reply, at agent.ts:529-551.

4.2 codex: Execute on Arrival, Queue Behind a Read-Write Lock

Codex applies three controls.

1. A model-side switch. The parallel_tool_calls parameter decides whether the model may return multiple calls in one round. It is true for normal dialogue turns, at turn.rs:1702, and switched off for some models, at client.rs:1118.
2. Execute on receipt. The reply streams in, and codex starts each complete call immediately without waiting for the reply to end, at turn.rs:3012-3014. Results come back in original order through FuturesOrdered, at turn.rs:2775.
3. A read-write lock. All calls in a round share one lock, at parallel.rs:51. Tools declared parallel take the read lock, and multiple read locks can be held at once. Other tools take the write lock and must be exclusive.

The lock logic is direct. If the tool supports parallelism, it takes a shared read lock; otherwise it takes an exclusive write lock, at parallel.rs:213-217. Tools are not parallel by default, at tool_executor.rs:122-124. Codex has no separate read tool. Reads and tests both go through exec_command, which is declared parallel and takes the read lock, at exec_command.rs:138-140. Edits use apply_patch, which is not declared and takes the write lock.

A critical detail: the lock is tokio's read-write lock, which queues first-come-first-served. A read-lock request does not jump ahead of a write-lock request already in front of it. This decides the fate of call 5.

In the demo, each call requests its lock at arrival. Calls 1 and 2 take read locks and run immediately. Call 3 waits for calls 1 and 2 to release. Call 4 queues behind call 3. Call 5 wants a read lock but cannot skip call 4 in front. Total time is 640ms.

The result is correct and faster than deepseek-harness. The speed comes from executing on arrival instead of waiting for the reply. Call 5 lands after calls 3 and 4 through the lock's first-come-first-served order, not because the harness detected the dependency. If the model had emitted call 5 before calls 3 and 4, it would take the read lock first. Like deepseek-harness, calls 3 and 4 still run one at a time even though they edit different files.

4.3 pi: Parallel Everywhere, Queue Writes Per File

Pi's defaults point the other way.

1. Parallel by default. The agent execution mode defaults to parallel, at agent.ts:261, and the coding agent does not change it, at sdk.ts:403-438. A tool may declare executionMode "sequential"; if any tool in the batch declares it, the whole batch becomes serial, at agent-loop.ts:526-532. None of the built-in tools declare it.
2. Prepare serially, execute in parallel. Pi waits for the full reply, at agent-loop.ts:279. It then walks the calls in the model's order, doing parameter validation and invoking the beforeToolCall hook for each, at agent-loop.ts:737-755. The hook can intercept a call. After everything is prepared, the batch runs in parallel, and results are collected in original order with Promise.all, at agent-loop.ts:656.
3. Queue file writes per file. The defense lives at the resource layer, not the scheduler. Edit and write tools pass through a queue grouped by file. Writes to the same file are serialized; writes to different files run independently. The queue key is the real path after symlink resolution. Reads and bash bypass this queue.

The implementation is a helper that serializes mutations targeting the same file while letting operations on different files run in parallel, in file-mutation-queue.ts:28-32. Edit at edit.ts:163 and write at write.ts:67 call it. The queue key is at file-mutation-queue.ts:16-26.

Pi has no built-in confirmation. Confirmation is implemented by extensions through the beforeToolCall hook, such as the permission-gate example that shows Yes/No for rm -rf or sudo. The hook runs in the serial preparation phase, so confirmation boxes appear one after another.

For the demo batch, all five calls start after the reply ends. Calls 3 and 4 edit different files, so each enters its own queue and neither waits. Call 5 passes through no queue and starts with calls 3 and 4. Total time is 500ms.

The result is the fastest, but this batch produces an error. Calls 3 and 4 on different files run simultaneously — the only harness of the three that can do this. But call 5 starts running the test at 200ms, while calls 3 and 4 do not finish until 300ms. The test runs against half-changed or completely unchanged code, so the result is untrustworthy. The file queue protects only multiple writes to the same file. It does not protect cross-tool dependencies such as a command that depends on write results.

5. Is This a Pi Bug?

No. It is a deliberate trade-off. Pi trusts the model to place only independent calls in the same batch, and the harness does not provide a fallback. The pi documentation states clearly that tool calls from one assistant message can run in parallel, at docs/extensions.md:236. The problem only appears when the model places calls with sequential dependencies in the same batch.

The premise usually holds. Models tend to group independent operations like reading files and searching code together. After editing, a model typically waits to see the result and runs tests in the next round. deepseek-harness and codex do not rely on this premise; their harnesses provide fallback protection at the cost of speed. Trusting the model favors speed. Harness fallback favors stability.

6. Side-by-Side Comparison

Dimension	deepseek-harness	codex	pi
Demo batch time	800ms	640ms	500ms
Call 5 runs after calls 3 and 4	Yes, via barrier	Yes, via lock FIFO order	No
Calls 3 and 4 on different files run together	No	No	Yes
Start timing	After streamed reply ends	On call receipt	After streamed reply ends
Defense layer	Scheduler	Scheduler	Resource layer (write queue)
Judgment granularity	Per call, per arguments	Per tool	Per file
Tool without declaration	Not parallel, exclusive; errors treated exclusive	Not parallel, write lock	Parallel
Implementation	Parallel groups plus exclusive barriers	First-come-first-served read-write lock	Queue grouped by real path
Concurrency cap	Default max 10	Unlimited	Unlimited
Main cost	Every tool needs a judgment function; conservative and slow	Coarse granularity; cannot differentiate by arguments	Cross-tool dependencies unprotected

7. Conclusion

The three harnesses represent three answers to the same scheduling question. deepseek-harness is conservative: judge each call, treat every unknown as exclusive, and pay with slower execution. Codex overlaps execution with streaming and leans on a fair lock to preserve order. Pi is the fastest, trusts the model's grouping, and accepts that cross-tool dependencies are not protected.

For teams building on these systems, the practical lesson is that parallelism is a correctness decision, not just a performance one. Latency, result ordering, write consistency, and approval flow are coupled. Before choosing a strategy, map which calls in a batch are independent, which share files, and which depend on earlier results. That mapping determines whether the fast path is safe.

Where network calls and external services dominate a batch, another layer of control becomes relevant. Independent network calls benefit from parallelism, but they share rate limits, authentication, and failure modes. Centralizing those calls behind a single routing surface, such as an API gateway, makes it easier to apply consistent retry, quota, and credential policies across the batch. A gateway like 4sapi gives teams one place to manage those external dependencies, so the harness can focus on ordering and consistency instead of re-implementing connection handling per tool.

The reference commits are fixed: openai/codex 7d592f6dd9, deepseek-ai/deepseek-harness 687f282e86, and earendil-works/pi 36be9c612, all verified on 2026-11-01.

---

International access: https://4sapi.com

Domestic access: https://4sapi.org