C2C AI Agents: KV Cache Transfer Explained
Introduction
Multi-agent pipelines commonly adopt a sequential workflow: a planner model generates instructions and hands them to a coder, whose outputs are then passed onward to a reviewer. In nearly all mainstream implementations, communication between these models follows the same pattern. The upstream model converts internal reasoning states into a sequence of text tokens, and the downstream model reads those tokens as new input. This token translation step carries non-trivial overhead. High-dimensional internal representations are compressed into linear text sequences, which discard a large volume of implicit information. Every transmission also forces the receiver to re-compute all token embeddings sequentially.
A 2026 IC LR paper from Tsinghua NIC-SFC Lab, Shanghai AI Lab and co-authors introduces a paradigm called Cache-to-Cache (C2C). Instead of serializing model state into natural language, C2C directly transmits the KV-Cache from one model to another. Benchmark results show an average end-to-end latency reduction of 2.5×, paired with 6.4 to 14.2 percentage point accuracy gains. While these performance metrics attract wide attention, the paper’s own discussion of inherent constraints determines whether C2C can be deployed into real production systems. This article breaks down the core mechanics, quantitative experimental results, practical bottlenecks, scenario suitability and decision frameworks for C2C multi-agent communication.
1. Three Core Costs of Text-to-Text (T2T) Communication
The conventional multi-model interaction method is formally named Text-to-Text (T2T). In T2T workflows, Model A decodes natural language output, and Model B ingests that text as prompt input. The authors decompose this pipeline into three inherent, mechanically generated costs, which cannot be eliminated merely by prompt tuning.
First, bandwidth loss. When an LLM processes a prompt, high-dimensional states are distributed across dozens of attention layers. These states contain semantic meanings, partial conclusions and unresolved reasoning ambiguities. When passed to a second model, all this information must be squeezed into linear token sequences. Any data that fails to fit within this text representation gets discarded. The receiving model reconstructs meaning purely from literal text and cannot access the true internal state retained by the sender.
The paper includes an intuitive example in Figure 2. A coder model sends an instruction to a writer model, requesting that content be wrapped inside designated separators. The paragraph divider tag loses structural semantic information once converted into plain text. The writer model treats the marker as ordinary words and misplaces the target content. Such structural metadata cannot be fully encoded inside plain-text communication channels.
Second, semantic ambiguity. Natural language is intrinsically fuzzy. Idioms, ambiguous references and vague phrasing appear constantly in human-style text. Protocols such as MCP and A2A can standardize message templates, yet rigid templates conflict with flexible, open-domain collaboration. Template-based specifications struggle to accommodate dynamic reasoning content generated during agent workflows.
Third, latency overhead, which is straightforward to quantify. Autoregressive generation forces the source model to emit tokens sequentially. The downstream model can only begin processing once the full text response completes. In multi-stage agent pipelines, these sequential generation delays accumulate. For branched inference pipelines and multi-step planning agents, token-by-token serialization dominates end-to-end latency and contributes significantly to inference expense.
2. C2C Core Architecture and Training Design
C2C belongs to a broader family of methods that inject external information into frozen neural networks. It shares core conceptual roots with Adapter and LoRA. The key difference lies in injection location: C2C inserts external data into KV-Cache activations rather than modifying model weight parameters. Two critical design choices are inherited from existing work: residual addition preserves the receiver’s native capabilities, and learnable gating controls the injection intensity.
The training setup imposes strict constraints. Both LLMs, the Sharer (sender model) and Receiver (recipient model), remain fully frozen during training. Only the Fuser module is optimized. Training uses standard next-token prediction loss, teaching the Receiver to predict outputs conditioned on fused cache data. The Fuser module contains 478 million parameters.
The internal pipeline of the Fuser can be broken down into two primary components:
Dynamic Weighting: Re-weight information based on the current query. Not all layers and cached states carry equal importance for a given problem.
Learnable Gate: A Gumbel-sigmoid gate makes layer-wise binary decisions, determining whether to incorporate external cache data or retain the Receiver’s original cached states.
The external cache is merged through residual addition instead of full replacement. The Receiver’s final KV-Cache is enhanced while its native representations are retained.
2.1 What Layer Gating Reveals About Model Behavior
The researchers analyzed gating decisions after training to observe layer selection preferences:
General tasks such as MMLU and ARC favor broad activation. Many layers accept injected cache information.
Specialized reasoning benchmarks like GSM8K math tasks prefer sparse selection, activating only a small subset of target layers.
The selection of which layers absorb information from the Sharer is not arbitrary hyperparameter tuning; it emerges statistically and varies by task category. This observation corroborates findings from Oracle 1, which noted distinct layer-wise behavior across model tasks.
3. Experimental Results: Improvements Extend Beyond Speed
The main benchmark suite includes OpenBookQA, ARC-Challenge, MMLU-Redux, C-Eval, and LongBenchV1 to evaluate long-context performance. GSM8K was used for mathematical reasoning tests. Training datasets were OpenHermes-2.5 with 500k samples and LongBench-E.
| Comparison Group | Measured Outcome |
|---|---|
| Versus standalone single model | Average accuracy rises by 6.4–14.2 percentage points |
| Versus T2T text communication | Accuracy improvement of 3.1–5.4 percentage points |
| Latency | Average 2.5× speedup; extreme cases reach 14.41× for slow Sharer models |
| Fusion overhead | Cache combination consumes roughly 90 ms for each response |
Ablation studies validate the source of performance gains:
| Ablation Variant | Performance Gain |
|---|---|
| Residual fusion, compared with discarding Receiver native cache | +24.18% |
| Add learnable gating against no-gating baseline | +3.07% |
The control groups are rigorously designed. Under identical training budgets, C2C outperforms intuitive alternative baselines:
It beats the setup without a Sharer, directly feeding data to Receiver, proving gains are not derived simply from extended training iterations.
It outperforms the configuration using one single model acting both as Sharer and Receiver, ruling out benefits from extra parameter count.
The C2C Fuser uses fewer trainable parameters (478M), compared with alternatives at 596M and 529M.
These three results eliminate two common false explanations for performance lift: increased model capacity and overfitting to training corpora. The measured improvements genuinely originate from complementary semantic information delivered by the Sharer cache.
Two secondary discoveries add depth to the analysis: First, fusion increases the effective rank of the Receiver KV-Cache. Higher effective rank means the merged cache embeds richer semantic dimensions. This provides secondary evidence beyond accuracy metrics, confirming that meaningful information propagates between models. Second, the Sharer may be a base model with weaker instruction-following ability. As long as the Sharer captures robust internal semantic representations, it can serve as the sender. Core semantic understanding and instruction compliance can be separated during transmission. This insight is valuable for building specialized modules using open base models.
4. Five Fundamental Engineering Roadblocks
This section covers the core limitations acknowledged by the paper authors in Section 5 and appendices. These constraints determine production readiness.
Roadblock 1: A Separate Fuser Must Be Trained for Every Pair of Models
The Fuser is not tied to one individual model. It belongs exclusively to a specific Sharer-Receiver pair. Changing either the sending model or receiving model requires retraining the Fuser.
At scale, N interconnected models would demand O(N²) separate Fuser modules. The paper references a sketch design using shared projectors and multi-Sharer schemes to reduce complexity to O(N), but this remains an early prototype rather than the core validated contribution.
Training cost is relatively modest in isolated experiments. A checkpoint converges within 300 training steps, consuming fewer than 9 GPU hours. A full training cycle takes around 45–54 GPU hours. Even so, multiplying this cost across all pairwise model combinations creates prohibitive overhead for large multi-agent systems.
Roadblock 2: White-Box Access Blocks Closed API Models
C2C requires direct reading access to internal KV-Cache of both participating models. This creates a hard constraint:
Closed API models such as GPT-6 Astra and Claude cannot be used as Sharers.
These closed models also cannot serve as Receivers.
The entire framework only works for models where developers own weights and can modify inference code.
This represents the fundamental tradeoff of C2C. Text communication works universally across any two models. Cache transmission delivers high-fidelity state transfer and acceleration, but only for white-box deployments. C2C trades generality for inference efficiency, and the benefit is not free.
Roadblock 3: Low-Quality Sharer Cache Degrades Receiver Performance
The rule holds for both T2T and C2C multi-model systems: semantic quality from the Sharer directly defines Receiver output quality. If a weaker upstream model transmits corrupted cache states, Receiver performance drops.
A subtler failure mode exists. The Sharer may build an incorrect understanding of context and embed flawed reasoning inside its cache. This misleads the Receiver into wrong conclusions. In T2T text workflows, human operators may spot errors from readable text. In C2C cache transmission, such hidden misdirection becomes harder to catch.
Roadblock 4: Opaque Cache Data Disables Logging, Audit and Content Filtering
This is the most overlooked yet devastating engineering limitation. Text messages can be printed, filtered, audited and intercepted mid-flow. KV-Cache transmission removes this observability:
Engineers cannot inspect what the Sharer “thought” inside cache data.
Content security filters cannot operate on cache tensors.
Attack traces leave no human-readable logs.
Poisoned cache data sent from Sharer can corrupt Receiver reasoning without detection.
Ironically, this property doubles as a privacy advantage. Transmitting cache avoids exposing raw plaintext reasoning content, useful for cloud collaboration scenarios where only compressed cache fragments are transferred instead of source text. The same trait that strengthens privacy creates observability failures required for compliance, debugging and incident response.
This problem is part of a broader trend. Internal state opacity is emerging across multiple research directions, including CoT and Plan Injection. C2C extends this observability challenge from single models to multi-agent systems.
Roadblock 5: Narrow Experimental Scope
The headline 6.4–14.2% accuracy improvement comes under tightly constrained experimental settings:
Receiver models are mostly small-scale models
Tasks are multiple-choice questions
Decoding strategy uses greedy decoding
Maximum answer length capped at 64 tokens
Long agent workflows exist only as case studies and are not quantified in main tables
This means the performance gains for long multi-step agent pipelines are only qualitatively demonstrated. Teams planning production deployment for long agent workflows must run custom benchmarking to measure real-world performance gaps.
Two additional caveats are noted by authors: token alignment and scheduling rules are not yet optimal. Robustness under substantial architecture divergence between Sharer and Receiver still needs further validation. The researchers also built an enhanced Fuser variant (C2C-C) that narrows performance gaps between weak Sharers and strong Receivers. This version was not released as primary result, indicating published numbers do not represent the theoretical upper bound for this paradigm.
5. Workflow Profitability: When C2C Adds Value, and When It Does Not
The decision criterion is not simply “whether multiple models exist in the pipeline.” It hinges on whether the link suffers heavy translation loss when converting internal states to text tokens.
| Scenario | Verdict | Rationale |
|---|---|---|
| Small Receiver paired with powerful, complementary Sharer | Profitable, highest priority | Larger gains when Receiver size is small and knowledge divergence between models is high |
| Strong Receiver paired with similar-capacity Sharer | Not profitable | Marginal improvement; gains shrink with larger Receivers and overlapping knowledge |
| Long sequential pipelines forced to wait for full text generation | Profitable | 2.5× latency reduction can eliminate serialization bottlenecks, up to 14.41× acceleration |
| Sharer output intended for human reading | Not profitable | Text must still be generated for human consumption, C2C does not remove that cost |
| Tool calls and messages intended for human consumption | Not applicable | MCP/A2A protocols remain better suited |
| Any stage relies on closed API models | Not applicable | White-box weight access requirement blocks adoption |
| Workflow requiring audit and compliance logs | Proceed cautiously | Opaque cache creates compliance risks |
| One-off short task | Not economical | Fuser training overhead outweighs runtime benefits |
In short: C2C delivers value by cutting costs associated with information translation. If the two models share nearly identical internal representations or human-readable output is mandatory, the translation cost is negligible and C2C delivers no net benefit.
6. Decision Tree and Practical Implementation Advice
When evaluating C2C adoption, teams follow this structured decision path:
Is the communication purely internal reasoning between models, or does it need human-readable text or tool invocation? If human-facing / tool calls are required, stick with MCP/A2A protocols instead of C2C.
Can both Sharer and Receiver be accessed as white-box deployments with full weight and internal KV-Cache access? If relying on closed third-party APIs, C2C cannot be used.
If white-box access is available, assess capability divergence between Sharer and Receiver. If models are highly similar, expected gains are minimal.
Evaluate pipeline iteration count. If the same Sharer-Receiver pair runs many thousands of inference cycles, the fixed Fuser training cost amortizes well. Low-volume use cases do not justify training investment.
Three actionable recommendations for engineering teams:
Benchmark with Oracle 1 first. Validate layer-wise interaction behavior before committing resources to full C2C training. Run T2T baselines and test whether Receiver performance improves when receiving enhanced cache data.
Start with lightweight training. 300 training steps require less than 9 GPU hours and deliver near-final checkpoint quality. Avoid launching full 45–54 hour training cycles in initial trials.
Retain fallback text communication paths. Even when C2C is enabled, preserve a text-branched pipeline for debugging and audit. The observability limitation (Roadblock 4) will surface in production incidents.
7. Final Assessment
C2C is not a direct replacement for MCP or A2A. It adds a new layer to the multi-agent communication stack. The hierarchy of agent communication paradigms can be framed as follows:
C2C KV-Cache transfer: High bandwidth, low latency, non-human-readable, white-box only. Designed for semantic state transmission between two white-box models.
Structured text protocol: Human-readable, auditable, cross-vendor compatible. Used for general-purpose agent interaction.
MCP/A2A tool and message protocol: Universal standard for tool invocation, human-facing outputs.
The true contribution of this paper is not merely the 2.5× speedup number. It experimentally proves that KV-Cache can function as a direct communication medium between separate LLMs. This reshapes multi-agent system design boundaries, even though performance metrics vary by task.
Three critical judgments for technical roadmap planning:
C2C remains an algorithmic research prototype. It cannot be plugged directly into production API stacks. Before adoption, teams must evaluate observability, compliance and cross-model robustness.
Observability is a core unsolved challenge. Multi-step agent workflows become black boxes once internal cache transmission replaces readable text. Resolving observability risks is mandatory before production rollout.
Fuser sharing across multiple model pairs remains unsolved. Without solving the O(N²) training cost problem, large-scale multi-agent deployment remains impractical. Treat C2C as a promising compiler-level optimization rather than a drop-in general framework.
When designing multi-agent routing pipelines, developers can leverage an API gateway to manage conventional text-based model orchestration across heterogeneous LLM endpoints. 4sapi, an API gateway, simplifies request routing, credential management and model fallback logic for standard T2T agent workflows. It serves as a practical complement to C2C research, which is limited to white-box model deployments.
C2C expands the design space for multi-agent systems, but its five fundamental constraints mean production teams must apply strict scenario screening before implementation. It excels for internal, closed white-box agent pipelines with heavy translation overhead, while standard text protocols and API gateway orchestration remain the safer default choice for most general agent deployments.
International access: https://4sapi.com
Domestic access: https://4sapi.cn
