Trophies Are Not Tools: On Priming LLM Agents
with Records of Their Own Past Success

1  Introduction

Consider a concrete proposal. An agent is about to begin a difficult, multi-session engineering task. Before it starts, we prepend to its context a list of its own prior accomplishments — a recompiler that reached 99.9% verified native coverage, a static translation that produced 1,698 basic blocks and passed 805 assertions with zero failures, a benchmark run scored 100.0, a set of candidate original proofs, a plugin validated 77/77 against a live protocol. The hypothesis is that this establishes a standard: the agent, conditioned on a record of rigorous work, will produce rigorous work.

The proposal is intuitive, cheap to implement, and — on the evidence — wrong in the form stated. It is also very close to a proposal that is right, which is why the distinction is worth establishing precisely rather than dismissing.

This report separates two things that the intuition conflates. The first is conditioning on outcomes: showing the agent that it succeeded, and by how much. The second is conditioning on method: showing the agent how the success was produced, as an executable procedure. These have opposite measured effects. Our contributions:

• A synthesis of the one published study that tests the outcome-conditioning hypothesis nearly directly, in the same domain (software engineering), with a uniformly negative result (§3).
• Identification of the operative mechanism as lexical rather than semantic, using the original authors' own two mechanism probes (§3.4), and a previously unremarked decisiveness inversion in their turn-count data (§3.5).
• A four-axis model of experiential transfer that separates the negative result from the positive one and predicts the sign of an intervention before it is run (§5).
• An in-situ natural experiment from a deployed agent system in which an identical tool sequence receives opposite automated verdicts 27.5 minutes apart, discriminated only by task analogy (§6).
• A content decomposition showing that in a representative achievement list, roughly one third of entries carry transferable content and the highest-transfer entry is a failure (§7–8).
• Eight implementable design rules for an agent memory system, plus three falsifiable predictions and a proposed in-house experiment (§9–10).

2  Background: Three Families of Experiential Memory

Work on giving language-model agents memory of their own past falls into three families that differ in what is stored, and that difference turns out to determine the sign of the result.

(i) Intra-task reflection. ReAct [1] conditions on the current trajectory only. Reflexion [2] adds verbal self-reflection over failed attempts within a task instance, then discards it at task boundary. Neither attempts cross-task transfer.

(ii) Cross-task knowledge distillation. ExpeL [3] gathers trajectories across a training set and distills natural-language "insights" from them — both from sets of successes and from success/failure tuples — then supplies the accumulated insight set at inference. ExpeL reports consistent gains on HotpotQA, WebShop and ALFWorld. This family is the closest published analogue of "recall your past successes before starting."

(iii) Workflow induction. Agent Workflow Memory (AWM) [4] consumes the same trajectories but emits a different object: a workflow, an abstracted, reusable routine with a description and an ordered step template, supplied selectively rather than wholesale. AWM composes workflows into more complex workflows over time.

Family (ii) and family (iii) are built from identical raw material. They differ in abstraction level and in delivery discipline. As §3 and §4 show, that difference is worth roughly twenty points of relative success rate in opposite directions.

Underneath all three sits a general result that constrains any of them: language models are measurably degraded by irrelevant context even when they retrieve perfectly. Shi et al. [5] constructed GSM-IC by inserting irrelevant sentences into grade-school math problems and found accuracy "dramatically decreased," with partial mitigation from self-consistency decoding and from an explicit instruction to ignore irrelevant material. Any memory scheme that injects content not needed for the current task is spending against this deficit.

3  A Direct Test: CTIM-Rover

3.1  Setup

Lindenbauer, Groh and Schütze [6] built CTIM-Rover to test whether ExpeL-style experiential learning scales to real software engineering. The construction is close to the proposal under examination here:

• Corpus. 236 past successful trajectories — 183 instances solved by a Reflexion-augmented AutoCodeRover on a 401-sample SWE-bench Verified training split, plus 53 successful AutoCodeRover trajectories from SWE-bench Lite.
• Distillation. A reasoning model (o1) extracted a general Cross-Task-Instance Memory (CTIM) from sets of successful trajectories and from success/failure tuples, then a second phase extracted repository-level knowledge conditioned on the general set. Memory size was capped at c(n) = ⌈√n⌉.
• Delivery. The full CTIM was supplied in the user prompt at the start of the trajectory — not a relevance-selected subset, and not per-turn. The authors state this explicitly as a departure from concurrent work.
• Exemplar arm. Separately, the most task-similar past successful trajectory was retrieved (CodeT5-base embeddings over problem statements, Milvus index, cosine threshold 0.90; an exemplar was available for ≈62% of samples).
• Evaluation. 45 held-out SWE-bench Verified samples, stratified on the annotation features statistically associated with solvability. Agent model GPT-4o at both training and evaluation.

This is a strong-form version of the hypothesis: high-quality successes only, distilled by a reasoning model, in-domain, with a repository-level specialization designed for exactly the transfer the hypothesis predicts.

3.2  Result: uniform degradation

No configuration beats the baseline. The full system loses 2 points; the arms that isolate the distilled memory lose 11. The authors' conclusion is unambiguous: "neither ExpeL nor DoT-Bank scale to real-world SE problems," with "noise introduced by distracting CTIM items or exemplar trajectories as the likely source of the performance degradation."

One detail deserves emphasis because it contradicts the most natural rescue of the hypothesis. The training corpus was heavily skewed toward Django, and the repository-level memory was designed precisely so that Django instances would benefit. Django performance fell under both isolated-memory arms (50 → 36, 50 → 41). More data about the exact repository under test made things worse, not better. The failure is not a coverage problem.

3.3  What it cost

The full system paid a 77.6% median context increase to lose two points. It is worth stating the implication for the proposal under examination: a preamble of past achievements is strictly a cost term unless it changes behavior favorably, and here the highest-fidelity, most carefully distilled version of that content changed behavior unfavorably.

3.4  Mechanism: lexical, not semantic

The two qualitative probes in [6] are, for our purposes, worth more than the aggregate table, because they identify the channel through which injected experience acts.

Probe 1 — django__django-13933. The baseline correctly patched to_python(). CTIM-Rover with repository-level memory instead pursued clean(), which the search API also returned, and failed. Inspecting the memory, the authors found an item about URL prefix handling — substantively unrelated to the bug — that happened to contain the word clean:

"[…] Ensure to separate resolution from the final redirect to keep path_info clean while preserving the prefix in the final URL, preventing forced […]"

Removing that single item and re-running produced a correct solve. A lexical coincidence in one memory item, at a distance from the task, redirected the agent's localization.

Probe 2 — django__django-15987. The problem statement named the constant FIXTURE_DIRS. The baseline searched for it; the memory-equipped agent did not. The authors observed that their memory contained no capitalized tokens and hypothesized a bias toward lower snake-case identifiers. They then added the item GRANDMA LIKES PASTA — arbitrary, semantically empty, and merely capitalized — to the memory. The agent solved the instance.

This is the finding that settles the question posed in §1. A string carrying no information about software engineering repaired a software-engineering failure, because it altered the typographic distribution of the context. The mechanism by which injected experience acts on an agent is next-token conditioning on surface statistics. It is not the induction of confidence, standard-setting, or self-efficacy; those are folk-psychological descriptions with no measured referent here.

The consequence for achievement-priming is direct. A preamble reading "ARM7TDMI → WASM, 99.9% native coverage, 1,698 blocks, 805 assertions" does not install rigor. It installs a prior over tokens: architecture mnemonics, coverage vocabulary, block/assertion counting. On a recompiler task that prior is aligned and probably helpful. On a database migration, a market analysis, or a proof, it is clean — a well-formed, confident, entirely irrelevant attractor.

3.5  The decisiveness inversion

Table 2 contains a result the original paper does not foreground, and it is the most transferable thing in the study.

The stated purpose of the memory was to make the agent explore less: the authors chose software engineering because they expected experiential learning to be "particularly valuable for uncovering the structure of a repository, reducing the number of turns taken exploring the codebase." The intervention achieved exactly that. Median turns fell from 9 to 7 (−22.2%); mean turns from 11.16 to 9.62 (−13.8%). The agent became more decisive.

Accuracy fell with it. The exploration that the memory eliminated was load-bearing.

This is the failure mode that achievement-priming should be expected to produce, and it is more dangerous than simple noise, because it is invisible in every cheap metric. An agent primed on a record of triumphs is an agent that has been shown, repeatedly, that its guesses were correct. The observable consequence is fewer verification steps, earlier commitment, and shorter trajectories — which read as efficiency on any dashboard that is not also measuring correctness. A memory system that optimizes for step reduction will select for this pathology.

4  The Positive Control: Workflow Induction

The negative result above is not a result about memory. It is a result about a particular form of memory. AWM [4] takes the same input — past successful trajectories — and produces the opposite outcome.

AWM improves the strongest autonomous baseline by 12.0 absolute points (+51.1% relative) and uses about 2.0 fewer steps per task than that baseline, and 40.8 fewer than AutoEval. On Mind2Web it improves cross-task step-wise success by 24.6% relative, and on the cross-website and cross-domain splits it exceeds baselines by 8.9–14.0 absolute points — with the margin widening as the train/test distribution gap grows. That last property is the signature of genuine abstraction: a memory of specifics would degrade as the distribution shifted.

The cleanest single comparison in the literature for our purposes is AWM against Synapse [4, §3.2.1]. Synapse retrieves the most relevant concrete training examples; AWM substitutes induced workflows in the same slot, under the same model and benchmark. AWM gains +5.0 element accuracy and +4.0 step success rate. Holding retrieval, model and task fixed, and varying only abstracted procedure versus concrete past experience, abstraction wins.

Note also that AWM reduces steps while raising accuracy, whereas CTIM-Rover reduced steps while lowering it (§3.5). Step reduction is not itself the signal; it is only benign when it comes from possessing the right routine rather than from skipping verification.

5  Four Axes of Experiential Transfer

The two results are reconcilable. Sorting the interventions along four axes predicts the sign of the effect without running the experiment.

Axis 1 — abstraction. ExpeL and CTIM-Rover store distilled rules and raw exemplars; AWM stores induced procedures. AWM beats exemplar retrieval head-to-head by +4.0 step SR under a fixed model (§4).

Axis 2 — retrieval gating. CTIM-Rover supplied the entire memory in the opening user prompt. AWM supplies workflows selectively. CTIM-Rover's own authors name this as their primary suspected defect and point at per-turn retrieval [7] as the remedy.

Axis 3 — task analogy. This is the axis on which achievement-priming fails hardest, because a trophy list is by construction a cross-domain sample: the whole appeal is that it spans a recompiler, a proof campaign, a Kaggle arena and a music video. Every item is off-distribution for every task except one.

Axis 4 — trajectory length. ExpeL succeeded on HotpotQA, WebShop and ALFWorld. CTIM-Rover's mean context is roughly 4× ExpeL's on HotpotQA, and it failed. Benefit erodes as horizon grows. Long-horizon multi-session engineering — the exact case the proposal is aimed at — is the least favorable regime.

6  An In-Situ Natural Experiment

The preceding sections are external evidence. A deployed AGNT instance provides an internal one, produced without prompting and without knowledge of this question.

AGNT's InsightEngine extracts structured insights from execution traces and classifies them, routing by blast radius and confidence. On 2026-06-27 it emitted two records, 27 minutes and 31 seconds apart, describing the same tool sequence: activation of the retro-rom-to-wasm-recompiler skill, recall queries for GBA and NES WASM recompilation, web_search for OpenRCT2, and file reads under gba-recomp and nes-recomp.

Record B's stored avoidance rule reads:

"Avoid mixing unrelated recalled memories, skills, or web research into a focused project execution trace."

Its description names the harm precisely: the unrelated recompilation material made it "harder to validate the PTCG-specific execution path."

Identical actions; opposite verdicts; the only varying factor is whether the recalled material was analogous to the task. That is Axis 3 of Figure 3, derived independently and locally.

What this is not. Both verdicts were produced by an LLM-based judge over traces, not by measuring task outcome. This is evidence that a second system, examining real traces, converges on the same discriminator — not evidence that the contaminated run scored worse. It is hypothesis-consistent, not confirmatory, and is reported at that strength.

The corresponding positive case is already deployed. Insight 6ce90af2 (2026-07-01, confidence 0.95, blast radius 0.10, routed direct under low_blast_high_confidence, status applied 2026-07-20) records the creation of the instrument-first-engineering skill: a 15,502-byte document distilling two recompiler projects into ground-truth recording, oracle speed gates, differential verification, deviation discipline and scoreboards, auto-activating on binary-lifting work. That artifact is an AWM-style induced workflow in everything but name, and it means the correct intervention is not hypothetical in this system — it is running. Replacing it with an achievement preamble would be a regression along Axes 1, 2 and 3 simultaneously.

7  What an Achievement List Actually Encodes

Decomposing a representative list clarifies why the intuition feels right. Some entries do carry transferable content; they are simply not the parts that read as impressive.

Six of ten entries are pure outcome scalars: they assert that something went well without constraining any future decision. Under §3.4 they are not inert — they are lexical mass biasing the model toward the vocabulary of whichever project they describe. The four entries with real transfer value are all procedural or evaluative: a gate, an assertion budget, a negative control, a tiered verification structure, a stopping rule. None of them require the surrounding trophy to function, and all of them are more usefully stored as a skill than as a memory of an event.

8  The Failure Asymmetry

The highest-transfer entry in Table 5 is the failed project.

This is not a moral observation; it follows from the construction of every system reviewed. ExpeL's distillation operates over success/failure contrast pairs as well as success sets. CTIM-Rover explicitly expected its highest-quality items from success-failure tuples, on the grounds that such pairs "already contain the insights that lead to an eventual resolution." A success shows that one path terminated well; it does not identify which decision was load-bearing, because nothing counterfactual was observed. A contrast pair localizes the causal edge.

The AGNT instance's own artifact demonstrates this at production scale. The instrument-first-engineering skill opens by naming a failure as its reason for existing:

"This skill exists because the same project was attempted twice. The first attempt (rct-x86-wasm-lift) burned 17 hours, 948 tool calls, and 135M tokens acting as a human-style debugger — reading frozen logs into context and guessing — and never reached its first real milestone."

The document's operative rules are derived from that failure's specific mechanics: a reference interpreter running at 29K instructions/second against a target needing 50–200M instructions per frame, producing 136 consecutive debugging iterations that were all snapshots of a process frozen mid-boot, and leaving 62 orphan scripts behind. From this it derives a hard numeric gate — set a performance floor on the oracle before deep debugging begins — and the effort-split targets that define the method (roughly 42% model / 58% script by effort, with ~90% of correctness claims originating from scripts rather than model judgment).

The successful project contributed the confirmation that the method works. The failed project contributed the method. An agent memory ranked by outcome quality inverts this and discards its most valuable material.

9  Design Rules for an Agent Memory System

The following are stated as implementable constraints rather than principles.

R1 — Store procedures; reject scalars. A memory whose content is an achievement number should fail admission. Enforceable as a write-time classifier: an item must contain at least one imperative constraining a future decision. "Build the oracle before the lifter and gate each phase on a differential verify" is admissible; "reached 99.9% coverage" is not.

R2 — Gate retrieval on task analogy, not recency or salience. Recency and importance ranking are the wrong retrieval keys; Table 4 shows the discriminator is structural match between recalled material and current task. Compute a domain fingerprint for the task and require a threshold match. Below threshold, inject nothing; empty is a valid and often optimal memory state.

R3 — Move from per-task to per-turn injection. The single change most strongly indicated by both [6] and [7]. AGNT's skill system currently activates once, at task start (Figure 3, Axis 2, neutral pole). Re-scoring a small candidate set against the live turn context and injecting only what clears threshold moves it to the improving pole. This is the highest-expected-value change available.

R4 — Budget-cap injected experience and account for it explicitly. CTIM-Rover spent +77.6% median context to lose 2 points. Cap experiential injection (a few hundred tokens at plan time is a reasonable starting budget), log the cost per task, and treat it as a term to be justified rather than a free improvement.

R5 — Mark injected priors revocable. Every injected item should carry an explicit defeasibility clause: prior from an analogous problem; discard on contradiction with observed evidence. The clean failure is precisely an undefeated prior surviving contact with a better local signal.

R6 — Prefer contrast pairs to successes. Where a task has both a failed and a successful attempt, distill from the pair. Where a skill exists, require it to carry a "this method failed when…" clause. A memory system that stores only wins is structurally unable to represent its most informative observations (§8).

R7 — Audit surface form, not just semantics. GRANDMA LIKES PASTA means the lexical composition of a memory set is a live variable. Practical checks: does the set contain identifier-shaped tokens, capitalized constants, and the notational registers the target domain uses? An item may be semantically correct and still harmful through vocabulary alone.

R8 — Instrument the null hypothesis permanently. Memory systems are assumed beneficial and rarely ablated. Retain a memory-off arm in evaluation so that a regression like Figure 1 is detectable in production rather than after a year of accumulated context. Monitor step-count reduction and success rate jointly; a fall in steps without a rise in success is the §3.5 pathology, not an efficiency win.

10  Falsifiable Predictions

P1 — Achievement preambles do not improve, and degrade with length. Injecting a list of past accomplishments before a structurally unrelated task yields Δsuccess ≤ 0, with the magnitude of degradation increasing in preamble length. Falsifier: a statistically significant positive Δ under a sequential test at α = 0.05.

P2 — The bias channel is lexical and measurable directly. A preamble dense in domain-X identifiers will raise the rate of X-flavored tool calls and search terms on domain-Y tasks, independently of any success-rate change. This is cheaper to measure than P1 and does not require a task suite — only tool-call logging. Falsifier: no shift in the tool-call distribution.

P3 — The persistence hypothesis (untested). This is the form of the original intuition that survives §3–§5 and that no work surveyed here measures. Every benchmark cited scores final success; none score premature abandonment. It is therefore possible that identity- or standard-priming improves long-horizon persistence — refusal to declare victory at 60% — while leaving per-task accuracy unchanged or slightly worse. Notably, the item best suited to producing that effect is the failure record (§8), not the wins. Falsifier: no difference in premature-termination rate between primed and unprimed arms on tasks with a known completion criterion.

Proposed experiment. P1 and P2 are directly testable on an existing sequential-testing gauntlet: two agent configurations, identical model, tools and seeds, differing only in the presence of an achievement preamble, run to statistical significance on a task suite structurally unrelated to the preamble's contents. P2 requires only tool-call distribution logging and can run as a side-channel on ordinary traffic. Benchmarks already at ceiling are unsuitable — a saturated score cannot exhibit a negative effect.

11  Limitations

• Single direct study. §3 rests on one paper with n = 45, a single agent model (GPT-4o), and no reported variance or confidence intervals. The 2-point gap between 42 and 40 is not distinguishable from noise at that sample size; the 11-point gaps are more robust but still single-run. The strength of the evidence lies in the uniformity of the direction across six configurations and in the two mechanism probes, not in any individual delta.
• Implementation-specific failure. The authors themselves propose embedding-based per-turn retrieval as an untested remedy. The negative result therefore bounds ungated wholesale injection, not episodic memory in general — which is exactly the distinction §5 formalizes.
• Domain transfer of the positive control. AWM is evaluated on web navigation. Its superiority over exemplar retrieval is measured there and assumed, not shown, to hold for software engineering.
• The in-situ experiment is n = 2 and judge-mediated. As stated in §6, both verdicts come from an LLM classifier over traces, not from measured task outcomes. No causal claim is made.
• Achievement preambles are not literally tested anywhere. The closest published construction (a distilled memory of 236 successes) is more favorable to the hypothesis than the proposal itself — better curated, in-domain, reasoning-model distilled. The inference from its failure to the failure of a rawer, cross-domain version is an extrapolation, though one that all four axes of §5 push in the same direction.
• P3 remains open. Nothing here refutes the persistence hypothesis; the literature simply does not measure it.

12  Conclusion

Context is a prior, not a trophy case. Loading an agent with a record of its own past victories does not install the disposition that produced them; it installs a distribution over tokens, and on any task not structurally analogous to those victories that distribution is a distractor. The strongest available test of this — 236 curated successes, in-domain, distilled by a reasoning model — degraded every configuration it was applied to while inflating context by 77.6%, and its own authors' mechanism probes show the channel to be surface-lexical rather than semantic: a stray occurrence of clean broke a solve, and an arbitrary capitalized string repaired one.

The same raw material, abstracted into reusable procedures and delivered selectively, produces the opposite result: +51.1% relative success with fewer steps, generalizing better as the distribution gap widens, and beating retrieval of concrete past experience head-to-head. The difference is not the presence of memory but its form — and the four axes of §5 predict the sign in advance.

Two consequences follow for anyone building agent memory. First, the unit of storage should be the method, not the outcome: an agent should carry build the oracle before the lifter and gate every phase on a differential verify, never you achieved 99.9% coverage. Second, and less intuitively, the failures should be carried hardest of all. A success shows that a path terminated well. Only a failure, paired against a success, identifies which decision was load-bearing — which is why in the corpus examined here, the project that burned 17 hours and never reached its first milestone is the one that generated a reusable methodology, and the projects that succeeded mostly generated numbers.

References

1. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, Y. Cao. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023. arXiv:2210.03629.
2. N. Shinn, F. Cassano, E. Berman, A. Gopinath, K. Narasimhan, S. Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS 2023. arXiv:2303.11366.
3. A. Zhao, D. Huang, Q. Xu, M. Lin, Y.-J. Liu, G. Huang. ExpeL: LLM Agents Are Experiential Learners. AAAI 2024, 38(17):19632–19642. arXiv:2308.10144.
4. Z. Z. Wang, J. Mao, D. Fried, G. Neubig. Agent Workflow Memory. ICML 2025, PMLR 267:63897–63911. arXiv:2409.07429. WebArena 23.5 → 35.5 overall SR; avg. steps 7.9 → 5.9; Mind2Web cross-task +24.6% relative step SR; cross-website/domain +8.9–14.0 absolute; +4.0 step SR and +5.0 element accuracy over Synapse exemplar retrieval.
5. F. Shi, X. Chen, K. Misra, N. Scales, D. Dohan, E. Chi, N. Schärli, D. Zhou. Large Language Models Can Be Easily Distracted by Irrelevant Context. ICML 2023. arXiv:2302.00093. Introduces GSM-IC; accuracy "dramatically decreased" by irrelevant context; partial mitigation via self-consistency and explicit ignore-instructions.
6. T. Lindenbauer, G. Groh, H. Schütze. From Knowledge to Noise: CTIM-Rover and the Pitfalls of Episodic Memory in Software Engineering Agents. REALM @ ACL 2025. arXiv:2505.23422. Code: github.com/Liqs-v2/ctim-rover. All figures in Tables 1–2 and Figures 1–2 of this report are transcribed from its Tables 1 and 3.
7. H. Su, R. Sun, J. Yoon, P. Yin, T. Yu, S. Ö. Arık. Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments. 2025. arXiv:2501.10893. Retrieves synthetic sub-trajectories at each agent step; cited by [6] as the indicated remedy for its own negative result.
8. Y. Zhang, H. Ruan, Z. Fan, A. Roychoudhury. AutoCodeRover: Autonomous Program Improvement. ISSTA 2024. arXiv:2404.05427. The baseline system in [6].
9. N. Chowdhury et al. Introducing SWE-bench Verified. OpenAI, 2024. The evaluation set used by [6].
10. AGNT Labs. InsightEngine records d9a5a00f, 7e949534, 6ce90af2, retrieved from a production instance 2026-07-25; and the instrument-first-engineering skill document (15,502 bytes, created 2026-07-01, applied 2026-07-20). Records reproduced in the artifact set.