AgentDiff: Meaning-Bearing Rewrites Trigger Deeper Divergence than Presentation Changes in LLM Agents

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2605.25981v3 Announce Type: replace Abstract: LLM agents should respond to what an input means, not how it is presented. We show that they do not treat these two kinds of variation equally. AgentDiff measures the difference between answer changes caused by meaning-bearing rewrites (paraphrase and synonym substitution) and presentation changes (reordering, formatting, and distractors), while matching perturbation severity. Across 68 model--benchmark--scaffold cells spanning ten LLMs from seven architecture families, three benchmarks, 1{,}530 original questions, and approximately 11{,}150 variants, meaning-bearing rewrites produce a $+19.69$ percentage-point higher inconsistency rate (paired $t=9.58$, $p<0.0001$; 64/68 cells positive). The result is stable under four severity proxies ($+18.9$ to $+20.9$~pp, all $p<0.0001$) and remains $+11.10$~pp on the 48 cells outside the qwen family. A fully held-out qwen2.5-14B-Instruct evaluation then tests the structure of this effect on 1{,}800 new trajectories: the pre-registered capable-and-tractable partition is positive in 3/4 held-out cells and remains sharply separated after pooling (Welch $t=3.81$, $p=9.6\!\times\!10^{-4}$). Trace analysis explains how the difference propagates. Meaning-bearing rewrites preserve the first action but reduce thought similarity from step 2 onward by $5.6$--$10.5$ points and extend the resulting cascade by $0.17$ steps (paired $t=7.69$, $p=2.5\!\times\!10^{-14}$), a pattern we call \emph{stealth divergence}. AgentDiff therefore establishes a reproducible directional robustness gap, …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2605.25981
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.