In-context superposition: human-like working memory interference in large language models

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2604.09670v2 Announce Type: replace-cross Abstract: Intelligent systems must maintain and manipulate task-relevant information online to adapt to dynamic environments and changing goals. This capacity, known as working memory, is fundamental to human reasoning and intelligence. Despite their radically different substrates, both biological and artificial systems exhibit limitations in working memory. This raises a key question: why do large language models (LLMs) show such limitations, given that transformers have full access to prior context and can retrieve relevant information through attention? We find that although a two-layer transformer can be trained to solve the N-back task perfectly, a diverse set of pretrained LLMs continues to show load-dependent working-memory limitations. LLMs also reproduce interference signatures observed in humans: retrieval is biased by recency and stimulus statistics, and stronger working-memory performance is associated with broader model capability, mirroring the established link between working memory and intelligence in humans. Mechanistically, LLMs represent multiple recent memory items in overlapping representations, which we call \emph{in-context superposition}. Across layers, models reorganize these representations to reduce interference and isolate the target for readout. Models retrieve the target more successfully when competing memories are better separated, and selectively suppressing interfering information can modestly improve performance. Together, these findings suggest that working-memory capacity depends on …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2604.09670
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.