The Concept Allocation Zone: Tracking How Concepts Form Across Transformer Depth

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2605.24856v2 Announce Type: replace-cross Abstract: Concept formation in transformer language models is a depth-extended process, not a single-layer event: a concept becomes separable across one or more contiguous regions of the residual stream - its Concept Allocation Zone (CAZ). A CAZ is not a concept but the depth segment where the model organizes its geometry to make one separable - concepts may share a CAZ, and typically span multiple across depth; the companion GEM paper shows the separating direction continues to rotate within a CAZ before stabilizing past its boundary. We formalize the CAZ through three layer-wise metrics - Separation, Concept Coherence, and Concept Velocity - with automated boundary detection that applies no significance threshold to CAZ membership (every segment is a CAZ; "strong" vs. "gentle" is score, never a binary cut). Empirical validation across 35 models, 8 architectural families, and 7 concepts shows the separation curve S(l) is frequently multimodal, and scored detection surfaces a further category of subtle allocation regions ("gentle CAZes") invisible to standard peak detection. The framework generates seven testable predictions; its contribution is the instrument and the phenomena it surfaces - the scored detector, the three metrics, and the multimodal/gentle-CAZ findings - not the predictions themselves. Released as the open-source rosetta_tools library (v1.3.1).

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2605.24856
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.