Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mamba

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2503.18970v4 Announce Type: replace Abstract: Structured State Space Models (SSMs) have become a prominent class of sequence models, developed against two long-standing difficulties: the sequential computation and gradient propagation limits of Recurrent Neural Networks (RNNs), and the quadratic time and memory cost of self-attention in Transformers. By combining structured recurrence with state-space representations, SSMs attain linear or near-linear scaling in sequence length and hold a constant-size recurrent state during autoregressive decoding, so that no key-value cache grows with context. This paper is a structured review of the lineage running from the Structured State Space Sequence model (S4), through its diagonal and simplified successors S4D, Diagonal State Spaces (DSS) and S5, to the selective models Mamba and Mamba-2, and on to SSM-attention hybrids. Rather than describing models one at a time, the analysis is organized around cross-cutting design dimensions: input-dependent selectivity, the recurrent and convolutional views and when each is preferable, diagonalization, cache behavior at inference, hardware-aware kernels, and hybridization. Structured state-space duality is treated as a central thread, because it accounts for the equivalence between the recurrent and the attention-like form and for why hybrid designs work. Reported efficiency and accuracy results are presented with their measurement configuration and evidence level, because observed speedups reflect the combined effects of the algorithm, implementation kernel and hardware rather than …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2503.18970
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.