MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2605.22949v3 Announce Type: replace Abstract: Foundation-model pools are increasingly used as black-box responders in coordinated systems where a coordinator must decide which response to trust. Raw self-reported confidence is the natural signal, but is not comparable across models and becomes stale under distribution shift when corrected only at design time. We study runtime confidence calibration for multi-model coordination, where per-model corrections are learned online from deployment outcomes with no model access, no held-out calibration data, and no retraining. Across 18 open-weight foundation models, 8 benchmarks, and over 44,000 observations, we find that online adaptation is a family property: simple same-information online calibrators close most of the calibration gap left by frozen design-time methods under shift, and the forgetting schedule is the dominant design axis. We present MARGIN (Multi-Agent Runtime Grading via Incremental Normalisation), a structured member of this family that maintains per-model, per-confidence-band multiplicative factors using symmetric exponentially weighted updates and shrinkage blending. MARGIN does not dominate the online family on expected calibration error (ECE) under abrupt shift. Its value lies in interpretable confidence-band trust factors, defined cold-start and returning-model behaviour, dynamic-pool support, and a scoped symmetric-update guarantee for fixed-policy non-strategic agents. Empirically, raw verbalized confidence is a weak or misleading pairwise selection signal on hard code-generation tasks, while …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2605.22949
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.