Which Decisions Low-Bit Quantization Breaks, and How to Predict Them

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2608.06564v2 Announce Type: replace-cross Abstract: Quantization is how large language models are actually deployed, and below four bits it hurts. What nobody can say is which decisions change at a given bit-width -- which matters most where a model acts rather than answers, since a tool call it declines to make is a failure no score reports. A compressed agent stops calling its tools, then loses half its safety refusals, while benchmark scores barely move. Prior work assumes the added noise has a roughly fixed size, which would make confident decisions safe. We measure the decision instead: the margin between the option a model picks and its best alternative, before and after quantization, across 16 models, three methods, and 8 down to 2 bits. Kinds of decision do not break together -- at 3 bits the decision to call a tool collapses toward inaction while the choice of which tool is untouched -- and the damage is proportional rather than fixed, the margin multiplied by a factor that collapses with bit-width (median 0.86 at 4 bits, 0.33 at 3, 0.00 at 2). Fitted against additive competitors, including one whose noise grows with the margin, no account with an additive mean wins a damaged tool or safety cell; that is the best description among those stated, not a proof of generative form. Given a condition's own constants the relation predicts flip rates on its held-out decisions to a median of 1.8 points, with calibrated per-decision probabilities (calibration error 0.004 over 131,758 predictions), and no flip was used in the fit. Borrowed constants are wrong by 18-33 …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2608.06564
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.