Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2605.12519v2 Announce Type: replace Abstract: Training language models to produce both correct answers and sound reasoning remains an open challenge. Reinforcement learning with verifiable rewards typically optimizes only final outcomes, which can improve task accuracy at the expense of reasoning quality, producing inaccurate, incomplete, or inconsistent traces. We propose verifiable process supervision (VPS), a post-training framework that jointly optimizes prediction accuracy and reasoning quality by supervising structured intermediate claims. We first apply supervised fine-tuning to induce a structured reasoning format, enabling deterministic extraction and verification of intermediate claims for process-level rewards. To address the heterogeneous difficulty of reasoning subtasks, we introduce adaptive weighting that prioritizes components with the largest remaining errors, creating an implicit curriculum. We evaluate VPS on chess as a controlled testbed where reasoning steps can be deterministically verified against engine signals. While outcome-only RL improves move accuracy, it sharply degrades reasoning quality, increasing win-rate error by up to 112% and reducing internal consistency by up to 69%. In contrast, VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation. A reasoning-space analysis further shows that, without a structured prior, outcome-only RL converges to budget-dependent shortcuts rather than sound multi-step reasoning. Beyond chess, we observe the …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2605.12519
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.