TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2603.00025v3 Announce Type: replace Abstract: Direct Preference Optimization (DPO) is effective for offline alignment but poorly matched to ontology-driven structured prediction, where preferred and rejected JSON often differ by only a few schema-defining tokens. In this low-edit-distance regime, sequence-level DPO spreads gradient mass across non-critical serialization tokens (gradient dilution) and can reduce likelihood on rare preferred schema tokens (token erosion). To address these limitations, we first develop a confusion-aware preference-construction strategy combining expert-curated ambiguity patterns with validation-set SFT structured-error modes, producing minimally perturbed, schema-valid negatives for ontology-level decision errors. We then introduce Token-Adaptive Barrier Preference Optimization (TAB-PO), a post-SFT objective for token-critical structured generation with a confidence-gated token-level barrier that anchors under-confident schema tokens. On SciERC, with Llama/Qwen models, TAB-PO improves ontology-critical semantic-label and relational-linking metrics by 11.59% relative to SFT, wins 100% of comparisons against strongest token-level/sequence-level DPO variants, and surpasses strongest frontier baselines on these metrics by 14.71% relative while improving textual grounding.

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2603.00025
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.