KrishokChat: A Provenance-Traceable Multi-Task Bengali Agricultural Benchmark with Safety-Critical Chemical Advisory

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2606.29243v2 Announce Type: replace Abstract: We introduce KrishokChat, an 85,979-instance Bengali agricultural benchmark built from 284 government publications, 13 institutions, and six regional dialects. The benchmark comprises four tracks: General Knowledge QA, Treatment QA, Safety Refusal and Re-query, and Table QA. It also includes a 1,000-query Real-World Farmer Benchmark collected independently from field interviews to measure transfer to authentic farmer language. Every extracted instance retains provenance at the citation level. Treatment QA also provides a structured chemical-trace array for dosage-level auditability. We evaluated five zero-shot baselines and one fine-tuned model. Closed-book knowledge proves insufficient regardless of model scale. Oracle evidence narrows the gap, but leaves a persistent floor of 4.05 to 7.00% of chemical hallucinations. Fine-tuning on KrishokChat substantially outperforms the strongest zero-shot baseline on General QA Token F1. However, structured table reasoning and farmer-language transfer remain largely unsolved. An exploratory analysis further reveals that fine-tuning reduces refusal behavior on safety-critical queries, indicating that additional safety alignment is required beyond supervised fine-tuning. We release KrishokChat as a traceable benchmark and audit resource to support grounded and safety-aware agricultural language modeling for Bengali.

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2606.29243
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.