Expert-Guided Multimodal Fusion for Unified Emotion and Sentiment Analysis

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

arXiv:2601.07565v2 Announce Type: replace Abstract: Multimodal emotion understanding requires the integration of heterogeneous data sources, including text, audio, and visual modalities, while simultaneously addressing discrete emotion recognition and continuous sentiment analysis. We propose EGMF, a unified framework that combines expert-guided multimodal fusion with large language models to achieve superior performance across both tasks. At the core of our framework is a multi-scale expert network, comprising a local expert for capturing subtle emotional nuances, a semantic correlation expert for modeling cross-modal relationships, and a global context expert for understanding long-range dependencies. These experts are adaptively integrated via hierarchical dynamic gating, enabling context-aware feature selection and modality weighting. The enhanced multimodal representations are seamlessly incorporated into the language model through pseudo token injection and prompt-based conditioning, allowing a single generative framework to handle both classification and regression tasks. We employ parameter-efficient LoRA fine-tuning to maintain computational efficiency. Extensive experiments on bilingual benchmark datasets (MELD, CHERMA, MOSEI, SIMS-V2) demonstrate that EGMF outperforms state-of-the-art methods in terms of accuracy, cross-lingual robustness, and the discovery of universal patterns in multimodal emotional expressions.

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://arxiv.org/abs/2601.07565
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.