Comparative Reasoning: Making an Audio Language Model Better at Comparing Emotions

Abstract

Large audio-language models (LALMs) can reason about audio, yet it remains unclear whether they can perform comparative judgments between two speech signals along emotional, environmental, linguistic, prosodic, and interpersonal dimensions. We study this question in the context of speech emotion recognition (SER), where the model determines which utterance exhibits higher arousal, valence, or dominance.

We introduce a reasoning-guided ordinal SER framework that conditions an LALM on paired speech inputs. The model is trained using reasoning traces generated from both semantic audio descriptions and acoustic evidence derived from GeMAPS features, enabling interpretable comparative decisions. Beyond direct supervision, we also employ direct preference optimization to encourage stronger separation for emotional differences.

Experiments show that the proposed framework improves preference prediction while requiring only 5% of the training data used by conventional ordinal SER systems.

Method Overview

Given two utterances, the model answers which one has the higher attribute value. We ground its reasoning in what can be heard and measured in the two clips.

1. Evidence from both clips. Qwen3-Omni-Captioner describes each clip (content, speaker, delivery), and 36 GeMAPS acoustic features (pitch, loudness, voice quality, spectral balance, ...) are discretized into low / medium / high.
2. Comparative reasoning traces. Given the evidence and an attribute-specific guideline, Qwen3-Next-80B-A3B-Thinking writes a short Clip 1 → Clip 2 → comparison trace. Only traces whose answer matches the label are kept (r⁺); traces that argue for the wrong clip (r⁻) are generated as rejected responses.
3. Training the audio LM. Qwen2.5-Omni-3B is fine-tuned with LoRA on 10k pairs per attribute, using SFT or DPO, with the answer only or with the reasoning trace. DPO-CoT prefers r⁺ + correct answer over r⁻ + wrong answer.

Results

Preference accuracy (%) on MSP-Podcast test pairs and on two unseen corpora: BIIC-Podcast (Mandarin) and WHiSER. Fine-tuning on 10k pairs per attribute outperforms SSL-based ranking models trained on 240k pairs per attribute.

Dataset Examples

MSP-Podcast training pairs with the evidence given to the reasoning model, the verified reasoning trace r⁺ used by SFT-CoT and DPO-CoT, and the wrong trace r⁻ used as the rejected response in DPO-CoT.

Model Reasoning

Responses of the DPO-CoT model on held-out pairs, including unseen conditions: a different language (Mandarin BIIC-Podcast) and an attribute it was not trained on.

BibTeX

If you find this work useful, please cite:

@inproceedings{naini2026comparative,
  title     = {Comparative Reasoning: Making an Audio Language Model Better at Comparing Emotions},
  author    = {Naini, Abinay Reddy and Kim, Jaeyeon and Yang, Chao-Han Huck and Watanabe, Shinji and Busso, Carlos},
  booktitle = {Interspeech},
  year      = {2026}
}