Engagement Estimation Leaderboard
This leaderboard comprises the results from both the MultiMediate'23 engagement challenge and the multi-domain engagement challenge - MultiMediate'24. The combined score is the combined concordance correlation coefficient (CCC) test score for NoXi (base), NoXi (additional languages), and MPIIGroupInteraction (MPII-GI). The 2023 challenge was only evaluated on NoXi, therefore, for comparison please sort by NoXi (base).
| Username and affiliation | Publication | Code | NoXi (base) | NoXi (add. languages) | MPII-GI | Combined score | Date tested | Challenge |
|---|---|---|---|---|---|---|---|---|
| Baseline 2024 | MultiMediate ’24: Multi-Domain Engagement Estimation, ACM Multimedia 2024 | 0.64 | 0.51 | 0.09 | 0.41 | 2024 | ||
| USTC-IAT-United | 0.72 | 0.73 | 0.59 | 0.68 | 12.07.2024 | 2024 | ||
| AI-lab | 0.69 | 0.72 | 0.54 | 0.65 | 12.07.2024 | 2024 | ||
| Li et al. (Hefei University of Technology, China) | DAT: Dialogue-Aware Transformer with Modality-Group Fusion for Human Engagement Estimation., ACM Multimedia 2024 | 0.76 | 0.67 | 0.49 | 0.64 | 12.07.2024 | 2024 | |
| Kumar et al. (IIT Roorkee, Uttarakhand, INDIA) | Towards Engagement Prediction: A Cross-Modality DualPipeline Approach using Visual and Audio Features, ACM Multimedia 2024 | 0.72 | 0.69 | 0.50 | 0.64 | 12.07.2024 | 2024 | |
| ashk | 0.72 | 0.69 | 0.42 | 0.61 | 12.07.2024 | 2024 | ||
| YKK | 0.68 | 0.66 | 0.40 | 0.58 | 12.07.2024 | 2024 | ||
| Xpace | 0.70 | 0.70 | 0.34 | 0.58 | 12.07.2024 | 2024 | ||
| nox | 0.68 | 0.70 | 0.31 | 0.56 | 12.07.2024 | 2024 | ||
| SP-team | 0.68 | 0.65 | 0.34 | 0.56 | 12.07.2024 | 2024 | ||
| YLYJ | 0.60 | 0.52 | 0.30 | 0.47 | 12.07.2024 | 2024 | ||
| Baseline 2023 | MultiMediate ’23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions, ACM Multimedia 2023 | 0.59 | 2023 | |||||
| He et al. (Australian National University, Australia) | TCA-NET: Triplet Concatenated-Attentional Network For Multimodal Engagement Estimation | 0.75 | ||||||
| Yu et al. (Hefei University of Technology, China) | Sliding Window Seq2seq Modeling for Engagement Estimation, ACM Multimedia 2023 | 0.71 | 14.07.2023 | 2023 | ||||
| Yang et al. (Hong Kong Polytechnic University, China) | MultiMediate 2023: Engagement Level Detection using Audio and Video Features, ACM Multimedia 2023 | 0.695 | 14.07.2023 | 2023 | ||||
| Tu et al. (Chonnam National University, South Korea) | DCTM: Dilated Convolutional Transformer Model for Multimodal Engagement Estimation in Conversation, ACM Multimedia 2023 | 0.66 | 14.07.2023 | 2023 |