MultiMediate: Multi-modal Group Behaviour Analysis for Artificial Mediation

Grand Challenge at ACM MM’21

teaser Artificial mediators are a promising approach to support group conversations but at present their abilities are limited by insufficient progress in group behaviour sensing and analysis. The MultiMediate challenge is designed to work towards the vision of effective artificial mediators by facilitating and measuring progress on key group behaviour sensing and analysis tasks. This year, the challenge will focus on eye contact detection and next speaker prediction. Eye contact detection is a foundational task in group behaviour analysis as eye contact is related to many important aspects of behaviour, including turn-taking, interpretation of emotional expressions, engagement, and dominance. The ability to predict who will speak next on the other hand will allow artificial mediators to engage in seamless turn-taking behaviour and even intervene proactively if a too dominant participant is about to take the speaking turn once again. For training and evaluation MultiMediate makes use of the MPIIGroupInteraction dataset consisting of 22 three- to four-person discussion as well as of an unpublished test set of six additional discussions. In the following years we will extend the MultiMediate challenge by incorporating increasingly complex tasks which will rely on advances made in earlier iterations of the challenge.