MultiMediate: Multi-modal Group Behaviour Analysis for Artificial Mediation

Eye Contact Detection Sub-challenge[data available]

This sub-challenge focuses on eye contact detection in group interactions from ambient RGB cameras. We define eye contact as a discrete indication of whether a participant is looking at another participants’ face, and if so, who this other participant is. Video and audio recordings over a 10 second context window will be provided as input to provide temporal context for the classification decision. Eye contact has to be detected for the last frame of this context window, making the task formulation also applicable to an online prediction scenario as encountered by artificial mediators.

Group Discussion

Next Speaker Prediction Sub-challenge[data available]

In the next speaker prediction sub-challenge, approaches need to predict which members of the group will be speaking at a future point in time. Similar to the eye contact detection sub-challenge, video and audio recordings over a 10 second context window will be provided as input. Based on this information, approaches need to predict the speaking status of each participant at one second after the end of the context window.

Evaluation of Participants’ Approaches

For both sub-challenges, approaches will be evaluated on an unpublished test set. Details will be announced shortly.

To compare the submissions of the challenge in a fair and coherent way, we will evaluate all participating models with fixed samples on the non-public test partition of the dataset (for more information regarding the evaluation process see the link)

Rules for participation

* The competition is team-based. A single person can only be part of a single team.
* Each team will have 5 evaluation runs on the test set (per sub-challenge).
* Additional datasets can be used, but they need to be publicly available.
* The Organisers will not participate in the challenge.
* The top-performing teams will be invited to submit a paper, which will be evaluated according to both scientific novelty and the achieved results in the competition. Certificates will be awarded for the first 3 places in each sub-challenge.