Annotations as described in Müller et al. (2020)

The gaze annotations were collected for videos from the youtube channels "TheSpaDr" and "WisdomFromNorth". The naming scheme of the files is <youtube_id>.csv. Each file contains three columns: - index indicating the annotated frame
- 'gaze_left': gaze annotations for the left person
- 'gaze_right': gaze annotations for the person on the right

Explanation of annotation values:
'm': eye contact
'b': blink
'r': gaze to the right
'l': gaze to the left
'o': gaze upwards
'u': gaze downwards
'or': gaze to upper right
'ol': gaze to upper left
'ul': gaze to lower left
'ur': gaze to lower right
'na': annotation was not possible (i.e. because of strong motion blur)
'NaN': no annotation (occurs because the frames annotated for left and right person differ)

Download (140k)

In case of questions, please do not hesitate to contact Philipp Müller ().

License: CC BY 4.0

If you use this dataset in a scientific publication, please cite the following paper:

1. 

   Anticipating Averted Gaze in Dyadic Interactions

   Philipp Müller, Ekta Sood, Andreas Bulling

   Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA), pp. 1-10, 2020.

   Abstract Links BibTeX Project

   We present the first method to anticipate averted gaze in natural dyadic interactions. The task of anticipating averted gaze, i.e. that a person will not make eye contact in the near future, remains unsolved despite its importance for human social encounters as well as a number of applications, including human-robot interaction or conversational agents. Our multimodal method is based on a long short-term memory (LSTM) network that analyses non-verbal facial cues and speaking behaviour. We empirically evaluate our method for different future time horizons on a novel dataset of 121 YouTube videos of dyadic video conferences (74 hours in total). We investigate person-specific and person-independent performance and demonstrate that our method clearly outperforms baselines in both settings. As such, our work sheds light on the tight interplay between eye contact and other non-verbal signals and underlines the potential of computational modelling and anticipation of averted gaze for interactive applications.

   doi: 10.1145/3379155.3391332

   Paper: mueller20_etra.pdf

   @inproceedings{mueller20_etra, title = {Anticipating Averted Gaze in Dyadic Interactions}, author = {Müller, Philipp and Sood, Ekta and Bulling, Andreas}, year = {2020}, booktitle = {Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA)}, doi = {10.1145/3379155.3391332}, pages = {1-10} }