Eye Movement Analysis for Activity Recognition Using Electrooculography

Abstract

This dataset was recorded to investigate the problem of recognising common office activities from eye movements. The experimental scenario involved five office-based activities – copying a text, reading a printed paper, taking handwritten notes, watching a video, and browsing the Web – and periods during which participants took a rest (the NULL class).

Download (20.9 Mb)

Contact: Andreas Bulling,

The data is only to be used for non-commercial scientific purposes. If you use this dataset in a scientific publication, please cite the following paper:

1. 

   Eye Movement Analysis for Activity Recognition Using Electrooculography

   Andreas Bulling, Jamie A. Ward, Hans Gellersen, Gerhard Tröster

   IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(4), pp. 741-753, 2011.

   Abstract Links BibTeX Project

   In this work we investigate eye movement analysis as a new sensing modality for activity recognition. Eye movement data was recorded using an electrooculography (EOG) system. We first describe and evaluate algorithms for detecting three eye movement characteristics from EOG signals - saccades, fixations, and blinks - and propose a method for assessing repetitive patterns of eye movements. We then devise 90 different features based on these characteristics and select a subset of them using minimum redundancy maximum relevance feature selection (mRMR). We validate the method using an eight participant study in an office environment using an example set of five activity classes: copying a text, reading a printed paper, taking hand-written notes, watching a video, and browsing the web. We also include periods with no specific activity (the NULL class). Using a support vector machine (SVM) classifier and a person-independent (leave-one-out) training scheme, we obtain an average precision of 76.1% and recall of 70.5% over all classes and participants. The work demonstrates the promise of eye-based activity recognition (EAR) and opens up discussion on the wider applicability of EAR to other activities that are difficult, or even impossible, to detect using common sensing modalities.

   doi: 10.1109/TPAMI.2010.86

   Paper: bulling11_pami.pdf

   @article{bulling11_pami, author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans and Tr{\"{o}}ster, Gerhard}, keywords = {Feature evaluation and selection, signal processing, Ubiquitous computing}, title = {Eye {M}ovement {A}nalysis for {A}ctivity {R}ecognition {U}sing {E}lectrooculography}, journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)}, volume = {33}, number = {4}, year = {2011}, pages = {741-753}, doi = {10.1109/TPAMI.2010.86} }

1. 

   Eye Movement Analysis for Activity Recognition

   Andreas Bulling, Jamie A. Ward, Hans Gellersen, Gerhard Tröster

   Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), pp. 41-50, 2009.

   Abstract Links BibTeX Project

   In this work we investigate eye movement analysis as a new modality for recognising human activity. We devise 90 different features based on the main eye movement characteristics: saccades, fixations and blinks. The features are derived from eye movement data recorded using a wearable electrooculographic (EOG) system. We describe a recognition methodology that combines minimum redundancy maximum relevance feature selection (mRMR) with a support vector machine (SVM) classifier. We validate the method in an eight participant study in an office environment using five activity classes: copying a text, reading a printed paper, taking hand-written notes, watching a video and browsing the web. In addition, we include periods with no specific activity. Using a person-independent (leave-one-out) training scheme, we obtain an average precision of 76.1% and recall of 70.5% over all classes and participants. We discuss the most relevant features and show that eye movement analysis is a rich and thus promising modality for activity recognition.

   doi: 10.1145/1620545.1620552

   Paper: bulling09_ubicomp.pdf

   @inproceedings{bulling09_ubicomp, author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans and Tr{\"{o}}ster, Gerhard}, keywords = {Activity Recognition, Electrooculography (EOG), Eye Movement Analysis, Recognition of Office Activities}, title = {Eye Movement Analysis for Activity Recognition}, booktitle = {Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp)}, year = {2009}, pages = {41-50}, doi = {10.1145/1620545.1620552} }

The dataset has the following characteristics: