Robust Recognition of Reading Activity in Transit Using Wearable Electrooculography

Abstract

This dataset was recorded to investigate the problem of recognising reading activity from eye movements. The experimental setup was designed with two main objectives in mind: (1) to record eye movements in an unobtrusive manner in a mobile real-world setting, and (2) to evaluate how well reading can be recognised for persons in transit. We defined a scenario of travelling to and from work containing a semi-naturalistic set of reading activities. It involved subjects reading freely chosen text without pictures while engaged in a sequence of activities such as sitting at a desk, walking along a corridor, walking along a street, waiting at a tram stop and riding a tram.

Download (20.2 Mb)

Contact: Andreas Bulling,

The data is only to be used for non-commercial scientific purposes. If you use this dataset in a scientific publication, please cite the following paper:

1. 

   Multimodal Recognition of Reading Activity in Transit Using Body-Worn Sensors

   Andreas Bulling, Jamie A. Ward, Hans Gellersen

   ACM Transactions on Applied Perception (TAP), 9 (1), pp. 1–21, 2012.

   Abstract Links BibTeX Project

   Reading is one of the most well studied visual activities. Vision research traditionally focuses on understanding the perceptual and cognitive processes involved in reading. In this work we recognise reading activity by jointly analysing eye and head movements of people in an everyday environment. Eye movements are recorded using an electrooculography (EOG) system; body movements using body-worn inertial measurement units. We compare two approaches for continuous recognition of reading: String matching (STR) that explicitly models the characteristic horizontal saccades during reading, and a support vector machine (SVM) that relies on 90 eye movement features extracted from the eye movement data. We evaluate both methods in a study performed with eight participants reading while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. We introduce a method to segment reading activity by exploiting the sensorimotor coordination of eye and head movements during reading. Using person-independent training, we obtain an average precision for recognising reading of 88.9% (recall 72.3%) using STR and of 87.7% (recall 87.9%) using SVM over all participants. We show that the proposed segmentation scheme improves the performance of recognising reading events by more than 24%. Our work demonstrates that the joint analysis of multiple modalities is beneficial for reading recognition and opens up discussion on the wider applicability of this recognition approach to other visual and physical activities.

   doi: 10.1145/2134203.2134205

   Paper: bulling12_tap.pdf

   @article{bulling12_tap, author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans}, title = {Multimodal {R}ecognition of {R}eading {A}ctivity in {T}ransit {U}sing {B}ody-{W}orn {S}ensors}, journal = {ACM Transactions on Applied Perception (TAP)}, volume = {9}, number = {1}, year = {2012}, pages = {1--21}, doi = {10.1145/2134203.2134205} }

1. 

   Robust Recognition of Reading Activity in Transit Using Wearable Electrooculography

   Andreas Bulling, Jamie A. Ward, Hans Gellersen, Gerhard Tröster

   Proc. International Conference on Pervasive Computing (Pervasive), pp. 19-37, 2008.

   Abstract Links BibTeX Project

   In this work we analyse the eye movements of people in transit in an everyday environment using a wearable electrooculographic (EOG) system. We compare three approaches for continuous recognition of reading activities: a string matching algorithm which exploits typical characteristics of reading signals, such as saccades and fixations; and two variants of Hidden Markov Models (HMMs) - mixed Gaussian and discrete. The recognition algorithms are evaluated in an experiment performed with eight subjects reading freely chosen text without pictures while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. A total dataset of roughly 6 hours was collected with reading activity accounting for about half of the time. We were able to detect reading activities over all subjects with a top recognition rate of 80.2% (71.0% recall, 11.6% false positives) using string matching. We show that EOG is a potentially robust technique for reading recognition across a number of typical daily situations.

   doi: 10.1007/978-3-540-79576-6_2

   Paper: bulling08_pervasive.pdf

   @inproceedings{bulling08_pervasive, author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans and Tr{\"{o}}ster, Gerhard}, keywords = {Activity Recognition, Electrooculography (EOG), Reading Activity, Recognition of Reading, Transit, wearable}, title = {Robust {R}ecognition of {R}eading {A}ctivity in {T}ransit {U}sing {W}earable {E}lectrooculography}, booktitle = {Proc. International Conference on Pervasive Computing (Pervasive)}, year = {2008}, pages = {19-37}, doi = {10.1007/978-3-540-79576-6_2} }

The dataset has the following characteristics: