Mihai Bâce

Institute for Visualisation and Interactive Systems, University of Stuttgart

  Pfaffenwaldring 5a, 70569 Stuttgart, Germany University of Stuttgart, SimTech Building, Room 01.029 Google Scholar

Biography

Mihai Bâce is a post-doctoral researcher in the Perceptual User Interfaces group since December 2020. He did his PhD at ETH Zurich, Switzerland, in the Distributed Systems Group at the Institute for Intelligent Interactive Systems. He studied for his Master's degree at École Polytechnique Fédérale de Lausanne (EPFL) and for his Bachelor's at the Technical University of Cluj-Napoca in Romania. His research interests are in Human-Computer Interaction with a focus on computational aspects and has influences from both Machine Learning and Computer Vision.

Teaching

2022

Machine Perception and Learning (Teaching Assistant) (summer)Master

 

Machine Perception and Learning (Teaching Assistant) (winter)Master

---

Supervision

2022

AI-Supported Labeling for Mobile Eye Tracking (M.Sc. Research Project) Swetha Lakshmana Murthy*

 

Sensing Covert Attention during Everyday Interactions (Internship) Christophe Romein (Utrecht University)

 

Generative Modelling of Gaze Behaviour (M.Sc.) Chuhan Jiao (Aalto University)

 

Generative Modelling for Appearance-based Eye Contact Detection (M.Sc. Research Project) Simhadri Holagundhi

 

Photo-realistic Eye Image Generation Through Image-to-Image Translation (M.Sc.) Abdelrahman Mechraoui

*Co-supervised with Kuno Kurzhals and Sandeep Vidyapu

---

Open Thesis Projects

 

Modelling Application Switching during Everyday Mobile Device InteractionsMaster

---

Publications

(Popescu et al., 2011; Popescu et al., 2011; Bâce & Pignolet, 2015; Bâce et al., 2016; Bâce, 2017; Bâce et al., 2017; Bâce et al., 2017; Bâce et al., 2017; Bâce et al., 2018; Bâce et al., 2020; Bâce et al., 2020; Bâce et al., 2020; Strohm et al., 2021; Müller et al., 2022; Bâce et al., 2022; Zhang et al., 2022; Wang et al., 2022; Abdessaied et al., 2022; Wang et al., 2022; Wang et al., 2023; Zhang et al., 2023; Sood et al., 2023; Penzkofer et al., 2023; Strohm et al., 2023; Jiao et al., 2023; Strohm et al., 2023)

1. 

   Scanpath Prediction on Information Visualisations

   Yao Wang, Mihai Bâce, Andreas Bulling

   IEEE Transactions on Visualization and Computer Graphics (TVCG), (), pp. 1–15, 2023.

   Abstract Links BibTeX Project

   We propose Unified Model of Saliency and Scanpaths (UMSS) – a model that learns to predict multi-duration saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect tto several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5 % for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6 % for saliency prediction. These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.

   doi: 10.1109/TVCG.2023.3242293

   Paper: wang23_tvcg.pdf

   Supplementary Material: wang23_tvcg_sup.pdf

   Dataset: https://darus.uni-stuttgart.de/dataset.xhtml?persistentId=doi:10.18419/darus-3361

   @article{wang23_tvcg, title = {Scanpath Prediction on Information Visualisations}, author = {Wang, Yao and Bâce, Mihai and Bulling, Andreas}, year = {2023}, journal = {IEEE Transactions on Visualization and Computer Graphics (TVCG)}, volume = {}, number = {}, pages = {1--15}, doi = {10.1109/TVCG.2023.3242293} }

1. 

   Exploring Natural Language Processing Methods for Interactive Behaviour Modelling

   Guanhua Zhang, Matteo Bortoletto, Zhiming Hu, Lei Shi, Mihai Bâce, Andreas Bulling

   Proc. IFIP TC13 Conference on Human-Computer Interaction (INTERACT), pp. 1–22, 2023.

   Abstract Links BibTeX Project Best Student Paper Nomination

   Analysing and modelling interactive behaviour is an important topic in human-computer interaction (HCI) and a key requirement for the development of intelligent interactive systems. Interactive behaviour has a sequential (actions happen one after another) and hierarchical (a sequence of actions forms an activity driven by interaction goals) structure, which may be similar to the structure of natural language. Designed based on such a structure, natural language processing (NLP) methods have achieved groundbreaking success in various downstream tasks. However, few works linked interactive behaviour with natural language. In this paper, we explore the similarity between interactive behaviour and natural language by applying an NLP method, byte pair encoding (BPE), to encode mouse and keyboard behaviour. We then analyse the vocabulary, i.e., the set of action sequences, learnt by BPE, as well as use the vocabulary to encode the input behaviour for interactive task recognition. An existing dataset collected in constrained lab settings and our novel out-of-the-lab dataset were used for evaluation. Results show that this natural language-inspired approach not only learns action sequences that reflect specific interaction goals, but also achieves higher F1 scores on task recognition than other methods. Our work reveals the similarity between interactive behaviour and natural language, and presents the potential of applying the new pack of methods that leverage insights from NLP to model interactive behaviour in HCI.

   Paper: zhang23_interact.pdf

   @inproceedings{zhang23_interact, title = {Exploring Natural Language Processing Methods for Interactive Behaviour Modelling}, author = {Zhang, Guanhua and Bortoletto, Matteo and Hu, Zhiming and Shi, Lei and B{\^a}ce, Mihai and Bulling, Andreas}, booktitle = {Proc. IFIP TC13 Conference on Human-Computer Interaction (INTERACT)}, pages = {1--22}, year = {2023}, publisher = {Springer} }
2. 

   Multimodal Integration of Human-Like Attention in Visual Question Answering

   Ekta Sood, Fabian Kögel, Philipp Müller, Dominike Thomas, Mihai Bâce, Andreas Bulling

   Proc. Workshop on Gaze Estimation and Prediction in the Wild (GAZE), CVPRW, pp. 2647–2657, 2023.

   Abstract Links BibTeX Project Tobii Sponsor award, oral presentation

   Human-like attention as a supervisory signal to guide neural attention has shown significant promise but is currently limited to uni-modal integration – even for inherently multi-modal tasks such as visual question answering (VQA). We present the Multimodal Human-like Attention Network (MULAN) – the first method for multimodal integration of human-like attention on image and text during training of VQA models. MULAN integrates attention predictions from two state-of-the-art text and image saliency models into neural self-attention layers of a recent transformer-based VQA model. Through evaluations on the challenging VQAv2 dataset, we show that MULAN achieves a new state-of-the-art performance of 73.98% accuracy on test-std and 73.72% on test-dev and, at the same time, has approximately 80% fewer trainable parameters than prior work. Overall, our work underlines the potential of integrating multimodal human-like and neural attention for VQA.

   Paper: sood23_gaze.pdf

   Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/MULAN-VQA/

   Paper Access: https://openaccess.thecvf.com/content/CVPR2023W/GAZE/papers/Sood_Multimodal_Integration_of_Human-Like_Attention_in_Visual_Question_Answering_CVPRW_2023_paper.pdf

   @inproceedings{sood23_gaze, author = {Sood, Ekta and Kögel, Fabian and Müller, Philipp and Thomas, Dominike and Bâce, Mihai and Bulling, Andreas}, title = {Multimodal Integration of Human-Like Attention in Visual Question Answering}, booktitle = {Proc. Workshop on Gaze Estimation and Prediction in the Wild (GAZE), CVPRW}, year = {2023}, pages = {2647--2657}, url = {https://openaccess.thecvf.com/content/CVPR2023W/GAZE/papers/Sood_Multimodal_Integration_of_Human-Like_Attention_in_Visual_Question_Answering_CVPRW_2023_paper.pdf} }
3. 

   Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning

   Anna Penzkofer, Simon Schaefer, Florian Strohm, Mihai Bâce, Stefan Leutenegger, Andreas Bulling

   Proc. Adaptive and Learning Agents Workshop (ALA), pp. 1–7, 2023.

   Abstract Links BibTeX Project

   While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the learning process. We show that intentions of human players, i.e. the precursor of goal-oriented decisions, can be robustly predicted from eye gaze even for the long-horizon sparse rewards task of Montezuma’s Revenge – one of the most challenging RL tasks in the Atari2600 game suite. We propose Int-HRL: Hierarchical RL with intention-based sub-goals that are inferred from human eye gaze. Our novel sub-goal extraction pipeline is fully automatic and replaces the need for manual sub-goal annotation by human experts. Our evaluations show that replacing hand-crafted sub-goals with automatically extracted intentions leads to a HRL agent that is significantly more sample efficient than previous methods.

   doi:

   Paper: penzkofer23_ala.pdf

   @inproceedings{penzkofer23_ala, author = {Penzkofer, Anna and Schaefer, Simon and Strohm, Florian and Bâce, Mihai and Leutenegger, Stefan and Bulling, Andreas}, title = {Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning}, booktitle = {Proc. Adaptive and Learning Agents Workshop (ALA)}, year = {2023}, doi = {}, pages = {1--7} }
4. 

   Facial Composite Generation with Iterative Human Feedback

   Florian Strohm, Ekta Sood, Dominike Thomas, Mihai Bâce, Andreas Bulling

   Proc. The 1st Gaze Meets ML workshop, PMLR, pp. 165–183, 2023.

   Abstract Links BibTeX Project

   We propose the first method in which human and AI collaborate to iteratively reconstruct the human’s mental image of another person’s face only from their eye gaze. Current tools for generating digital human faces involve a tedious and time-consuming manual design process. While gaze-based mental image reconstruction represents a promising alternative, previous methods still assumed prior knowledge about the target face, thereby severely limiting their practical usefulness. The key novelty of our method is a collaborative, it- erative query engine: Based on the user’s gaze behaviour in each iteration, our method predicts which images to show to the user in the next iteration. Results from two human studies (N=12 and N=22) show that our method can visually reconstruct digital faces that are more similar to the mental image, and is more usable compared to other methods. As such, our findings point at the significant potential of human-AI collaboration for recon- structing mental images, potentially also beyond faces, and of human gaze as a rich source of information and a powerful mediator in said collaboration.

   Paper: strohm23_gmml.pdf

   Paper Access: https://proceedings.mlr.press/v210/strohm23a.html

   @inproceedings{strohm23_gmml, title = {Facial Composite Generation with Iterative Human Feedback}, author = {Strohm, Florian and Sood, Ekta and Thomas, Dominike and B{\^a}ce, Mihai and Bulling, Andreas}, booktitle = {Proc. The 1st Gaze Meets ML workshop, PMLR}, pages = {165--183}, year = {2023}, editor = {Lourentzou, Ismini and Wu, Joy and Kashyap, Satyananda and Karargyris, Alexandros and Celi, Leo Anthony and Kawas, Ban and Talathi, Sachin}, volume = {210}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v210/strohm23a/strohm23a.pdf}, url = {https://proceedings.mlr.press/v210/strohm23a.html} }
5. 

   SUPREYES: SUPer Resolution for EYES Using Implicit Neural Representation Learning

   Chuhan Jiao, Zhiming Hu, Mihai Bâce, Andreas Bulling

   Proc. ACM Symposium on User Interface Software and Technology (UIST), pp. 1–13, 2023.

   Abstract Links BibTeX Project

   We introduce SUPREYES – a novel self-supervised method to increase the spatio-temporal resolution of gaze data recorded using low(er)-resolution eye trackers. Despite continuing advances in eye tracking technology, the vast majority of current eye trackers – particularly mobile ones and those integrated into mobile devices – suffer from low-resolution gaze data, thus fundamentally limiting their practical usefulness. SUPREYES learns a continuous implicit neural representation from low-resolution gaze data to up-sample the gaze data to arbitrary resolutions. We compare our method with commonly used interpolation methods on arbitrary scale super-resolution and demonstrate that SUPREYES outperforms these baselines by a significant margin. We also test on the sample downstream task of gaze-based user identification and show that our method improves the performance of original low-resolution gaze data and outperforms other baselines. These results are promising as they open up a new direction for increasing eye tracking fidelity as well as enabling new gaze-based applications without the need for new eye tracking equipment.

   doi: 10.1145/3586183.3606780

   Paper: jiao23_uist.pdf

   @inproceedings{jiao23_uist, author = {Jiao, Chuhan and Hu, Zhiming and B{\^a}ce, Mihai and Bulling, Andreas}, title = {SUPREYES: SUPer Resolution for EYES Using Implicit Neural Representation Learning}, booktitle = {Proc. ACM Symposium on User Interface Software and Technology (UIST)}, year = {2023}, pages = {1--13}, doi = {10.1145/3586183.3606780} }
6. 

   Usable and Fast Interactive Mental Face Reconstruction

   Florian Strohm, Mihai Bâce, Andreas Bulling

   Proc. ACM Symposium on User Interface Software and Technology (UIST), pp. 1–15, 2023.

   Abstract Links BibTeX Project

   We introduce an end-to-end interactive system for mental face reconstruction – the challenging task of visually reconstructing a face image a person only has in their mind. In contrast to existing methods that suffer from low usability and high mental load, our approach only requires the user to rank images over multiple iterations according to the perceived similarity with their mental image. Based on these rankings, our mental face reconstruction system extracts image features in each iteration, combines them into a joint feature vector, and then uses a generative model to visually reconstruct the mental image. To avoid the need for collecting large amounts of human training data, we further propose a computational user model that can simulate human ranking behaviour using data from an online crowd-sourcing study (N=215). Results from a 12-participant user study show that our method can reconstruct mental images that are visually similar to existing approaches but has significantly higher usability, lower perceived workload, and is 40% faster. In addition, results from a third 22-participant lineup study in which we validated our reconstructions on a face ranking task show a identification rate of 55.3%, which is in line with prior work. These results represent an important step towards new interactive intelligent systems that can robustly and effortlessly reconstruct a user’s mental image.

   doi: https://doi.org/10.1145/3586183.3606795

   Paper: strohm23_uist.pdf

   @inproceedings{strohm23_uist, author = {Strohm, Florian and B{\^a}ce, Mihai and Bulling, Andreas}, title = {Usable and Fast Interactive Mental Face Reconstruction}, booktitle = {Proc. ACM Symposium on User Interface Software and Technology (UIST)}, year = {2023}, pages = {1--15}, doi = {https://doi.org/10.1145/3586183.3606795} }

1. 

   VisRecall: Quantifying Information Visualisation Recallability via Question Answering

   Yao Wang, Chuhan Jiao, Mihai Bâce, Andreas Bulling

   IEEE Transactions on Visualization and Computer Graphics (TVCG), 28(12), pp. 4995-5005, 2022.

   Abstract Links BibTeX Project Oral presentation

   Despite its importance for assessing the effectiveness of communicating information visually, fine-grained recallability of information visualisations has not been studied quantitatively so far. In this work, we propose a question-answering paradigm to study visualisation recallability and present VisRecall - a novel dataset consisting of 200 visualisations that are annotated with crowd-sourced human (N = 305) recallability scores obtained from 1,000 questions of five question types. Furthermore, we present the first computational method to predict recallability of different visualisation elements, such as the title or specific data values. We report detailed analyses of our method on VisRecall and demonstrate that it outperforms several baselines in overall recallability and FE-, F-, RV-, and U-question recallability. Our work makes fundamental contributions towards a new generation of methods to assist designers in optimising visualisations.

   doi: 10.1109/TVCG.2022.3198163

   Paper: wang22_tvcg.pdf

   Supplementary Material: wang22_tvcg_sup.pdf

   Dataset: https://darus.uni-stuttgart.de/dataset.xhtml?persistentId=doi:10.18419/darus-2826

   @article{wang22_tvcg, title = {VisRecall: Quantifying Information Visualisation Recallability via Question Answering}, author = {Wang, Yao and Jiao, Chuhan and Bâce, Mihai and Bulling, Andreas}, year = {2022}, journal = {IEEE Transactions on Visualization and Computer Graphics (TVCG)}, volume = {28}, number = {12}, pages = {4995-5005}, doi = {10.1109/TVCG.2022.3198163} }

1. 

   Designing for Noticeability: The Impact of Visual Importance on Desktop Notifications

   Philipp Müller, Sander Staal, Mihai Bâce, Andreas Bulling

   Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1–13, 2022.

   Abstract Links BibTeX Project

   Desktop notifications should be noticeable but are also subject to a number of design choices, e.g. concerning their size, placement, or opacity. It is currently unknown, however, how these choices interact with the desktop background and their influence on noticeability. To address this limitation, we introduce a software tool to automatically synthesize realistically looking desktop images for major operating systems and applications. Using these images, we present a user study (N=34) to investigate the noticeability of notifications during a primary task. We are first to show that visual importance of the background at the notification location significantly impacts whether users detect notifications. We analyse the utility of visual importance to compensate for suboptimal design choices with respect to noticeability, e.g. small notification size. Finally, we introduce noticeability maps - 2D maps encoding the predicted noticeability across the desktop and inform designers how to trade-off notification design and noticeability.

   doi: 10.1145/3491102.3501954

   Paper: mueller22_chi.pdf

   @inproceedings{mueller22_chi, title = {Designing for Noticeability: The Impact of Visual Importance on Desktop Notifications}, author = {Müller, Philipp and Staal, Sander and B{\^a}ce, Mihai and Bulling, Andreas}, year = {2022}, pages = {1--13}, booktitle = {Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)}, doi = {10.1145/3491102.3501954} }
2. 

   PrivacyScout: Assessing Vulnerability to Shoulder Surfing on Mobile Devices

   Mihai Bâce, Alia Saad, Mohamed Khamis, Stefan Schneegass, Andreas Bulling

   Proc. on Privacy Enhancing Technologies (PETs), pp. 650–669, 2022.

   Abstract Links BibTeX Project

   One approach to mitigate shoulder surfing attacks on mobile devices is to detect the presence of a bystander using the phone’s front-facing camera. However, a person’s face in the camera’s field of view does not always indicate an attack. To overcome this limitation, in a novel data collection study (N=16), we analysed the influence of three viewing angles and four distances on the success of shoulder surfing attacks. In contrast to prior works that mainly focused on user authentication, we investigated three common types of content susceptible to shoulder surfing: text, photos, and PIN authentications. We show that the vulnerability of text and photos depends on the observer’s location relative to the device, while PIN authentications are vulnerable independent of the observation location. We then present PrivacyScout - a novel method that predicts the shoulder-surfing risk based on visual features extracted from the observer’s face as captured by the front-facing camera. Finally, evaluations from our data collection study demonstrate our method’s feasibility to assess the risk of a shoulder surfing attack more accurately.

   doi: 10.56553/popets-2022-0090

   Paper: bace22_pets.pdf

   @inproceedings{bace22_pets, title = {PrivacyScout: Assessing Vulnerability to Shoulder Surfing on Mobile Devices}, author = {B{\^a}ce, Mihai and Saad, Alia and Khamis, Mohamed and Schneegass, Stefan and Bulling, Andreas}, year = {2022}, booktitle = {Proc. on Privacy Enhancing Technologies (PETs)}, doi = {10.56553/popets-2022-0090}, pages = {650--669}, issue = {3} }
3. 

   Predicting Next Actions and Latent Intents during Text Formatting

   Guanhua Zhang, Susanne Hindennach, Jan Leusmann, Felix Bühler, Benedict Steuerlein, Sven Mayer, Mihai Bâce, Andreas Bulling

   Proc. the CHI Workshop Computational Approaches for Understanding, Generating, and Adapting User Interfaces, pp. 1–6, 2022.

   Abstract Links BibTeX Project

   In this work we investigate the challenging task of predicting user intents from mouse and keyboard input as well as gaze behaviour. In contrast to prior work we study intent prediction at two different resolutions on the behavioural timeline: predicting future input actions as well as latent intents to achieve a high-level interaction goal. Results from a user study (N=15) on a sample text formatting task show that the sequence of prior actions is more informative for intent prediction than gaze. Only using the action sequence, we can predict the next action and the high-level intent with an accuracy of 66% and 96%, respectively. In contrast, accuracy when using features extracted from gaze behaviour was significantly lower, at 41% and 46%. This finding is important for the development of future anticipatory user interfaces that aim to proactively adapt to user intents and interaction goals.

   Paper: zhang22_caugaui.pdf

   @inproceedings{zhang22_caugaui, author = {Zhang, Guanhua and Hindennach, Susanne and Leusmann, Jan and Bühler, Felix and Steuerlein, Benedict and Mayer, Sven and Bâce, Mihai and Bulling, Andreas}, title = {Predicting Next Actions and Latent Intents during Text Formatting}, booktitle = {Proc. the CHI Workshop Computational Approaches for Understanding, Generating, and Adapting User Interfaces}, year = {2022}, pages = {1--6} }
4. 

   Neuro-Symbolic Visual Dialog

   Adnen Abdessaied, Mihai Bâce, Andreas Bulling

   Proceedings of the 29th International Conference on Computational Linguistics (COLING), pp. 1–11, 2022.

   Abstract Links BibTeX Project Oral presentation

   We propose Neuro-Symbolic Visual Dialog (NSVD) —the first method to combine deep learning and symbolic program execution for multi-round visually-grounded reasoning. NSVD significantly outperforms existing purely-connectionist methods on two key challenges inherent to visual dialog: long-distance co-reference resolution as well as vanishing question-answering performance. We demonstrate the latter by proposing a more realistic and stricter evaluation scheme in which we use predicted answers for the full dialog history when calculating accuracy. We describe two variants of our model and show that using this new scheme, our best model achieves an accuracy of 99.72% on CLEVR-Dialog —a relative improvement of more than 10% over the state of the art —while only requiring a fraction of training data. Moreover, we demonstrate that our neuro-symbolic models have a higher mean first failure round, are more robust against incomplete dialog histories, and generalise better not only to dialogs that are up to three times longer than those seen during training but also to unseen question types and scenes.

   Paper: abdessaied22_coling.pdf

   Code: Contact Author for further information

   @inproceedings{abdessaied22_coling, author = {Abdessaied, Adnen and Bâce, Mihai and Bulling, Andreas}, title = {Neuro-Symbolic Visual Dialog}, booktitle = {Proceedings of the 29th International Conference on Computational Linguistics (COLING)}, year = {2022}, pages = {1--11} }
5. 

   Impact of Gaze Uncertainty on AOIs in Information Visualisations

   Yao Wang, Maurice Koch, Mihai Bâce, Daniel Weiskopf, Andreas Bulling

   ETRA Workshop on Eye Tracking and Visualization (ETVIS), pp. 1–6, 2022.

   Abstract Links BibTeX Project

   Gaze-based analysis of areas of interest (AOIs) is widely used in information visualisation research to understand how people explore visualisations or assess the quality of visualisations concerning key characteristics such as memorability. However, nearby AOIs in visualisations amplify the uncertainty caused by the gaze estimation error, which strongly influences the mapping between gaze samples or fixations and different AOIs. We contribute a novel investigation into gaze uncertainty and quantify its impact on AOI-based analysis on visualisations using two novel metrics: the Flipping Candidate Rate (FCR) and Hit Any AOI Rate (HAAR). Our analysis of 40 real-world visualisations, including human gaze and AOI annotations, shows that gaze uncertainty frequently and significantly impacts the analysis conducted in AOI-based studies. Moreover, we analysed four visualisation types and found that bar and scatter plots are usually designed in a way that causes more uncertainty than line and pie plots in gaze-based analysis.

   doi: 10.1145/3517031.3531166

   Paper: wang22_etvis.pdf

   Supplementary Material: wang22_etvis_sup.pdf

   @inproceedings{wang22_etvis, title = {Impact of Gaze Uncertainty on AOIs in Information Visualisations}, author = {Wang, Yao and Koch, Maurice and B{\^a}ce, Mihai and Weiskopf, Daniel and Bulling, Andreas}, year = {2022}, pages = {1--6}, booktitle = {ETRA Workshop on Eye Tracking and Visualization (ETVIS)}, doi = {10.1145/3517031.3531166} }

1. 

   Neural Photofit: Gaze-based Mental Image Reconstruction

   Florian Strohm, Ekta Sood, Sven Mayer, Philipp Müller, Mihai Bâce, Andreas Bulling

   Proc. IEEE International Conference on Computer Vision (ICCV), pp. 245-254, 2021.

   Abstract Links BibTeX Project

   We propose a novel method that leverages human fixations to visually decode the image a person has in mind into a photofit (facial composite). Our method combines three neural networks: An encoder, a scoring network, and a decoder. The encoder extracts image features and predicts a neural activation map for each face looked at by a human observer. A neural scoring network compares the human and neural attention and predicts a relevance score for each extracted image feature. Finally, image features are aggregated into a single feature vector as a linear combination of all features weighted by relevance which a decoder decodes into the final photofit. We train the neural scoring network on a novel dataset containing gaze data of 19 participants looking at collages of synthetic faces. We show that our method significantly outperforms a mean baseline predictor and report on a human study that shows that we can decode photofits that are visually plausible and close to the observer’s mental image. Code and dataset available upon request.

   doi: 10.1109/ICCV48922.2021.00031

   Paper: strohm21_iccv.pdf

   Code: Available upon request.

   Dataset: Available upon request.

   @inproceedings{strohm21_iccv, title = {Neural Photofit: Gaze-based Mental Image Reconstruction}, author = {Strohm, Florian and Sood, Ekta and Mayer, Sven and Müller, Philipp and Bâce, Mihai and Bulling, Andreas}, year = {2021}, booktitle = {Proc. IEEE International Conference on Computer Vision (ICCV)}, doi = {10.1109/ICCV48922.2021.00031}, pages = {245-254} }

1. 

   How far are we from quantifying visual attention in mobile HCI?

   Mihai Bâce, Sander Staal, Andreas Bulling

   IEEE Pervasive Computing, 19(2), pp. 46-55, 2020.

   Abstract Links BibTeX Project

   With an ever-increasing number of mobile devices competing for attention, quantifying when, how often, or for how long users look at their devices has emerged as a key challenge in mobile human-computer interaction. Encouraged by recent advances in automatic eye contact detection using machine learning and device-integrated cameras, we provide a fundamental investigation into the feasibility of quantifying overt visual attention during everyday mobile interactions. We discuss the main challenges and sources of error associated with sensing visual attention on mobile devices in the wild, including the impact of face and eye visibility, the importance of robust head pose estimation, and the need for accurate gaze estimation. Our analysis informs future research on this emerging topic and underlines the potential of eye contact detection for exciting new applications towards next-generation pervasive attentive user interfaces.

   doi: 10.1109/MPRV.2020.2967736

   Paper: bace20_pcm.pdf

   @article{bace20_pcm, title = {How far are we from quantifying visual attention in mobile HCI?}, author = {B{\^a}ce, Mihai and Staal, Sander and Bulling, Andreas}, journal = {IEEE Pervasive Computing}, year = {2020}, volume = {19}, number = {2}, doi = {10.1109/MPRV.2020.2967736}, pages = {46-55} }

1. 

   Combining Gaze Estimation and Optical Flow for Pursuits Interaction

   Mihai Bâce, Vincent Becker, Chenyang Wang, Andreas Bulling

   Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA), pp. 1-10, 2020.

   Abstract Links BibTeX Project Best paper award

   Pursuit eye movements have become widely popular because they enable spontaneous eye-based interaction. However, existing methods to detect smooth pursuits require special-purpose eye trackers. We propose the first method to detect pursuits using a single off-the-shelf RGB camera in unconstrained remote settings. The key novelty of our method is that it combines appearance-based gaze estimation with optical flow in the eye region to jointly analyse eye movement dynamics in a single pipeline. We evaluate the performance and robustness of our method for different numbers of targets and trajectories in a 13-participant user study. We show that our method not only outperforms the current state of the art but also achieves competitive performance to a consumer eye tracker for a small number of targets. As such, our work points towards a new family of methods for pursuit interaction directly applicable to an ever-increasing number of devices readily equipped with cameras.

   doi: 10.1145/3379155.3391315

   Paper: bace20_etra.pdf

   @inproceedings{bace20_etra, title = {Combining Gaze Estimation and Optical Flow for Pursuits Interaction}, author = {B{\^a}ce, Mihai and Becker, Vincent and Wang, Chenyang and Bulling, Andreas}, year = {2020}, booktitle = {Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA)}, doi = {10.1145/3379155.3391315}, pages = {1-10} }
2. 

   Quantification of Users’ Visual Attention During Everyday Mobile Device Interactions

   Mihai Bâce, Sander Staal, Andreas Bulling

   Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1–14, 2020.

   Abstract Links BibTeX Project

   We present the first real-world dataset and quantitative evaluation of visual attention of mobile device users in-situ, i.e. while using their devices during everyday routine. Understanding user attention is a core research challenge in mobile HCI but previous approaches relied on usage logs or self-reports that are only proxies and consequently do neither reflect attention completely nor accurately. Our evaluations are based on Everyday Mobile Visual Attention (EMVA) – a new 32-participant dataset containing around 472 hours of video snippets recorded over more than two weeks in real life using the front-facing camera as well as associated usage logs, interaction events, and sensor data. Using an eye contact detection method, we are first to quantify the highly dynamic nature of everyday visual attention across users, mobile applications, and usage contexts. We discuss key insights from our analyses that highlight the potential and inform the design of future mobile attentive user interfaces.

   doi: 10.1145/3313831.3376449

   Paper: bace20_chi.pdf

   Dataset: http://www.emva-dataset.org/

   News: https://ethz.ch/en/news-and-events/eth-news/news/2020/09/our-actual-attention-is-now-measurable.html

   Video: https://www.youtube.com/watch?v=SzLn3LujIqw

   @inproceedings{bace20_chi, title = {Quantification of Users' Visual Attention During Everyday Mobile Device Interactions}, author = {B{\^a}ce, Mihai and Staal, Sander and Bulling, Andreas}, year = {2020}, pages = {1--14}, booktitle = {Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)}, doi = {10.1145/3313831.3376449}, news = {https://ethz.ch/en/news-and-events/eth-news/news/2020/09/our-actual-attention-is-now-measurable.html}, video = {https://www.youtube.com/watch?v=SzLn3LujIqw} }

1. Wearable Eye Tracker Calibration at Your Fingertips

   Mihai Bâce, Sander Staal, Gábor Sörös

   ACM Symposium on Eye Tracking Research & Applications, 2018.

   Links BibTeX Project

   doi: 10.1145/3204493.3204592

   @inproceedings{bace18_etra, author = {B{\^a}ce, Mihai and Staal, Sander and S{\"o}r{\"o}s, G{\'a}bor}, title = {Wearable Eye Tracker Calibration at Your Fingertips}, year = {2018}, booktitle = {ACM Symposium on Eye Tracking Research & Applications}, address = {Warsaw, Poland}, series = {ETRA '18}, publisher = {ACM}, doi = {10.1145/3204493.3204592} }

1. Collocated Multi-user Gestural Interactions with Unmodified Wearable Devices

   Mihai Bâce, Sander Staal, Gábor Sörös, Giorgio Corbellini

   Augmented Human Research, 2, 2017.

   Links BibTeX Project

   doi: 10.1007/s41133-017-0009-z

   @article{bace17_ahr, author = {B{\^a}ce, Mihai and Staal, Sander and S{\"o}r{\"o}s, G{\'a}bor and Corbellini, Giorgio}, title = {Collocated Multi-user Gestural Interactions with Unmodified Wearable Devices}, year = {2017}, journal = {Augmented Human Research}, volume = {2}, doi = {10.1007/s41133-017-0009-z} }

1. Augmenting Human Interaction Capabilities with Proximity, Natural Gestures, and Eye Gaze

   Mihai Bâce

   Proceedings of the 19th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017), 2017.

   Links BibTeX Project

   doi: 10.1145/3098279.3119924

   @inproceedings{bace17_mobilehci, author = {B{\^a}ce, Mihai}, title = {Augmenting Human Interaction Capabilities with Proximity, Natural Gestures, and Eye Gaze}, year = {2017}, booktitle = {Proceedings of the 19th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017)}, address = {Vienna, Austria}, series = {MobileHCI '17}, publisher = {ACM}, isbn = {978-1-4503-4835-5}, doi = {10.1145/3098279.3119924} }

2. Facilitating Object Detection and Recognition through Eye Gaze

   Mihai Bâce, Philippe Schlattner, Vincent Becker, Gábor Sörös

   Proceedings of the Workshop on Object Recognition for Input and Mobile Interaction at the 19th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017), 2017.

   Links BibTeX Project

   doi: 10.3929/ethz-b-000221545

   @inproceedings{bace17_mobilehci_2, author = {B{\^a}ce, Mihai and Schlattner, Philippe and Becker, Vincent and S{\"o}r{\"o}s, G{\'a}bor}, title = {Facilitating Object Detection and Recognition through Eye Gaze}, year = {2017}, booktitle = {Proceedings of the Workshop on Object Recognition for Input and Mobile Interaction at the 19th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017)}, address = {Vienna, Austria}, series = {MobileHCI '17}, publisher = {ACM}, isbn = {978-1-4503-4835-5}, doi = {10.3929/ethz-b-000221545} }

3. HandshakAR: Wearable Augmented Reality System for Effortless Information Sharing

   Mihai Bâce, Gábor Sörös, Sander Staal, Giorgio Corbellini

   Proceedings of the Augmented Human 2017 Conference (AH 2017), 2017.

   Links BibTeX Project

   doi: 10.1145/3041164.3041203

   @inproceedings{bace17_ah, author = {B{\^a}ce, Mihai and S{\"o}r{\"o}s, G{\'a}bor and Staal, Sander and Corbellini, Giorgio}, title = {HandshakAR: Wearable Augmented Reality System for Effortless Information Sharing}, year = {2017}, booktitle = {Proceedings of the Augmented Human 2017 Conference (AH 2017)}, address = {Mountain View, CA, USA}, series = {AH '17}, publisher = {ACM}, isbn = {978-1-4503-4835-5}, doi = {10.1145/3041164.3041203} }

1. ubiGaze: Ubiquitous Augmented Reality Messaging Using Gaze Gestures

   Mihai Bâce, Teemu Leppänen, Argenis Ramirez Gomez, David Gil Gomez

   SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications, pp. 11:1–11:5, 2016.

   Links BibTeX Project

   doi: 10.1145/2999508.2999530

   @inproceedings{bace16_mgia, author = {B{\^a}ce, Mihai and Lepp{\"a}nen, Teemu and Gomez, Argenis Ramirez and de Gomez, David Gil}, title = {ubiGaze: Ubiquitous Augmented Reality Messaging Using Gaze Gestures}, year = {2016}, booktitle = {SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications}, address = {Macau}, series = {SA '16}, pages = {11:1--11:5}, publisher = {ACM}, isbn = {978-1-4503-4551-4}, doi = {10.1145/2999508.2999530} }

1. Lightweight Indoor Localization System

   Mihai Bâce, Yvonne-Anne Pignolet

   Proceedings of the 8th IFIP Wireless and Mobile Networking Conference (WMNC 2015). Munich, Germany., pp. 160–167, 2015.

   Links BibTeX Project

   doi: 10.1109/WMNC.2015.22

   @inproceedings{bace15_wmnc, author = {B{\^a}ce, Mihai and Pignolet, Yvonne-Anne}, title = {Lightweight Indoor Localization System}, year = {2015}, booktitle = {Proceedings of the 8th IFIP Wireless and Mobile Networking Conference (WMNC 2015). Munich, Germany.}, pages = {160--167}, doi = {10.1109/WMNC.2015.22} }

1. Lane identification and ego-vehicle accurate global positioning in intersections

   Voichita Popescu, Mihai Bâce, Sergiu Nedevschi

   Intelligent Vehicles Symposium (IV 2011). Baden-Baden, Germany., pp. 870 - 875, 2011.

   Links BibTeX Project

   doi: 10.1109/IVS.2011.5940523

   @inproceedings{bace11_iv, author = {Popescu, Voichita and B{\^a}ce, Mihai and Nedevschi, Sergiu}, title = {Lane identification and ego-vehicle accurate global positioning in intersections}, year = {2011}, booktitle = {Intelligent Vehicles Symposium (IV 2011). Baden-Baden, Germany.}, pages = {870 - 875}, publisher = {IEEE}, isbn = {978-1-4577-0890-9}, doi = {10.1109/IVS.2011.5940523} }

2. Probabilistic Approach for Automated Reasoning for Lane Identification in Intelligent Vehicles

   Voichita Popescu, Mihai Bâce, Sergiu Nedevschi

   International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC 2011). Timisoara, Romania., pp. 255 - 258, 2011.

   Links BibTeX Project

   doi: 10.1109/SYNASC.2011.10

   @inproceedings{bace11_synasc, author = {Popescu, Voichita and B{\^a}ce, Mihai and Nedevschi, Sergiu}, title = {Probabilistic Approach for Automated Reasoning for Lane Identification in Intelligent Vehicles}, year = {2011}, booktitle = {International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC 2011). Timisoara, Romania.}, pages = {255 - 258}, publisher = {IEEE}, isbn = {978-1-4673-0207-4}, doi = {10.1109/SYNASC.2011.10} }