Fabian Kögel

Institute for Visualisation and Interactive Systems, University of Stuttgart

Pfaffenwaldring 5a, 70569 Stuttgart, Germany University of Stuttgart, SimTech Building, Room 01.033 Google Scholar

Biography

Fabian Kögel is a PhD student in the Perceptual User Interface group since July 2024. He holds a Bachelor`s degree (with honors) in Audiovisual Media from the Media University Stuttgart and a Master's Degree in Information Technology from the University of Stuttgart. Since 2019 he has been doing research in the Perceptual User Interface group as a research assistant. Before returning to the institute, he has worked two years in industry as a research engineer at Sony Europe in the field of Expressive Speech Synthesis, Speech Analysis and Generative AI. His focus now lies in explainability, cognitive reasoning and fairness as well as integrating human feedback such as eye gaze to improve model usefulness.

Publications

1. 

   InteRead: An Eye Tracking Dataset of Interrupted Reading

   Francesca Zermiani, Prajit Dhar, Ekta Sood, Fabian Kögel, Andreas Bulling, Maria Wirzberger

   Proc. 31st Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), pp. 9154–9169, 2024.

   Abstract Links BibTeX Project

   Eye movements during reading offer a window into cognitive processes and language comprehension, but the scarcity of reading data with interruptions – which learners frequently encounter in their everyday learning environments – hampers advances in the development of intelligent learning technologies. We introduce InteRead – a novel 50-participant dataset of gaze data recorded during self-paced reading of real-world text. InteRead further offers fine-grained annotations of interruptions interspersed throughout the text as well as resumption lags incurred by these interruptions. Interruptions were triggered automatically once readers reached predefined target words. We validate our dataset by reporting interdisciplinary analyses on different measures of gaze behavior. In line with prior research, our analyses show that the interruptions as well as word length and word frequency effects significantly impact eye movements during reading. We also explore individual differences within our dataset, shedding light on the potential for tailored educational solutions. InteRead is accessible from our datasets web-page: https://www.ife.uni-stuttgart.de/en/llis/research/datasets/.

   doi:

   Paper: zermiani24_coling.pdf

   Paper Access: https://aclanthology.org/2024.lrec-main.802/

   @inproceedings{zermiani24_coling, title = {InteRead: An Eye Tracking Dataset of Interrupted Reading}, author = {Zermiani, Francesca and Dhar, Prajit and Sood, Ekta and Kögel, Fabian and Bulling, Andreas and Wirzberger, Maria}, year = {2024}, booktitle = {Proc. 31st Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING)}, pages = {9154--9169}, doi = {}, url = {https://aclanthology.org/2024.lrec-main.802/} }

1. 

   Multimodal Integration of Human-Like Attention in Visual Question Answering

   Ekta Sood, Fabian Kögel, Philipp Müller, Dominike Thomas, Mihai Bâce, Andreas Bulling

   Proc. Workshop on Gaze Estimation and Prediction in the Wild (GAZE), CVPRW, pp. 2647–2657, 2023.

   Abstract Links BibTeX Project Tobii Sponsor award, Oral Presentation

   Human-like attention as a supervisory signal to guide neural attention has shown significant promise but is currently limited to uni-modal integration – even for inherently multi-modal tasks such as visual question answering (VQA). We present the Multimodal Human-like Attention Network (MULAN) – the first method for multimodal integration of human-like attention on image and text during training of VQA models. MULAN integrates attention predictions from two state-of-the-art text and image saliency models into neural self-attention layers of a recent transformer-based VQA model. Through evaluations on the challenging VQAv2 dataset, we show that MULAN achieves a new state-of-the-art performance of 73.98% accuracy on test-std and 73.72% on test-dev and, at the same time, has approximately 80% fewer trainable parameters than prior work. Overall, our work underlines the potential of integrating multimodal human-like and neural attention for VQA.

   Paper: sood23_gaze.pdf

   Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/MULAN-VQA/

   Paper Access: https://openaccess.thecvf.com/content/CVPR2023W/GAZE/papers/Sood_Multimodal_Integration_of_Human-Like_Attention_in_Visual_Question_Answering_CVPRW_2023_paper.pdf

   @inproceedings{sood23_gaze, author = {Sood, Ekta and Kögel, Fabian and Müller, Philipp and Thomas, Dominike and Bâce, Mihai and Bulling, Andreas}, title = {Multimodal Integration of Human-Like Attention in Visual Question Answering}, booktitle = {Proc. Workshop on Gaze Estimation and Prediction in the Wild (GAZE), CVPRW}, year = {2023}, pages = {2647--2657}, url = {https://openaccess.thecvf.com/content/CVPR2023W/GAZE/papers/Sood_Multimodal_Integration_of_Human-Like_Attention_in_Visual_Question_Answering_CVPRW_2023_paper.pdf} }

1. 

   VQA-MHUG: A gaze dataset to study multimodal neural attention in VQA

   Ekta Sood, Fabian Kögel, Florian Strohm, Prajit Dhar, Andreas Bulling

   Proc. ACL SIGNLL Conference on Computational Natural Language Learning (CoNLL), pp. 27–43, 2021.

   Abstract Links BibTeX Project Oral Presentation

   We present VQA-MHUG - a novel 49-participant dataset of multimodal human gaze on both images and questions during visual question answering (VQA) collected using a high-speed eye tracker. We use our dataset to analyze the similarity between human and neural attentive strategies learned by five state-of-the-art VQA models: Modulated Co-Attention Network (MCAN) with either grid or region features, Pythia, Bilinear Attention Network (BAN), and the Multimodal Factorized Bilinear Pooling Network (MFB). While prior work has focused on studying the image modality, our analyses show - for the first time - that for all models, higher correlation with human attention on text is a significant predictor of VQA performance. This finding points at a potential for improving VQA performance and, at the same time, calls for further research on neural text attention mechanisms and their integration into architectures for vision and language tasks, including but potentially also beyond VQA.

   doi: 10.18653/v1/2021.conll-1.3

   Paper: sood21_conll.pdf

   Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/vqa-mhug-interpretability

   Dataset: https://perceptualui.org/research/datasets/VQA-MHUG/

   @inproceedings{sood21_conll, title = {VQA-MHUG: A gaze dataset to study multimodal neural attention in VQA}, author = {Sood, Ekta and Kögel, Fabian and Strohm, Florian and Dhar, Prajit and Bulling, Andreas}, booktitle = {Proc. ACL SIGNLL Conference on Computational Natural Language Learning (CoNLL)}, year = {2021}, pages = {27--43}, doi = {10.18653/v1/2021.conll-1.3}, publisher = {Association for Computational Linguistics} }