Chuhan Jiao

Institute for Visualisation and Interactive Systems, University of Stuttgart

  Pfaffenwaldring 5a, 70569 Stuttgart, Germany University of Stuttgart, SimTech Building, Room 01.035 Google Scholar

Biography

Chuhan Jiao is a PhD student in the Collaborative Artificial Intelligence group since December 2022. He holds a Bachelor's degree in Computer Science and Technology from Donghua University, and a Master's degree (with honours) in Computer Science from Aalto University. His research interest lies at the intersection of computer vision and human-computer interaction. He now works on Unsupervised Learning of Deep Gaze Behavior Representations within MSCA Doctoral Network Eyes4ICU. He did a research stay at ETH in with Peter Kiefer in October 2024.

Teaching

2024

Machine Perception and Learning (Tutor)Master

2023

Machine Perception and Learning (Tutor)Master

---

Supervision

2024

Multimodal User Authentication for Head-mounted Devices (M.Sc.) Yuxuan Cheng

2023

Offline Resumption Detection in Reading (B.Sc.) Abel Gitzing+

+Co-supervised with Francesca Zermiani and Ekta Sood

---

Open Thesis Projects

 

Gaze and Head Redirection with Implicit Neural RepresentationsMaster

---

Publications

(Jiao et al., 2024; Jiao et al., 2024; Jiao et al., 2023; Wang et al., 2022)

1. 

   DiffEyeSyn: Diffusion-based User-specific Eye Movement Synthesis

   Chuhan Jiao, Guanhua Zhang, Zhiming Hu, Andreas Bulling

   arXiv:2409.01240, pp. 1–22, 2024.

   Abstract Links BibTeX Project

   High-frequency components in eye gaze data contain user-specific information promising for various applications, but existing gaze modelling methods focus on low frequencies of typically not more than 30 Hz. We present DiffEyeSyn – the first computational method to synthesise high-frequency gaze data, including eye movement characteristics specific to individual users. The key idea is to consider the high-frequency, user-specific information as a special type of noise in eye movement data. This perspective reshapes eye movement synthesis into the task of injecting this user-specific noise into any given eye movement sequence. We formulate this injection task as a conditional diffusion process in which the synthesis is conditioned on user-specific embeddings extracted from the gaze data using pre-trained models for user authentication. We propose user identity guidance – a novel loss function that allows our model to preserve user identity while generating human-like eye movements in the spatial domain. Experiment results on two public high-frequency eye movement biometric datasets show that our synthetic eye movements are indistinguishable from real human eye movements. Furthermore, we demonstrate that DiffEyeSyn can be used to synthesise eye gaze data at scale and for different downstream tasks, such as gaze data imputation and gaze data super-resolution. As such, our work lays the methodological foundations for personalised eye movement synthesis that has significant application potential, such as for character animation, eye movement biometrics, or gaze-based activity and context recognition.

   Paper Access: https://arxiv.org/abs/2409.01240

   @techreport{jiao24_arxiv_2, title = {DiffEyeSyn: Diffusion-based User-specific Eye Movement Synthesis}, author = {Jiao, Chuhan and Zhang, Guanhua and Hu, Zhiming and Bulling, Andreas}, year = {2024}, pages = {1--22}, url = {https://arxiv.org/abs/2409.01240} }
2. 

   DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images

   Chuhan Jiao, Yao Wang, Guanhua Zhang, Mihai Bâce, Zhiming Hu, Andreas Bulling

   arXiv:2403.17477, pp. 1–13, 2024.

   Abstract Links BibTeX Project

   We present DiffGaze, a novel method for generating realistic and diverse continuous human gaze sequences on 360° images based on a conditional score-based denoising diffusion model. Generating human gaze on 360° images is important for various human-computer interaction and computer graphics applications, e.g. for creating large-scale eye tracking datasets or for realistic animation of virtual humans. However, existing methods are limited to predicting discrete fixation sequences or aggregated saliency maps, thereby neglecting crucial parts of natural gaze behaviour. Our method uses features extracted from 360° images as condition and uses two transformers to model the temporal and spatial dependencies of continuous human gaze. We evaluate DiffGaze on two 360° image benchmarks for gaze sequence generation as well as scanpath prediction and saliency prediction. Our evaluations show that DiffGaze outperforms state-of-the-art methods on all tasks on both benchmarks. We also report a 21-participant user study showing that our method generates gaze sequences that are indistinguishable from real human sequences. Taken together, our evaluations not only demonstrate the effectiveness of DiffGaze but also point towards a new generation of methods that faithfully model the rich spatial and temporal nature of natural human gaze behaviour.

   Paper Access: https://arxiv.org/abs/2403.17477

   @techreport{jiao24_arxiv, title = {DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images}, author = {Jiao, Chuhan and Wang, Yao and Zhang, Guanhua and B{\^a}ce, Mihai and Hu, Zhiming and Bulling, Andreas}, year = {2024}, pages = {1--13}, url = {https://arxiv.org/abs/2403.17477} }

1. 

   SUPREYES: SUPer Resolution for EYES Using Implicit Neural Representation Learning

   Chuhan Jiao, Zhiming Hu, Mihai Bâce, Andreas Bulling

   Proc. ACM Symposium on User Interface Software and Technology (UIST), pp. 1–13, 2023.

   Abstract Links BibTeX Project

   We introduce SUPREYES – a novel self-supervised method to increase the spatio-temporal resolution of gaze data recorded using low(er)-resolution eye trackers. Despite continuing advances in eye tracking technology, the vast majority of current eye trackers – particularly mobile ones and those integrated into mobile devices – suffer from low-resolution gaze data, thus fundamentally limiting their practical usefulness. SUPREYES learns a continuous implicit neural representation from low-resolution gaze data to up-sample the gaze data to arbitrary resolutions. We compare our method with commonly used interpolation methods on arbitrary scale super-resolution and demonstrate that SUPREYES outperforms these baselines by a significant margin. We also test on the sample downstream task of gaze-based user identification and show that our method improves the performance of original low-resolution gaze data and outperforms other baselines. These results are promising as they open up a new direction for increasing eye tracking fidelity as well as enabling new gaze-based applications without the need for new eye tracking equipment.

   doi: 10.1145/3586183.3606780

   Paper: jiao23_uist.pdf

   @inproceedings{jiao23_uist, author = {Jiao, Chuhan and Hu, Zhiming and B{\^a}ce, Mihai and Bulling, Andreas}, title = {SUPREYES: SUPer Resolution for EYES Using Implicit Neural Representation Learning}, booktitle = {Proc. ACM Symposium on User Interface Software and Technology (UIST)}, year = {2023}, pages = {1--13}, doi = {10.1145/3586183.3606780} }

1. 

   VisRecall: Quantifying Information Visualisation Recallability via Question Answering

   Yao Wang, Chuhan Jiao, Mihai Bâce, Andreas Bulling

   IEEE Transactions on Visualization and Computer Graphics (TVCG), 28 (12), pp. 4995-5005, 2022.

   Abstract Links BibTeX Project

   Despite its importance for assessing the effectiveness of communicating information visually, fine-grained recallability of information visualisations has not been studied quantitatively so far. In this work, we propose a question-answering paradigm to study visualisation recallability and present VisRecall - a novel dataset consisting of 200 visualisations that are annotated with crowd-sourced human (N = 305) recallability scores obtained from 1,000 questions of five question types. Furthermore, we present the first computational method to predict recallability of different visualisation elements, such as the title or specific data values. We report detailed analyses of our method on VisRecall and demonstrate that it outperforms several baselines in overall recallability and FE-, F-, RV-, and U-question recallability. Our work makes fundamental contributions towards a new generation of methods to assist designers in optimising visualisations.

   doi: 10.1109/TVCG.2022.3198163

   Paper: wang22_tvcg.pdf

   Supplementary Material: wang22_tvcg_sup.pdf

   Dataset: https://darus.uni-stuttgart.de/dataset.xhtml?persistentId=doi:10.18419/darus-2826

   @article{wang22_tvcg, title = {VisRecall: Quantifying Information Visualisation Recallability via Question Answering}, author = {Wang, Yao and Jiao, Chuhan and Bâce, Mihai and Bulling, Andreas}, year = {2022}, journal = {IEEE Transactions on Visualization and Computer Graphics (TVCG)}, volume = {28}, number = {12}, pages = {4995-5005}, doi = {10.1109/TVCG.2022.3198163} }