Paper accepted at ISMAR 2024

We are pleased to announce that the following paper was accepted for publication in the 23rd IEEE International Symposium on Mixed and Augmented Reality (ISMAR), which will be published in a special issue of the IEEE Transactions on Visualization and Computer Graphics (TVCG).

Congratulations to the authors!

1. 

   HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxes

   Zhiming Hu, Zheming Yin, Daniel Haeufle, Syn Schmitt, Andreas Bulling

   IEEE Transactions on Visualization and Computer Graphics (TVCG), , pp. 1–11, 2024.

   Abstract Links BibTeX Project Best Journal Paper Award

   We present HOIMotion – a novel approach for human motion forecasting during human-object interactions that integrates information about past body poses and egocentric 3D object bounding boxes. Human motion forecasting is important in many augmented reality applications but most existing methods have only used past body poses to predict future motion. HOIMotion first uses an encoder-residual graph convolutional network (GCN) and multi-layer perceptrons to extract features from body poses and egocentric 3D object bounding boxes, respectively. Our method then fuses pose and object features into a novel pose-object graph and uses a residual-decoder GCN to forecast future body motion. We extensively evaluate our method on the Aria digital twin (ADT) and MoGaze datasets and show that HOIMotion consistently outperforms state-of-the-art methods by a large margin of up to 8.7% on ADT and 7.2% on MoGaze in terms of mean per joint position error. Complementing these evaluations, we report a human study (N=20) that shows that the improvements achieved by our method result in forecasted poses being perceived as both more precise and more realistic than those of existing methods. Taken together, these results reveal the significant information content available in egocentric 3D object bounding boxes for human motion forecasting and the effectiveness of our method in exploiting this information.

   Paper: hu24_ismar.pdf

   @article{hu24_ismar, author = {Hu, Zhiming and Yin, Zheming and Haeufle, Daniel and Schmitt, Syn and Bulling, Andreas}, title = {HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxes}, journal = {IEEE Transactions on Visualization and Computer Graphics (TVCG)}, year = {2024}, pages = {1--11} }

Here are some related news you might like to read next:

• Best Journal Paper Award at ISMAR
• Paper accepted in Neural Computing and Applications
• Paper accepted in Frontiers in Cognition
• Paper accepted in TVCG
• Paper accepted at PACM HCI