Yao Wang

Institute for Visualisation and Interactive Systems, University of Stuttgart

+49 711 685 Pfaffenwaldring 5a, 70569 Stuttgart, Germany University of Stuttgart, SimTech Building, Room 01.031 Google Scholar

Biography

Yao Wang is a PhD student in the Perceptual User Interfaces group since September 2020. He holds a Bachelor's degree in Intelligence Science and Technology and a Master’s degree in Computer Science from Peking University. During his Master’s studies he specialized in Image Processing and 3D Reconstruction. His research interest now focuses on Visual Attention Modeling for abstract data visualizations.

Teaching

2021

Mensch-Computer-Interaktion (Teaching Assistant)Bachelor

---

Supervision

2021

Joint Learning Model for Saliency and Scanpath Prediction (M.Sc.) Jiaqi Qin

 

QA Multi Duration Saliency Dataset Annotation on Data Visualizations by CodeCharts UI (HiWi Project) Luwei Liu

 

Deep Learning Engineering in Saliency Modeling (HiWi Project) Wei Zhang

---

Publications

(Wei, Wang, Li, Chen, & Wang, 2020; Hu et al., 2018; Chen, Wang, Lu, Chen, & Wang, 2018)

1. 

   Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images

   Zizhuang Wei, Yao Wang, Hongwei Li, Yisong Chen, Guoping Wang

   Applied Sciences, 10(4), pp. 1275, 2020.

   Abstract Links BibTeX Project

   Semantic modeling is a challenging task that has received widespread attention in recent years. With the help of mini Unmanned Aerial Vehicles (UAVs), multi-view high-resolution aerial images of large-scale scenes can be conveniently collected. In this paper, we propose a semantic Multi-View Stereo (MVS) method to reconstruct 3D semantic models from 2D images. Firstly, 2D semantic probability distribution is obtained by Convolutional Neural Network (CNN). Secondly, the calibrated cameras poses are determined by Structure from Motion (SfM), while the depth maps are estimated by learning MVS. Combining 2D segmentation and 3D geometry information, dense point clouds with semantic labels are generated by a probability-based semantic fusion method. In the final stage, the coarse 3D semantic point cloud is optimized by both local and global refinements. By making full use of the multi-view consistency, the proposed method efficiently produces a fine-level 3D semantic point cloud. The experimental result evaluated by re-projection maps achieves 88.4% Pixel Accuracy on the Urban Drone Dataset (UDD). In conclusion, our graph-based semantic fusion procedure and refinement based on local and global information can suppress and reduce the re-projection error.

   doi: 10.3390/app10041275

   Paper: wei20_as.pdf

   @article{wei20_as, title = {Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images}, author = {Wei, Zizhuang and Wang, Yao and Li, Hongwei and Chen, Yisong and Wang, Guoping}, year = {2020}, journal = {Applied Sciences}, doi = {10.3390/app10041275}, pages = {1275}, volume = {10}, number = {4} }

1. 

   Sobel Heuristic Kernel for Aerial Semantic Segmentation

   Tao Hu, Yao Wang, Yisong Chen, Peng Lu, Heng Wang, Guoping Wang

   IEEE International Conference on Image Processing (ICIP), pp. 3074-3078, 2018.

   Abstract Links BibTeX Project

   Misclassification in semantic segmentation mostly occurs in the pixels around the semantic contour. In this work, we address the task of aerial image segmentation by borrowing the kernel prior from classical edge detecting operator. We propose a module called Sobel Heuristic Kernel(SHK). Our work makes several main contributions and experimentally shows good performance. To the best of our knowledge, we are the first to combine traditional edge detection method and deep learning method in semantic segmentation. Our SHK module reaches state of the art in the Inria Aerial Image Labeling dataset.

   doi: 10.1109/ICIP.2018.8451170

   Paper: hu18_icip.pdf

   @inproceedings{hu18_icip, title = {Sobel Heuristic Kernel for Aerial Semantic Segmentation}, author = {Hu, Tao and Wang, Yao and Chen, Yisong and Lu, Peng and Wang, Heng and Wang, Guoping}, year = {2018}, booktitle = {IEEE International Conference on Image Processing (ICIP)}, doi = {10.1109/ICIP.2018.8451170}, pages = {3074-3078} }
2. 

   Large-Scale Structure from Motion with Semantic Constraints of Aerial Images

   Yu Chen, Yao Wang, Peng Lu, Yisong Chen, Guoping Wang

   Proc. Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 347-359, 2018.

   Abstract Links BibTeX Project

   Structure from Motion (SfM) and semantic segmentation are two branches of computer vision. However, few previous methods integrate the two branches together. SfM is limited by the precision of traditional feature detecting method, especially in complicated scenes. As the research field of semantic segmentation thrives, we could gain semantic information of high confidence in each specific task with little effort. By utilizing semantic segmentation information, our paper presents a new way to boost the accuracy of feature point matching. Besides, with the semantic constraints taken from the result of semantic segmentation, a new bundle adjustment method with equality constraint is proposed. By exploring the sparsity of equality constraint, it indicates that constrained bundle adjustment can be solved by Sequential Quadratic Programming (SQP) efficiently. The proposed approach achieves state of the art accuracy, and, by grouping the descriptors together by their semantic labels, the speed of putative matches is slightly boosted. Moreover, our approach demonstrates a potential of automatic labeling of semantic segmentation. In a nutshell, our work strongly verifies that SfM and semantic segmentation benefit from each other.

   doi: 10.1007/978-3-030-03398-9_30

   Paper: chen18_prcv.pdf

   @inproceedings{chen18_prcv, title = {Large-Scale Structure from Motion with Semantic Constraints of Aerial Images}, author = {Chen, Yu and Wang, Yao and Lu, Peng and Chen, Yisong and Wang, Guoping}, year = {2018}, booktitle = {Proc. Chinese Conference on Pattern Recognition and Computer Vision (PRCV)}, doi = {10.1007/978-3-030-03398-9_30}, pages = {347-359} }