Yu Xiang's homepage

Biography

Yu Xiang is a Research Scientist at NVIDIA. He received his Ph.D. in electrical engineering from the University of Michigan at Ann Arbor in 2016 advised by Prof. Silvio Savarese. He was a postdoctoral researcher with Prof. Dieter Fox in Computer Science & Engineering at the University of Washington from 2016 to 2017, and was a visiting student researcher in the artificial intelligence lab at Stanford University from 2013 to 2016. He received M.S. degree in computer science from Fudan University in 2010 advised by Prof. Xiangdong Zhou, and B.S. degree in computer science from Fudan University in 2007. (CV, Google Scholar)

Research Interests

My research interests primarily focus on computer vision and perception for robotics. I am interested in studying how can an intelligent system or a robot understand its 3D environment from sensing, which is a very challenging and unsolved problem. Perception serves as an interface between an intelligent system and the 3D world, which provides useful information for planning and control of the system in conducting different tasks. I am interested in integrating perception, planning and control in a systematic way and deploying robots in the real world which are capable of accomplishing tasks for human. I apply machine learning, especially deep learning, to tackle the challenges in robot perception. I explore how to introduce domain knowledge such as geometric constraints into a deep neural network architecture to learn a useful representation of the 3D environment for perception. I am also interested in how to learn a joint representation for perception, planning and control with deep neural networks.

News

• 7/3/2018 Our work on Deep Iterative Matching for 6D Pose Estimation is accepted to ECCV 2018 as oral presentation!
• 4/12/2018 Our work on PoseCNN is accepted to RSS 2018!
• 1/1/2018 I join the Robotics Research Lab at NVIDIA.
• 10/20/2017 Our work on online multi-object tracking is accepted to WACV 2018!
• 4/30/2017 Our work on DA-RNN is accepted to RSS 2017!
• 12/20/2016 Our work on SubCNN is accepted to WACV 2017!
• 10/28/2016 We organized the 3D Object Geometry from Single Image tutorial at 3DV 2016.
• 7/19/2016 Two papers related to 3D object recognition are accepted to ECCV 2016!
• 6/10/2016 I am joining Prof. Dieter Fox's group as a postdoc in August!
• 3/6/2016 Our work on deep metric learning is accepted to CVPR 2016!
• 2/1/2016 I started as a Postdoctoral Researcher at Stanford University.
• 12/4/2015 I successfully defended my doctoral thesis!
• 9/4/2015 Call for papers: 5th Workshop on 3D Representation and Recognition (3dRR-15) in ICCV 2015
• 8/29/2015 Our work on multi-object tracking with MDP is accepted to ICCV 2015 as oral presentation!
• 6/22/2015 I started a 3-month internship at NEC Labs America in Cupertino.
• 3/8/2015 Two papers accepted in CVPR 2015! 3DVP is accepted as oral presentation!
• 1/23/2015 Finish my thesis proposal: 3D Object Representations for Recognition.
• 6/15/2014 Our work on multiview object tracking is accepted to ECCV 2014!
• 5/18/2014 PASCAL3D+ version 1.1 is available now! Check it out to see how it can benefit your research!
• 5/5/2014 I started a 3-month internship at NEC Labs America in Cupertino.
• 2/15/2014 Our PASCAL3D+ benchmark (version 1.0) is released!
• 1/30/2014 Our work on building a large scale dataset for 3D object detection and pose estimation is accepted to WACV 2014!
• 10/5/2013 Our paper is accepted to the 3dRR workshop in conjunction with ICCV 2013.
• 9/1/2013 I moved to Stanford as a visiting student.
• Our paper has been accepted to the ECCV 2012 conference.
• I received the Outstanding Master's Thesis Award of Shanghai.
  
• Our CVPR 2012 paper and code are available!

Publications

	DeepIM: Deep Iterative Matching for 6D Pose Estimation Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang and Dieter Fox In European Conference on Computer Vision (ECCV), 2018. arXiv, PDF, Bibtex, Technical_Report, Project (Oral) @inproceedings{li2017deepim, author = {Yi Li and Gu Wang and Xiangyang Ji and Yu Xiang and Dieter Fox}, title = {DeepIM: Deep Iterative Matching for 6D Pose Estimation}, booktitle = {European Conference Computer Vision (ECCV)}, year = {2018} }
	PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes Yu Xiang, Tanner Schmidt, Venkatraman Narayanan and Dieter Fox In Robotics: Science and Systems (RSS), 2018. arXiv, PDF, Bibtex, Code, Project @article{xiang2017posecnn, author = {Xiang, Yu and Schmidt, Tanner and Narayanan, Venkatraman and Fox, Dieter}, title = {PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes}, journal = {Robotics: Science and Systems (RSS)}, year = {2018} }
	Recurrent Autoregressive Networks for Online Multi-Object Tracking Kuan Fang, Yu Xiang, Xiaocheng Li and Silvio Savarese In IEEE Winter Conference on Applications of Computer Vision (WACV), 2018. arXiv, PDF, Bibtex, Poster, Slides @inproceedings{fang2018recurrent, author = {Fang, Kuan and Xiang, Yu and Li, Xiaocheng and Savarese, Silvio}, title = {Recurrent Autoregressive Networks for Online Multi-Object Tracking}, booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)}, year = {2018} }

2017

DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks Yu Xiang and Dieter Fox In Robotics: Science and Systems (RSS), 2017. arXiv, PDF, Bibtex, Poster, Slides, Code, Project @incollection{xiang2017darnn, author = {Xiang, Yu and Fox, Dieter}, title = {DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks}, booktitle = {Robotics: Science and Systems (RSS)}, year = {2017} }

Subcategory-aware Convolutional Neural Networks for Object Proposals and Detection Yu Xiang, Wongun Choi, Yuanqing Lin and Silvio Savarese In IEEE Winter Conference on Applications of Computer Vision (WACV), 2017. arXiv, PDF, Bibtex, Technical_Report, Poster, Slides, KITTI_Results @inproceedings{xiang2017subcategory, author = {Xiang, Yu and Choi, Wongun and Lin, Yuanqing and Savarese, Silvio}, title = {Subcategory-aware Convolutional Neural Networks for Object Proposals and Detection}, booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)}, year = {2017} }

2016

	Anticipating Accidents in Dashcam Videos Fu-Hsiang Chan, Yu-Ting Chen, Yu Xiang and Min Sun In Asian Conference on Computer Vision (ACCV), 2016. PDF, Bibtex, Project (Oral) @inproceedings{chan2016anticipating, title = {Anticipating Accidents in Dashcam Videos}, author = {Chan, Fu-Hsiang and Chen, Yu-Ting and Xiang, Yu and Sun, Min}, booktitle = {Asian Conference Computer Vision (ACCV)}, year = {2016} }
	ObjectNet3D: A Large Scale Database for 3D Object Recognition Yu Xiang, Wonhui Kim, Wei Chen, Jingwei Ji, Christopher Choy, Hao Su, Roozbeh Mottaghi, Leonidas Guibas and Silvio Savarese In European Conference on Computer Vision (ECCV), pp. 160-176, 2016. PDF, Bibtex, Technical_Report, Poster, Slides, ObjectNet3D (Spotlight Oral) @inproceedings{xiang2016objectnet3d, title = {ObjectNet3D: A Large Scale Database for 3D Object Recognition}, author = {Xiang, Yu and Kim, Wonhui and Chen, Wei and Ji, Jingwei and Choy, Christopher and Su, Hao and Mottaghi, Roozbeh and Guibas, Leonidas and Savarese, Silvio}, booktitle = {European Conference Computer Vision (ECCV)}, pages = {160--176}, year = {2016} }
	Pose Estimation Errors, the Ultimate Diagnosis Carolina Redondo-Cabrera, Roberto López-Sastre, Yu Xiang, Tinne Tuytelaars and Silvio Savarese In European Conference on Computer Vision (ECCV), pp. 118-134, 2016. PDF, Bibtex, Code @inproceedings{cabrera2016pose, title = {Pose Estimation Errors, the Ultimate Diagnosis}, author = {Redondo-Cabrera, Carolina and L\'{o}pez-Sastre, Roberto and Xiang, Yu and Tuytelaars, Tinne and Savarese, Silvio}, booktitle = {European Conference Computer Vision (ECCV)}, pages = {118--134}, year = {2016} }
	Deep Metric Learning via Lifted Structured Feature Embedding Hyun Oh Song, Yu Xiang, Stefanie Jegelka and Silvio Savarese In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4004-4012, 2016. arXiv, PDF, Bibtex, Technical_Report, Code, Project (Spotlight Oral) @inproceedings{song2016deep, author = {Song, Hyun Oh and Xiang, Yu and Jegelka, Stefanie and Savarese, Silvio}, title = {Deep Metric Learning via Lifted Structured Feature Embedding}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, pages = {4004--4012}, year = {2016} }

2015

	Learning to Track: Online Multi-Object Tracking by Decision Making Yu Xiang, Alexandre Alahi and Silvio Savarese In International Conference on Computer Vision (ICCV), pp. 4705-4713, 2015. PDF, Bibtex, Technical_Report, Poster, Slides, MOT_Results, KITTI_Results, Code, Project (Oral) @inproceedings{xiang2015learning, author = {Xiang, Yu and Alahi, Alexandre and Savarese, Silvio}, title = {Learning to Track: Online Multi-Object Tracking by Decision Making}, booktitle = {International Conference on Computer Vision (ICCV)}, pages = {4705--4713}, year = {2015} }
	Data-Driven 3D Voxel Patterns for Object Category Recognition Yu Xiang, Wongun Choi, Yuanqing Lin and Silvio Savarese In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1903-1911, 2015. PDF, Bibtex, Technical_Report, Poster, Slides, KITTI_Results, Code, Project (Oral) @inproceedings{xiang2015data, title = {Data-Driven 3D Voxel Patterns for Object Category Recognition}, author = {Xiang, Yu and Choi, Wongun and Lin, Yuanqing and Savarese, Silvio}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, pages = {1903--1911}, year = {2015} }
	A Coarse-to-Fine Model for 3D Pose Estimation and Sub-category Recognition Roozbeh Mottaghi, Yu Xiang and Silvio Savarese In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 418-426, 2015. PDF, Bibtex, Technical_Report, Poster, Project @inproceedings{mottaghi2015coarse, title = {A Coarse-to-Fine Model for 3D Pose Estimation and Sub-Category Recognition}, author = {Mottaghi, Roozbeh and Xiang, Yu and Savarese, Silvio}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, pages = {418--426}, year = {2015} }

2014

Monocular Multiview Object Tracking with 3D Aspect Parts Yu Xiang*, Changkyu Song*, Roozbeh Mottaghi and Silvio Savarese (*equal contribution) In European Conference on Computer Vision (ECCV), pp. 220-235, 2014. PDF, Bibtex, Technical_Report, Poster, Slides, Code, Project @inproceedings{xiang2014monocular, title = {Monocular multiview object tracking with 3d aspect parts}, author = {Xiang, Yu and Song, Changkyu and Mottaghi, Roozbeh and Savarese, Silvio}, booktitle = {European Conference Computer Vision (ECCV)}, pages = {220--235}, year = {2014} }

Beyond PASCAL: A Benchmark for 3D Object Detection in the Wild Yu Xiang, Roozbeh Mottaghi and Silvio Savarese In IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 75-82, 2014. PDF, Bibtex, Poster, Slides, PASCAL3D+ @inproceedings{xiang2014beyond, title = {Beyond pascal: A benchmark for 3d object detection in the wild}, author = {Xiang, Yu and Mottaghi, Roozbeh and Savarese, Silvio}, booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)}, pages = {75--82}, year = {2014} }

2013

Object Detection by 3D Aspectlets and Occlusion Reasoning Yu Xiang and Silvio Savarese In the 4th International IEEE Workshop on 3D Representation and Recognition in ICCV (3dRR), pp. 530-537, 2013. PDF, Bibtex, Technical_Report, Slides, Code, Project @inproceedings{xiang2013object, title = {Object detection by 3d aspectlets and occlusion reasoning}, author = {Xiang, Yu and Savarese, Silvio}, booktitle = {IEEE International Conference on Computer Vision Workshops (ICCVW)}, pages = {530--537}, year = {2013} }

2012

Object Co-detection Sid Yingze Bao, Yu Xiang and Silvio Savarese In European Conference on Computer Vision (ECCV), vol. 7572, pp. 86-101, 2012. PDF, Bibtex, Poster, Slides, Project @inproceedings{bao2012object, title = {Object co-detection}, author = {Bao, Sid Yingze and Xiang, Yu and Savarese, Silvio}, booktitle = {European Conference Computer Vision (ECCV)}, pages = {86--101}, year = {2012} }

Estimating the Aspect Layout of Object Categories Yu Xiang and Silvio Savarese In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3410-3417, 2012. PDF, Bibtex, Technical Report, Poster, Slides, Code, Project @inproceedings{xiang2012estimating, title = {Estimating the aspect layout of object categories}, author = {Xiang, Yu and Savarese, Silvio}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, pages = {3410--3417}, year = {2012} }

2010

Semantic Context Modeling with Maximal Margin Conditional Random Fields for Automatic Image Annotation Yu Xiang, Xiangdong Zhou, Zuotao Liu, Tat-Seng Chua and Chong-Wah Ngo In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3368-3375, 2010. PDF, Bibtex, Technical Report @inproceedings{xiang2010semantic, title = {Semantic context modeling with maximal margin conditional random fields for automatic image annotation}, author = {Xiang, Yu and Zhou, Xiangdong and Liu, Zuotao and Chua, Tat-Seng and Ngo, Chong-Wah}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, pages = {3368--3375}, year = {2010} }

Learning Contextual Metrics for Automatic Image Annotation Zuotao Liu, Xiangdong Zhou, Yu Xiang and Yan-Tao Zheng In Advances in Multimedia Information Processing - PCM, vol. 6297, pp. 124-135, 2010. PDF, Bibtex @inproceedings{liu2010learning, title = {Learning contextual metrics for automatic image annotation}, author = {Liu, Zuotao and Zhou, Xiangdong and Xiang, Yu and Zheng, Yan-Tao}, booktitle = {Advances in Multimedia Information Processing-PCM}, pages = {124--135}, year = {2010} }

2009

	A Revisit of Generative Model for Automatic Image Annotation using Markov Random Fields Yu Xiang, Xiangdong Zhou, Tat-Seng Chua and Chong-Wah Ngo In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1153-1160, 2009. PDF, Bibtex @inproceedings{xiang2009revisit, title = {A revisit of generative model for automatic image annotation using markov random fields}, author = {Xiang, Yu and Zhou, Xiangdong and Chua, Tat-Seng and Ngo, Chong-Wah}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, pages = {1153--1160}, year = {2009} }
	Adaptive Model for Web Image Semantic Automatic Image Annotation Hongtao Xu, Xiangdong Zhou, Yu Xiang and Baile Shi In Journal of Software (in Chinese), vol. 21, no. 9, pp. 2183-2195, 2009. PDF, Bibtex @article{xu2009adaptive, author = {Hongtao Xu and Xiangdong Zhou and Yu Xiang and Baile Shi}, title = {Adaptive model for web image semantic automatic image annotation}, booktitle = {Journal of Software (in Chinese)}, volume = {21}, number = {9}, pages = {2183-2195}, year = {2009} }
	Exploiting Flickr's Related Tags for Semantic Annotation of Web Images Hongtao Xu, Xiangdong Zhou, Mei Wang, Yu Xiang and Baile Shi In Proceedings of the ACM International Conference on Image and Video Retrieval (CIVR), no. 46, 2009. PDF, Bibtex @inproceedings{xu2009exploring, title = {Exploring Flickr's related tags for semantic annotation of web images}, author = {Xu, Hongtao and Zhou, Xiangdong and Wang, Mei and Xiang, Yu and Shi, Baile}, booktitle = {ACM International Conference on Image and Video Retrieval (CIVR)}, pages = {46:1--46:8}, year = {2009} }
	Automatic Web Image Annotation via Web-Scale Image Semantic Space Learning Hongtao Xu, Xiangdong Zhou, Lan Lin, Yu Xiang and Baile Shi In Advances in Data and Web Management, vol. 5446, pp. 211-222, 2009. PDF, Bibtex @inproceedings{xu2009automatic, title = {Automatic web image annotation via web-scale image semantic space learning}, author = {Xu, Hongtao and Zhou, Xiangdong and Lin, Lan and Xiang, Yu and Shi, Baile}, booktitle = {Advances in Data and Web Management}, pages = {211--222}, year = {2009} }

PhD Thesis

• 3D Object Representations for Recognition (PDF)
  University of Michigan, PhD thesis, 2016.

Master Thesis

• Graphical Models for Semantic Context Modeling in Automatic Image Annotation (PDF)
  Fudan University, Master thesis (in Chinese), Outstanding Master's Thesis Award of Shanghai, 2010.

Talks

• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (PDF)
  In Robotics: Science and Systems (RSS), CMU, Pittsburgh, Pennsylvania, 6/26/2018.


• Perceiving the 3D World from Images and Videos (PDF)
  Nvidia Research, Redmond, Washington, 11/07/2017; University of Michigan, 3/15/2018.


• 3D Object Recognition and Scene Understanding from RGB-D Videos (PDF)
  GRASP Lab at Penn, 10/11/2017; Microsoft Research, 10/17/2017; Vision Lab at Stanford, 10/23/2017.


• 3D Object Recognition and Scene Understanding (PDF)
  In Mitsubishi Electric Research Laboratories, Boston, Massachusetts, 7/14/2017.


• DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks (PDF)
  In Robotics: Science and Systems (RSS), MIT, Massachusetts, 7/13/2017.


• Subcategory-aware Convolutional Neural Networks for Object Proposals and Detection (PDF)
  In IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, California, 3/29/2017.


• 3D Object Recognition (PDF)
  In the International Conference on 3D Vision, Stanford University, 10/28/2016.


• 3D Object Representations for Recognition (PDF)
  VASC Seminar, CMU, 3/28/2016; University of Toronto, 4/4/2016; MIT, 4/12/2016; UC Berkeley, 4/21/2016; UIUC, 5/5/2016; University of Washington, 5/31/2016.


• 3D Object Detection and Pose Estimation (PDF)
  In the 1st International Workshop on Recovering 6D Object Pose in conjunction with ICCV, Santiago, Chile, 12/17/2015.


• Learning to Track: Online Multi-Object Tracking by Decision Making (PDF)
  In International Conference on Computer Vision, Santiago, Chile, 12/16/2015.


• Data-Driven 3D Voxel Patterns for Object Category Recognition (PDF)
  In IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, 06/08/2015.


• Monocular Multiview Object Tracking with 3D Aspect Parts (PDF)
  In the 1st Stanford-SNU Workshop on Automated Driving, Stanford University, 02/24/2015.


• Beyond PASCAL: A Benchmark for 3D Object Detection in the Wild (PDF)
  In IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, Colorado, 03/24/2014.


• Object Detection by 3D Aspectlets and Occlusion Reasoning (PDF)
  In the 4th International IEEE Workshop on 3D Representation and Recognition in conjunction with ICCV, Sydney, Australia, 12/08/2013.


• Estimating the Aspect Layout of Object Categories (PDF)
  In Midwest Vision Workshop, University of Illinois at Urbana-Champaign, 09/21/2012.

Links

• Robotics and State Estimation Lab at the University of Washington
• Computational Vision and Geometry Lab at Stanford University
• My Google Scholar Citations
• My github