Advances in 3D Reconstruction

CSE 291 Topics in Computer Vision, Winter 2017

Instructor: Manmohan Chandraker

Time and Place: TuTh 8-9:20pm in WLH 2110


The field of 3D reconstruction has made tremendous progress in recent years. Building upon the foundations of correspondence estimation and multiview geometry, systems have been built that now reconstruct entire cities or autonomously navigate complex environments. Yet, significant challenges and unsolved problems remain, for instance, in generalizing to complex materials, textureless scenes, fast motions, non-rigid deformations or semantic concepts. On the other hand, the advent of deep learning frameworks has led to significant gains in image classification and recognition. This has opened the possibility of translating those advances to tackle some of the above challenges in 3D reconstruction. In this course, we will review some of the fundamental concepts that play a role in 3D reconstruction and study recent developments that enhance them or use them in novel ways. Consistent with the recent advances in computer vision, several of our topics will exploit frameworks based on deep learning.


This is an advanced class, covering recent developments in computer vision research. Students at all levels including undergraduates, masters and PhD, with a strong interest in computer vision may enroll. Prior background in computer vision and machine learning is required, preferably through research experience or as covered by CSE 152, 252 and similar offerings. Students are encouraged to contact the instructor if unsure about meeting any criteria for enrollment.

Course Format and Requirements

The course will primarily involve presenting and discussing recent papers related to 3D reconstruction. The papers will be grouped by concept. These will be supported with lectures by the instructor to provide historical background or introduce fundamentals. Students will be required to do a project or write a critical analysis on a focused topic. Wide flexibility is available for the choice of topic, but should be discussed with the instructor beforehand. The best analyses or projects may consist of conceptual development that pushes the boundaries of the state-of-art, or a practical implementation that goes beyond the ones associated with the papers read in class.

Students may take the class with a letter grade option, which requires one or two in-class presentations (depending on enrollment strength), participation in discussions and completion of the project. Alternately, a 1 unit, S-U or P-NP registration is available, which require in-class presentations and discussions, but not the project. You may also audit, but a registration is preferred since only one or two presentations are required.

Participation has two components: reviews and in-class discussions. The day before every class that covers papers, a brief review of one of the papers to be discussed must be sent to the instructor. Besides, students are expected to actively particpiate in discussions in the class.

Grades will be weighted as 40% for in-class presentations, 20% for participation and 40% for the final project. The final project may be in groups of two. The goal of the course is to develop an understanding of the current state of 3D reconstruction and gain appreciation of its limits and potential. In general, students who fulfil the requirements may easily receive an A in the course.


The course will cover a diverse range of topics in 3D computer vision. An inexhaustive list includes:


Jan 10: Lecture Jan 12: Lecture Jan 17: Correspondence estimation (Presenter: Meng Song) [PDF] Jan 19: No class

Jan 24: Similarity and metric learning for correspondences (Presenter: Zhengqin Li) [PDF] Jan 26: Lecture Jan 31: Weakly supervised correspondence estimation (Presenter: Zhiwei Jia) [PDF] Feb 07: Binocular stereo (Presenter: Yangyue Wan) [PDF] Feb 09: Viewpoint estimation (Presenter: Mo Shan) [PDF] Feb 14: Single-view depth estimation (Presenter: Xiangyun Zhao) [PDF] Feb 16: Optical flow (Presenter: Nimish Srivastava) [PDF] Feb 21: Real-time SFM systems (Presenter: Sudhanshu Behaty) [PDF] Feb 23: Real-time RGB-D systems (Presenter: Sudhanshu Behaty) [PDF] Feb 28: View synthesis (Presenter: Shradha Agrawal) [PDF] Mar 02: Light Fields (Guest Lecture: Nima Khademi Kalantari) [PDF] Mar 07: Face Alignment and Reconstruction (Presenter: Ajitesh Gupta) [PDF] Mar 09: Human Pose Estimation (Presenter: Ranti Dev Sharma) [PDF] Mar 14: Reflectance and Illumination Estimation (Presenter: Jingwen Wang) [PDF] Mar 16: Indoor Semantic Layout (Presenter: Kwokfung Tang and Akshay Rangesh) [PDF] [PDF]


Manmohan Chandraker
Last modified: Thu, Feb 23, 2017