Advances in 3D Reconstruction

CSE 291 Topics in Computer Vision, Winter 2019

Instructor: Manmohan Chandraker
Email: mkchandraker [AT] eng [DOT] ucsd [DOT] edu

Lectures: WF 5-6:30pm in EBU3B 2154
Instructor office hours: Th 3-4pm in CSE 4122

TA: Dewal Gupta (
TA office hours: M 5-6pm in EBU3B B215


The field of 3D reconstruction has made tremendous progress in recent years. Building upon the foundations of correspondence estimation and multiview geometry, systems have been built that now reconstruct entire cities or autonomously navigate complex environments. Yet, significant challenges and unsolved problems remain, for instance, in generalizing to complex materials, textureless scenes, fast motions, non-rigid deformations or semantic concepts. On the other hand, the advent of deep learning frameworks has led to significant gains in image classification and recognition. This has opened the possibility of translating those advances to tackle some of the above challenges in 3D reconstruction. In this course, we will review some of the fundamental concepts that play a role in 3D reconstruction and study recent developments that enhance them or use them in novel ways. Consistent with the recent advances in computer vision, several of our topics will exploit frameworks based on deep learning.


This is an advanced class, covering recent developments in computer vision research. Students at all levels including undergraduates, masters and PhD, with a strong interest in computer vision may enroll. Prior background in computer vision and machine learning is required, preferably through research experience or as covered by CSE 152, 252 and similar offerings. Students are encouraged to contact the instructor if unsure about meeting any criteria for enrollment.

Course Format and Requirements

The course will primarily involve presenting and discussing recent papers related to 3D reconstruction. The papers will be grouped by concept. These will be supported with lectures by the instructor to provide historical background or introduce fundamentals. Students will be required to do a project on a focused topic. Wide flexibility is available for the choice of topic, but should be discussed with the instructor beforehand. The instructor can also provide a topic for group prpjects. The best projects may consist of conceptual development that pushes the boundaries of the state-of-art, or a practical implementation that goes beyond the ones associated with the papers read in class.

Students may take the class with a letter grade option, which requires one or two in-class presentations (depending on enrollment strength), participation in discussions and completion of the project. Alternately, a 1 unit, S-U or P-NP registration is available, which require in-class presentations and discussions, but not the project. You may also audit, but a registration is preferred since only one or two presentations are required.

Participation has two components: reviews and in-class discussions. The day before every class that covers papers, a brief review of one of the papers to be discussed must be sent to the instructor. Besides, students are expected to actively particpiate in discussions in the class.

Grades will be weighted as 20% for reviews, 30% for in-class presentations, 10% for participation and 40% for the final project. The final project should be in groups of four. The goal of the course is to develop an understanding of the current state of 3D reconstruction and gain appreciation of its limits and potential.


The course will cover a diverse range of topics in 3D computer vision. An inexhaustive list includes:


Jan 09: Lecture Jan 11: Lecture Jan 16: Correspondence (Presenter: Manmohan Chandraker) Jan 18: Keypoint detectors (Presenter: Manmohan Chandraker) Jan 23: Learning for SFM Jan 25: Robust estimation - I Jan 30: Robust estimation - II Feb 01: Bundle adjustment Feb 06: Visual odometry Feb 08: Learnable representations for SLAM Feb 13: Multiview stereo Feb 15: Object pose Feb 20: Face reconstruction Feb 22: Project mid-term presentations

Feb 27: Face generation Mar 01: Human pose estimation Mar 06: Human pose generation


Manmohan Chandraker
Last modified: Tue, Jan 08, 2019