All Publications

My photo  Single-Shot Neural Relighting and SVBRDF Estimation
Shen Sang, Manmohan Chandraker
ECCV 2020      [PDF]

A physically-motivated network for joint shape and material estimation, as well as relighting under novel illumination conditions, using a single image captured by a mobile phone camera.

My photo  Single-View Metrology in the Wild
Rui Zhu, Xingyi Wang, Yannick Hold-Geoffroy, Federico Perazzi, Jonathan Eisenmann, Kalyan Sunkavalli, Manmohan Chandraker
ECCV 2020      [PDF]

Recover object height and camera parameters from a single unconstrained image, through a network that imbibes weakly supervised geometric constraints through bounding boxes and the horizon.

My photo  Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes
Zhengqin Li, Yu-Ying Yeh, Manmohan Chandraker
CVPR 2020 (Oral)      [PDF]

A physically-motivated network that models refractions and reflections to recover high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images.

My photo  Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker
CVPR 2020 (Oral)      [PDF]

A physically-motivated network that recovers shape, complex material and spatially varying lighting from a single mobile phone image to enable photorealistic indoor AR applications.

My photo  Peek-a-Boo: Occlusion Reasoning in Indoor Scenes with Plane Representations
Ziyu Jiang, Buyu Liu, Samuel Schulter, Zhangyang Wang, Manmohan Chandraker
CVPR 2020 (Oral)      [PDF]

A novel planar representation for indoor 3D scene understanding that reasons about occlusions, with new metrics and a new dataset.

My photo  Towards Universal Representation Learning for Deep Face Recognition
Yichun Shi, Xiang Yu, Kihyuk Sohn, Manmohan Chandraker, Anil K. Jain
CVPR 2020      [PDF]

A universal representation for face recognition that achieves robustness and accuracy across several factors of variation such as blur, resolution and occlusion.

My photo  Understanding Road Layout from Videos as a Whole
Buyu Liu, Bingbing Zhuang, Samuel Schulter, Pan Ji, Manmohan Chandraker
CVPR 2020      [PDF]

Predict top-view layout of a complex 3D scene using a video sequence as input, with spatial consistency among objects and scene elements, as well as temporal coherence.

My photo  Private-kNN: Practical Differential Privacy for Computer Vision
Yuqing Zhu, Xiang Yu, Manmohan Chandraker, Yu-Xiang Wang
CVPR 2020      [PDF]

Making differential privacy practical for computer vision by reducing more than 90% privacy cost compared to prior methods, while achieving better accuracy.

My photo  Active Adversarial Domain Adaptation
Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, Manmohan Chandraker
WACV 2020      [PDF]

An active learning framework by integrating domain adversarial learning and importance sampling for continuous semi-supervised domain adaptation, applied to object recognition and detection.

My photo  DAVID: Dual-Attentional Video Deblurring
Junru Wu, Xiang Yu, Ding Liu, Manmohan Chandraker, Zhangyang Wang
WACV 2020      [PDF]

Video deblurring with an internal attention module to select optimal temporal scales for restoring the sharp center frame and an external attention module to aggregate such estimates across different blur levels.

My photo  Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones
Jinwoo Choi, Gaurav Sharma, Manmohan Chandraker, Jia-Bin Huang
WACV 2020      [PDF]

Action recognition on drone videos without using labeled data, through adversarial domain adaptation to labeled Internet video datasets.

My photo  Adversarial Learning of Privacy-Preserving and Task-Oriented Representations
T. Xiao, Y.-H. Tsai, K. Sohn, M.K. Chandraker, M.-H. Yang
AAAI 2020      [PDF]

Learning deep features that protect against model inversion attacks by an adversary with access to large-scale public data, with application to facial attribute recognition while protecting identity.

My photo  EcSeg: Semantic Segmentation of Metaphase Images Containing Extrachromosomal DNA
Utkrisht Rajkumar, Kristen Turner, Jens Luebeck, Viraj Deshpande, Manmohan Chandraker, Paul Mischel, Vineet Bafna
iScience 2019      [PDF]

Semantic segmentation for automatic microscopy image analysis to quantify oncogenes located on extrachromosomal DNA.

My photo  Universal Semi-Supervised Semantic Segmentation
T. Kalluri, G. Varma, M.K. Chandraker and C.V. Jawahar
ICCV 2019      [PDF]

Learning fair representations that perform well across both data-rich and data-poor domains, applied to semantic segmentation that minimizes annotation and deployment costs.

My photo  Domain Adaptation for Structured Output via Disentangled Patch Representations
Y.-H. Tsai, K. Sohn, S. Schulter and M.K. Chandraker
ICCV 2019 (Oral)      [PDF]

Unsupervised domain adaptation for semantic segmentation by learning discriminative feature representations of patches in the source domain that correspond to modes of the output space distribution.

My photo  A Parametric Top-View Representation of Complex Road Scenes
Z. Wang, B. Liu, S. Schulter and M.K. Chandraker
CVPR 2019      [PDF]

A parametric representation for 3D scene understanding of complex road scenes that is intuitive for human visualization and interpretable for higher-level decision making.

My photo  Feature Transfer Learning for Face Recognition with Under-Represented Data
X. Yin, X. Yu, K. Sohn, X. Liu and M.K. Chandraker
CVPR 2019      [PDF]

Mitigating bias against under-represented categories in face recognition by augmenting the feature space of under-represented subjects using the distribution of features for subjects that have sufficiently diverse samples.

My photo  Learning Structure-And-Motion-Aware Rolling Shutter Correction
B. Zhuang, Q.-H. Tran, P. Ji, L.F. Cheong and M.K. Chandraker
CVPR 2019      [PDF]

Theoretical limits on SFM with a rolling-shutter camera and leveraging data-driven priors through a network that learns camera motion and scene stucture to undistort a single rolling shutter image.

My photo  Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild
L. Tran, K. Sohn, X. Yu, X. Liu, and M.K. Chandraker
CVPR 2019      [PDF] [Supplementary]

Unsupervised domain adaptation that combines insights from semi-supervised learning for feature-level adaptation and 3D geometry-guided image synthesis for pixel-level adaptation.

My photo  Learning to Simulate
N. Ruiz, S. Schulter and M.K. Chandraker
ICLR 2019      [PDF]

A reinforcement learning-based method for automatically adjusting the parameters of any non-differentiable simulator, thereby controlling the distribution of synthesized data in order to maximize the accuracy of a model trained on that data.

My photo  Unsupervised Domain Adaptation for Distance Metric Learning
K. Sohn, W. Shang, X. Yu and M.K. Chandraker
ICLR 2019      [PDF]

Making face recognition fair across ethnicities through unsupervised adaptation across domains with non-overlapping label spaces, retaining identification power on all ethnicities while keeping representations for all identities well-separated.

My photo  Single-Shot Analysis of Refractive Shapes Using Convolutional Neural Networks
J. Stets, Z. Li, J. Frisvad and M.K. Chandraker
WACV 2019      [PDF]

Single-image depth map and normal estimation for transparent shapes using a network trained on a new synthetic dataset.

My photo  AutoNUE: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments
G. Varma, A. Subramanian, A. Namboodiri, M.K. Chandraker and C.V. Jawahar
WACV 2019      [PDF]

Taking a step towards self-driving on Indian roads through a novel large-scale dataset that provides instance segmentation and object detection labels.

My photo  Memory Warps for Learning Long-Term Online Video Representations
T.-H. Vu, S. Schulter, W. Choi and M.K. Chandraker
WACV 2019      [PDF]

A memory-based online video representation that achieves efficiency and accuracy through spatiotemporal warping to compensate for motions, while enabling prediction of feature representations in future frames.

My photo  Learning to Reconstruct Shape and Spatially-Varying Reflectance with a Single Image
Z. Li, Z. Xu, R. Ramamoorthi, K. Sunkavalli and M.K. Chandraker
SIGGRAPH Asia 2018      [PDF]

A differential rendering layer with global illumination to train a network that recovers shape and complex, spatially-varying material using a single image acquired with a mobile phone camera.

My photo  Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image
Z. Li, K. Sunkavalli and M.K. Chandraker
ECCV 2018 (Oral)      [PDF]

A differentiable rendering layer that models image formation with a complex spatially-varying BRDF to recover high-quality material information using a single mobile phone image.

My photo  Learning to Look around Objects for Top-View Representations of Outdoor Scenes
S. Schulter, M. Zhai, N. Jacobs and M.K. Chandraker
ECCV 2018      [PDF]

Predict occluded portions of the scene layout by looking around foreground objects like cars or pedestrians, with learned priors and rules about typical road layouts from simulated or, if available, map data.

My photo  Hierarchical Metric Learning and Matching for 2D and 3D Geometric Correspondences
M. Fathy, Q.-H. Tran, Z. Zia, P. Vernaza and M.K. Chandraker
ECCV 2018      [PDF]

Metric learning and deep supervision to leverage feature hierarchies in a deep convolutional network for 2D and 3D geometric matching.

My photo  Learning to Adapt Structured Output Space for Semantic Segmentation
Y.-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang and M.K. Chandraker
CVPR 2018 (Spotlight)      [PDF]

Unsupervised domain adaptation for semantic segmentation that aligns spatial similarities across domains through a structured output space.

My photo  Learning to See through Turbulent Water
Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi and M.K. Chandraker
WACV 2018      [PDF]

Learning appearance and geometric priors for single-image undistortion of an image observed through a dynamic refractive medium such as water waves.

My photo  Learning Efficient Object Detection Models with Knowledge Distillation
G. Chen, W. Choi, X. Yu, T. Han and M.K. Chandraker
NeurIPS 2017      [PDF]

Fast and accurate object detection network through knowledge distillation that transfers insights to a compact student model from a higher-capacity teacher model.

My photo  Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
K. Sohn, S. Liu, G. Zhong, X. Yu, M.-H. Yang and M.K. Chandraker
ICCV 2017      [PDF]

Video face recognition that utilizes large-scale unlabeled video data for adversarial domain alignment while transferring discriminative knowledge from large-scale labeled still images.

My photo  Towards Large-Pose Face Frontalization
X. Yin, X. Yu, K. Sohn, X. Liu and M.K. Chandraker
ICCV 2017      [PDF]

A GAN that utilizes 3D geometric priors to frontalize a profile view of a face, achieving high quality as well as maintaining identity.

My photo  Feature Reconstruction-Based Disentanglement for Pose-Invariant Face Recognition
X. Peng, X. Yu, K. Sohn, D. Metaxas and M.K. Chandraker
ICCV 2017      [PDF]

Metric learning to explicitly disentangle identity and pose, by demanding alignment between feature reconstructions through various combinations of identity and pose features.

My photo  Weakly Supervised Generative Adversarial Networks for 3D Reconstruction
J. Gwak, C. Choy, A. Garg, M.K. Chandraker and S. Savarese
3DV 2017      [PDF]

Weakly supervised 3D reconstruction using silhouettes through a raytrace pooling layer that enables perspective projection and backpropagation, along with an adversarial constraint.

My photo  Robust Energy Minimization for BRDF-Invariant Shape from Light Fields
Z. Li, Z. Xu, R. Ramamoorthi and M.K. Chandraker
CVPR 2017      [PDF]

A variational energy minimization framework for robust recovery of shape in multiview stereo with unknown BRDF, with applications to light field cameras.

My photo  Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
C. Li, Z. Zia, Q.-H. Tran, X. Yu, G. Hager and M.K. Chandraker
CVPR 2017      [PDF]

Deep supervision to sequentially infer intermediate concepts associated with the final task allows better generalization, demonstrated with an application to 3D semantic parsing from 2D images.

My photo  DESIRE: Distant Future Prediction in Dynamic Scenes with Multiple Interacting Agents
N. Lee, W. Choi, P. Vernaza, C. Choy, P. Torr and M.K. Chandraker
CVPR 2017      [PDF]

Predicting future trajectories in complex 3D scenes by accounting for the multimodaility of future behaviors, scene semantics and interactions among objects.

My photo  Deep Network Flow for Multi-Object Tracking
S. Schulter, P. Vernaza, W. Choi and M.K. CHandraker
CVPR 2017      [PDF]

Learning differentiable cost functions for multi-target tracking by association, with a bilevel optimization to minimize a loss defined on the solution of a liner program.

My photo  Person Re-Identification in the Wild
L. Zheng, H. Zhang, S. Sun, M.K. Chandraker, Y. Yang and Q. Tian
CVPR 2017      [PDF]

The PRW dataset, methods and metrics for jointly performing person detection and re-identification in videos.

My photo  Universal Correspondence Network
C. Choy, J. Gwak, S. Savarese and M.K. Chandraker
NeurIPS 2016 (Oral)      [PDF]

Deep metric learning to learn a feature space for geometric and semantic correspondence, with novel architectures such as a convolutional spatial transformer to handle local variations.

My photo  Deep Deformation Network for Object Landmark Localization
X. Yu, F. Zhou and M.K. Chandraker
ECCV 2016      [PDF]

Landmark localization for non-rigid objects by incorporating geometric insights such as shape bases and thin-plate spline deformations in a deep convolutional network.

My photo  A 4D Light-Field Dataset and CNN Architectures for Material Recognition
T.-C. Wang, E. Hiroaki, J. Zhu, M.K. Chandraker, A. Efros and R. Ramamoorthi
ECCV 2016      [PDF]

A dataset of light field images and novel 4D convolutional architectures to improve material recognition by exploiting multiple sub-aperture views and view-dependent reflectance effects.

My photo  WarpNet: Weakly Supervised Matching for Single-View Reconstruction
A. Kanazawa, D. Jacobs and M.K. Chandraker
CVPR 2016      [PDF]

Single-view reconstruction of non-rigid objects like birds without using part annotations, through a deformation-aware synthetic data generation and spatial priors in a deep network.

My photo  SVBRDF-Invariant Shape and Reflectance Recovery from Light Fields
T.-C. Wang, M.K. Chandraker, A. Efros and R. Ramamoorthi
CVPR 2016 (Oral)      [PDF]

Shape and spatially-varying material recovery from a single light field image of an object, using theoretical invariants from differential stereo with general reflectance.

My photo  A Continuous Occlusion Model for Road Scene Understanding
V. Dhiman, Q.-H. Tran, J. Corso and M.K. Chandraker
CVPR 2016      [PDF]

Interpretable occlusion reasoning for 3D scene understanding that assigns points to objects by modeling reflection or transmission probabilities for the corresponding camera ray.

My photo  Atomic Scenes for Scalable Traffic Scene Recognition
C.-Y. Chen, W. Choi and M.K. Chandraker
WACV 2016      [PDF]

Scalable framework for monocular 3D scene understanding with a hierarchical model that captures co-occurence and mutual exclusion relationships while incorporating both low-level trajectory features and high-level scene features.

My photo  The Information Available to a Moving Observer on Shape with Unknown, Isotropic BRDFs
M.K. Chandraker
IEEE PAMI 2015 [Spl. Issue, Best of CVPR 2014]      [PDF]

Theoretical inavriants that eliminate BRDF from a system of differential stereo equations to yield PDE constraints that delineate the extent of shape recovery.

My photo  High Accuracy Monocular SFM and Scale Correction for Autonomous Driving
S. Song, M.K. Chandraker and C. Guest
IEEE PAMI 2015      [PDF]

Robust monocular SFM in road scenes through novel designs for long-term feature tracking and scale drift correction.

My photo  Joint SFM and Detection Cues in 3D Object Localization for Autonomous Driving
S. Song and M.K. Chandraker
CVPR 2015 (Oral)      [PDF]

Single-camera 3D localization of objects in traffic scenes by combining cues from SFM point tracks and object detection bounding boxes.

My photo  On Joint Shape and Material Recovery from Motion Cues
M.K. Chandraker
ECCV 2014      [PDF]

Differential invariants under object or camera motion that constrain shape recovery or material estimation under various conditions on camera, material or lighting.

My photo  What Camera Motion Reveals About Shape with Unknown BRDF
M.K. Chandraker
CVPR 2014 (Oral) [Best Paper Award]      [PDF] [Tech Report]

Specification of the extent of shape recovery through differential stereo invariants obtained using images of an object with unknown material observed under a small camera motion.

My photo  Robust Scale Estimation in Real-Time Monocular SFM for Autonomous Driving
S. Song and M.K. Chandraker
CVPR 2014      [PDF]

Scale drift correction in monocular SFM using cues such as sparse features, dense stereo and object bounding boxes, by relating observation covariances of cues to error behaviors of their underlying variables.

My photo  What Motion Reveals About Shape with Unknown BRDF and Lighting
M.K. Chandraker, D. Reddy, Y. Wang and R. Ramamoorthi
CVPR 2013 (Oral)      [PDF]

Theoretical analysis of shape recovery using differential flow invariants obtained under small motions of an object relative to its environment.

My photo  Dense Object Reconstruction with Semantic Priors
Y. Bao, M.K. Chandraker, Y. Lin and S. Savarese
CVPR 2013 (Oral)      [PDF]

Semantic 3D reconstruction using shape priors consisting of a mean shape for the commonality of shapes across a category and weighted anchor points to encode similarities between instances in the form of appearance and spatial consistency.

My photo  Parallel, Real-Time Monocular Visual Odometry
S. Song and M.K. Chandraker
ICRA 2013      [PDF]

The first demonstration of practical monocular visual odometry in extended traffic scenes, through novel multithreaded design for robust feature tracking and scale drift correction.

My photo  On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs
M.K. Chandraker, J. Bai and R. Ramamoorthi
IEEE PAMI 2013 [Spl. Issue, Best of CVPR 2011]      [PDF]

A photometric flow relation that specifies theoretical extent of shape recovery possible with differential motion of a light source.

My photo  What An Image Reveals About Material Reflectance
M.K. Chandraker and R. Ramamoorthi
ICCV 2011 (Oral)      [PDF]

A semiparametric regression for single-image material estimation that achieves better generalization and interpretability, enabling applications such as relighting and material editing.

My photo  A Theory of Photometric Reconstruction for Unknown Isotropic Reflectances
M.K. Chandraker, J. Bai and R. Ramamoorthi
CVPR 2011 (Oral)      [PDF]

Differential invariants under light source motion that determine the extent of shape recovery and prior information needed for it, using images of an object with unknown material.

My photo  On the Duality of Forward and Inverse Light Transport
M.K. Chandraker, J. Bai, T.-T. Ng and R. Ramamoorthi
IEEE PAMI 2011      [PDF]

Fast computational methods for light transport inversion through a duality with forward rendering that allows analogues to radiosity, Monte Carlo and wavelet-based methods.

My photo  Globally Optimal Algorithms for Stratified Autocalibration
M.K. Chandraker, S. Agarwal, D.J. Kriegman and S. Belongie
IJCV 2009      [PDF]

Tight convex relations in a branch and bound framework for globally optimal estimation of the plane at infinity with chirality constraints and dual image of the absolute conic with semidefinite constraints, allowing metric upgrade of a projective reconsturction.

My photo  A Dual Theory of Inverse and Forward Light Transport
J. Bai, M.K. Chandraker, T.-T. Ng and R. Ramamoorthi
ECCV 2010      [PDF] [Tech Report] [Project Site]

Establishes light transport inversion as the dual problem of forward rendering, enabling efficient new methods for projector radiometric compensation and separation of bounces of global illumination.

My photo  Moving in Stereo: Efficient Structure and Motion Using Lines
M.K. Chandraker, J. Lim and D.J. Kriegman
ICCV 2009      [PDF]

A fast solution for structure and motion from line correspondences through an overdetermined polynomial system, used in a robust RANSAC framework for line-based SFM.

My photo  Globally Optimal Bilinear Programming for Computer Vision Applications
M.K. Chandraker and D.J. Kriegman
CVPR 2008 (Oral)      [PDF] [Code]

Globally optimal solutions to bilinear problems through convex relations in a branch and bound framework, demonstrated for face 3D morphable model fitting and non-rigid SFM.

My photo  Practical Global Optimization for Multiview Geometry
F. Kahl, S. Agarwal, M.K. Chandraker, D.J. Kriegman and S. Belongie
IJCV 2008      [PDF] [Code]

Insights from fractional programming and the theory of convex underestimators for globally optimal solutions in multiview geometry under the standard L2-norm of reprojection errors and the L1-norm which is less sensitive to outliers.

My photo  Globally Optimal Affine and Metric Upgrades in Stratified Autocalibration
M.K. Chandraker, S. Agarwal, D.J. Kriegman and S. Belongie
ICCV 2007 (Oral) [Marr Prize Honorable Mention]      [PDF] [Project Site]

Globally optimal metric upgrade of a projective reconstruction, through efficient convex relaxations for estimating the plane at infinity and camera intrinsic parameters.

My photo  High Precision Multi-touch Sensing on Surfaces using Overhead Cameras
A. Agarwal, S. Izadi, M.K. Chandraker and A. Blake
IEEE Tabletop 2007      [PDF]

Enable multi-touch interactions on an arbitrary flat surface using a pair of cameras mounted above the surface, by robustly identifying fingertips and precisely detecting touch.

My photo  ShadowCuts: Photometric Stereo with Shadows
M.K. Chandraker, S. Agarwal and D.J. Kriegman
CVPR 2007      [PDF] [Data+Code]

An algorithm for performing Lambertian photometric stereo in the presence of shadows. It reduces the low frequency bias inherent to the normal integration process and ensures that the recovered surface is consistent with the shadowing configuration.

My photo  Autocalibration via Rank-Constrained Estimation of the Absolute Quadric
M.K. Chandraker, S. Agarwal, F. Kahl, D. Nistér and D.J. Kriegman
CVPR 2007      [PDF] [VRML]

An autocalibration algorithm for upgrading a projective reconstruction to a metric reconstruction by estimating the absolute dual quadric.

My photo  Practical Global Optimization for Multiview Geometry
S. Agarwal, M.K. Chandraker, F. Kahl, D.J. Kriegman and S. Belongie
ECCV 2006 (Oral)      [PDF] [Code]

A practical method for finding the provably globally optimal solution to numerous problems in projective geometry including multiview triangulation, camera resectioning and homography estimation.

My photo  Reflections on the Generalized Bas-Relief Ambiguity
M.K. Chandraker, F. Kahl and D.J. Kriegman
CVPR 2005 (Oral)      [PDF]

For general nonconvex surfaces, interreflections completely resolve the GBR ambiguity. The full Euclidean geometry can be recovered from uncalibrated photometric stereo for which the light source directions and strengths are unknown.

My photo  Real-Time Camera Pose in a Room
M.K. Chandraker, C. Stock and A. Pinz
ICVS 2003      [PDF]

A new, fully mobile, purely vision-based pose tracking system that works indoors in a prepared room, using artificial landmarks.

My photo  Subpixel Corner Detection for Tracking Applications using CMOS Camera Technology
C. Stock, U. Mühlmann, M.K. Chandraker and A. Pinz
AAPR 2002      [PDF]

A multistage approach to gray-level corner detection that detects corners as the intersection points of the involved edges only by using a small neighborhood of the estimated corner position.