Publications

Selected publications at major vision, graphics and learning venues.
(Full list on Google Scholar)

SAFE-SIM: Safety-Critical Simulation with Diffusion-Controllable Adversaries

Diffusion model for generating safety-critical traffic scenarios with controllable adversarial behaviors.

Authors: Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

Self-Training Large Language Models for Improved Visual Program Synthesis

An LLM agent for visual program synthesis trainable with reinforced self-training using just weak supervision from vision-language tasks.

Authors: Zaid Khan, Vijay Kumar, Samuel Schulter, Yun Fu, Manmohan Chandraker

LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes

A Lidar-enhanced neural radiance field to transform drive videos into photorealistic sensor simulation testbeds.

Authors: Shanlin Sun, Bingbing Zhuang, Ziyu Jiang, Buyu Liu, Xiaohui Xie, Manmohan Chandraker

AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving

A VLM and LLM-based framework for issue-finding, auto-labeling, continual training and automated verification for visual perception in autonomous driving.

Authors: Mingfu Liang, Jong-Chyi Su, Samuel Schulter, Sparsh Garg, Shiyu Zhao, Ying Wu, Manmohan Chandraker

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

TextureDreamer: Image-Guided Synthesis through Geometry-Aware Diffusion

Transfer photorealistic, high-fidelity and geometry-aware textures from 3-5 images to arbitrary 3D meshes.

Authors: Yu-Ying Yeh, Jia-Bin Huang, Changil Kim, Lei Xiao, Thu Nguyen-Phuoc, Numair Khan, Cheng Zhang, Manmohan Chandraker, Carl S Marshall, Zhao Dong, Zhengqin Li

What You See Is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

What You See Is What You GAN: Rendering Every Pixel for High-Fidelity 3D GANs

Scale neural volume rendering to high resolution by rendering every pixel to ensure that "what you see in 2D, is what you get in 3D".

Authors: Alex Trevithick, Matthew Chan, Towaki Takikawa, Umar Iqbal, Shalini De Mello, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

Tell, Don't Show! Language Guidance for Visual Transfer across Domains

Natural language to guide the transfer of discriminative visual knowledge from labeled source to unlabeled target domains.

Authors: Tarun Kalluri, Bodhisattwa Majumder, Manmohan Chandraker

UDABench: A Standardized Framework for Unsupervised Domain Adaptation

A comprehensive PyTorch framework for training and evaluating unsupervised domain adaptation methods.

Authors: Tarun Kalluri, Sreyas Ravichandran, Manmohan Chandraker

Long-HOT: A Hierarchical Approach for Long-Horizon Object Transport

A modular topological graph-based transport policy for temporally extended navigation and object transport.

Authors: Sriram Narayanan, Dinesh Jayaraman, Manmohan Chandraker

MCNeRF: Monte Carlo Rendering and Denoising for Real-Time NeRFs

A general Monte Carlo volume rendering algorithm to accelerate any NeRF representation using importance sampling based on ray density distributions.

Authors: Kunal Gupta, Milos Hasan, Zexiang Xu, Fujun Luan, Kalyan Sunkavalli, Xin Sun, Manmohan Chandraker, Sai Bi

Spatiotemporally Consistent HDR Indoor Lighting Estimation

A physically-motivated deep network that predicts spatiotemporally consistent lighting given a single LDR video of an indoor scene.

Authors: Zhengqin Li, Li Yu, Mikhail Okunev, Manmohan Chandraker, Zhao Dong

Exploring Question Decomposition for Zero-Shot VQA

Question decomposition that allows multi-billion scale vision-language models to approach reasoning-heavy visual question-answering as a two-step rather than a single-step problem.

Authors: Zaid Khan, Vijay Kumar, Samuel Schulter, Manmohan Chandraker, Raymond Fu

NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localization

Learn dense 3D object shapes for autonomous driving, using differentiable rendering with instance masks and 3D boxes, along with category-level priors.

Authors: Zhixiang Min, Bingbing Zhuang, Samuel Schulter, Buyu Liu, Enrique Dunn, Manmohan Chandraker

GeoNet: Benchmarking Unsupervised Adaptation across Geographies

A large-scale dataset and study on the robustness of computer vision algorithms and large pre-trained models across geographical biases.

Authors: Tarun Kalluri, Wangdong Xu, Manmohan Chandraker

FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation

FLAVR: Flow-Agnostic Video Representation for Fast Frame Interpolation

A flow-free and single-shot prediction approach for video frame interpolation, allowing efficient ultra-slow motion effects using just 3D convolutions.

Authors: Tarun Kalluri, Deepak Pathak, Manmohan Chandraker, Du Tran

Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimation

Joint estimation of emitters and materials in inverse rendering, to enable high-quality AR applications like scene relighting and object insertion.

Authors: Liwen Wu, Rui Zhu, Mustafa B. Yaldiz, Yinhao Zhu, Hong Cai, Janarbek Matai, Fatih Porikli, Tzu-Mao Li, Manmohan Chandraker, Ravi Ramamoorthi

Q: How to Specialize Large VLMs to Data-Scarce VQA? A: Train on Unlabeled Images!

Pseudo-label question-answers in unlabeled images, for fine-tuning large vision-language models on visual question answering in a data-scarce target dataset.

Authors: Zaid Khan, Vijay Kumar, Samuel Schulter, Raymond Fu, Manmohan Chandraker

A Theory of Topological Derivatives for Inverse Rendering of Geometry

Inverse rendering with topology changes by nucleating hole or phase changes, with applications to multiview 3D reconstruction and text-to-image generation.

Authors: Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi

Real-Time Radiance Fields for Single-Image Portrait View Synthesis

A real-time 3D GAN for photorealistic 3D shape generation from a single unposed image.

Authors: Alex Trevithick, Matthew Chan, Michael Stengel, Eric Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

Physically-Based Editing of Indoor Scene Lighting from a Single Image

Estimation of visible and invisible light sources from a single image of an indoor scene, to enable scene relighting and object insertion with full global effects.

Authors: Zhengqin Li, Jia Shi, Sai Bi, Rui Zhu, Kalyan Sunkavalli, Milos Hasan, Zexiang Xu, Ravi Ramamoorthi, Manmohan Chandraker

A Level Set Theory for Neural Implicit Evolution under Explicit Flows

A level-set theory bridge between explicit and implicit shape representations that allows user-defined editing of neural implicit geometry.

Authors: Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi

MemSAC: Memory Augmented Sample Consistency for Large Scale Domain Adaptation

MemSAC: Memory Sample Consistency for Large Scale Domain Adaptation

Unsupervised domain adaptation that scales to a large number of categories using a memory bank.

Authors: Tarun Kalluri, Astuti Sharma, Manmohan Chandraker

Learning Semantic Segmentation from Multiple Datasets with Label Shifts

Semantic segmentation training across multiple datasets with differing label spaces for better generalization to unseen domains.

Authors: Dongwan Kim, Yi-Hsuan Tsai, Yumin Suh, Masoud Faraki, Sparsh Garg, Manmohan Chandraker, Bohyung Han

Exploiting Unlabeled Data with Vision and Language Models for Object Detection

Object detection for unseen categories using pseudo-labels generated with large pre-trained vision-language models.

Authors: Shiyu Zhao, Zhixing Zhang, Samuel Schulter, Long Zhao, Vijay Kumar BG, Anastasis Stathopoulos, Manmohan Chandraker, Dimitris N. Metaxas

Single-Stream Multi-Level Alignment for Vision-Language Pretraining

Single-stream architecture for aligning vision and language representations at instance, patch and semantic levels..

Authors: Zaid Khan, Vijay Kumar BG, Xiang Yu, Samuel Schulter, Manmohan Chandraker, Yun Fu

TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments

Photorealistic virtual environments with high-quality multi-modal ground truth for geometry, material and semantics in road scenes.

Authors: Shubham Dokania, Anbumani Subramanian, Manmohan Chandraker, C. V. Jawahar

IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes

IRISformer: Dense Vision Transformers for Inverse Rendering in Indoor Scenes

A vision transformer architecture for inverse rendering in indoor scenes that allows learning long-range global interactions.

Authors: Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, Manmohan Chandraker

PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes

A framework that uses a set of images of an indoor scene to create a photorealistic digital twin with high-quality material and lighting.

Authors: Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, Manmohan Chandraker

Weakly But Deeply Supervised Occlusion-Reasoned Parametric Layouts

Single-image occlusion-reasoned road layout estimation in both perspective and parametric top-view space.

Authors: Buyu Liu, Bingbing Zhuang, Manmohan Chandraker

On Generalizing Beyond Domains in Cross-Domains Continual Learning

A continual learning method that allows generalization to new tasks in new domains with limited labels.

Authors: Christian Simon, Masoud Faraki, Yi-Hsuan Tsai, Xiang Yu, Samuel Schulter, Yumin Suh, Mehrtash Harandi, Manmohan Chandraker

Learning to Learn across Diverse Data Biases in Deep Face Recognition

Meta-learning to address biases due to long-tailed data and factors of variation such as ethnicity, head pose, occlusion and blur.

Authors: Chang Liu, Xiang Yu, Yi-Hsuan Tsai, Masoud Faraki, Ramin Moslemi, Manmohan Chandraker, Yun Fu

Controllable Dynamic Multi-Task Architectures

Multi-task learning with a dynamic architecture that satisfies resource constraints and user preferences.

Authors: Dripta S. Raychaudhuri, Yumin Suh, Samuel Schulter, Xiang Yu, Masoud Faraki, Amit K. Roy-Chowdhury, Manmohan Chandraker

Cluster-To-Adapt: Few Shot Domain Adaptation for Semantic Segmentation Across Disjoint Labels

Cluster-To-Adapt: Few Shot Adaptation for Segmentation Across Disjoint Labels

A clustering objective enforced in a transformed feature space for domain adaptation across segmentation datasets with completely different categories.

Authors: Tarun Kalluri, Manmohan Chandraker

Modulated Periodic Activations for Generalizable Local Functional Representations

Modulated Activations for Generalizable Local Functional Representations

Local functional representations that encode signals while achieving both high accuracy and generalization ability.

Authors: Ishit Mehta, Michael Gharbi, Connelly Barnes, Eli Shechtman, Ravi Ramamoorthi, Manmohan Chandraker

Learning Cross-Modal Contrastive Features for Video Domain Adaptation

Unsupervised domain adaptation for video representations in action recognition by aligning features across color and optical flow modalities.

Authors: Donghyun Kim, Yi-Hsuan Tsai, Bingbing Zhuang, Xiang Yu, Stan Sclaroff, Kate Saenko, Manmohan Chandraker

OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets

OpenRooms: An End-to-End Framework for Photorealistic Indoor Scene Datasets

An open source dataset of indoor scenes with high-quality tools and ground truth for shape, material and lighting, for augmented reality and robotics applications.

Authors: Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gundavarapu, Jia Shi, Sai Bi, Hong-Xing Yu, Zexiang Xu, Kalyan Sunkavalli, Milos Hasan, Ravi Ramamoorthi, Manmohan Chandraker

Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction

A multi-choice learning objective for diverse future trajectory prediction and a lane anchor network for 3D scene-consistent outputs.

Authors: Sriram Narayanan, Ramin Moslemi, Francesco Pittaluga, Buyu Liu, Manmohan Chandraker

Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation

An instance-affinity based criterion with a multi-sample contrastive loss for improved domain alignment.

Authors: Astuti Sharma, Tarun Kalluri, Manmohan Chandraker

Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

Fusing Old with the New: Relative Camera Pose with Geometry-Guided Uncertainty

A probabilistic fusion that learns to account for relative uncertainties in pose estimation by traditional geometric methods and deep neural networks, to achieve both better generalization and handling of ambiguities.

Authors: Bingbing Zhuang, Manmohan Chandraker

Looking Farther in Parametric Scene Parsing with Ground and Aerial Imagery

Road scene parsing in bird-eye view combining the relative benefits of ground-based proximate images with aerial remote sensing inputs.

Authors: Raghava Modhugu, Harish Sethuram, Manmohan Chandraker, C.V. Jawahar

Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows

Neural ODEs to generate 3D meshes with guaranteed manifoldness, enabling physically meaningful applications like rendering, simulations and 3D printing.

Authors: Kunal Gupta, Manmohan Chandraker

Single-Shot Neural Relighting and SVBRDF Estimation

A physically-motivated network for joint shape and material estimation, as well as relighting under novel illumination conditions, using a single image captured by a mobile phone camera.

Authors: Shen Sang, Manmohan Chandraker

SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction

Constant-time, scene consistent and diverse trajectory prediction regardless of number of agents, along with a dynamics simulator that diversifies top-views of real scenes.

Authors: Sriram Narayanan, Francesco Pittaluga, Buyu Liu, Manmohan Chandraker

Single-View Metrology in the Wild

Recover object height and camera parameters from a single unconstrained image, through a network that imbibes weakly supervised geometric constraints through bounding boxes and the horizon.

Authors: Rui Zhu, Xingyi Wang, Yannick Hold-Geoffroy, Federico Perazzi, Jonathan Eisenmann, Kalyan Sunkavalli, Manmohan Chandraker

Improving Face Recognition by Clustering Unlabeled Faces in the Wild

A clustering method to improve face recognition using unlabeled data, exploiting extreme value theory to account for overlapping identies between labeled and unlabeled data.

Authors: Aruni RoyChowdhury, Xiang Yu, Kihyuk Sohn, Erik Learned-Miller, Manmohan Chandraker

Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling

A two-layer convolutional LSTM that uses insights from keyframe-based methods to learn visual odometry from very long monocular sequences, in contrast to prior methods that are limited to short snippets.

Authors: Yuliang Zou, Pan Ji, Quoc-Huy Tran, Jia-Bin Huang, Manmohan Chandraker

Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction

A self-improving framework for monocular SLAM that iterates between unsupervised depth estimation and robust geometric SLAM to achieve state-of-the-art on KITTI and TUM sequences.

Authors: Lokender Tiwari, Pan Ji, Quoc-Huy Tran, Bingbing Zhuang, Saket Anand, Manmohan Chandraker

Domain Adaptive Semantic Segmentation Using Weak Labels

Domain adaptation that uses weak image-level labels to achieve category-wise alignment despite distribution shifts in dense structured prediction problems like semantic segmentation.

Authors: Sujoy Paul, Yi-Hsuan Tsai, Samuel Schulter, Amit K. RoyChowdhury, Manmohan Chandraker

Object Detection with a Unified Label Space from Multiple Datasets

A method for training a single object detector across multiple datasets, despite overlaps or mismatches between their label spaces.

Authors: Xiangyun Zhao, Samuel Schulter, Gaurav Sharma, Yi-Hsuan Tsai, Manmohan Chandraker, Ying Wu

Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes

A physically-motivated network that models refractions and reflections to recover high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images.

Authors: Zhengqin Li, Yu-Ying Yeh, Manmohan Chandraker

Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image

Single-Image Shape, Spatially-Varying Lighting, SVBRDF in Indoor Scenes

A physically-motivated network that recovers shape, complex material and spatially varying lighting from a single mobile phone image to enable photorealistic indoor AR applications.

Authors: Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker

Peek-a-Boo: Occlusion Reasoning in Indoor Scenes with Plane Representations

A novel planar representation for indoor 3D scene understanding that reasons about occlusions, with new metrics and a new dataset.

Authors: Ziyu Jiang, Buyu Liu, Samuel Schulter, Zhangyang Wang, Manmohan Chandraker

Towards Universal Representation Learning for Deep Face Recognition

A universal representation for face recognition that achieves robustness and accuracy across several factors of variation such as blur, resolution and occlusion.

Authors: Yichun Shi, Xiang Yu, Kihyuk Sohn, Manmohan Chandraker, Anil K. Jain

Understanding Road Layout from Videos as a Whole

Predict top-view layout of a complex 3D scene using a video sequence as input, with spatial consistency among objects and scene elements, as well as temporal coherence.

Authors: Buyu Liu, Bingbing Zhuang, Samuel Schulter, Pan Ji, Manmohan Chandraker

Private-kNN: Practical Differential Privacy for Computer Vision

Making differential privacy practical for computer vision by reducing more than 90% privacy cost compared to prior methods, while achieving better accuracy.

Authors: Yuqing Zhu, Xiang Yu, Manmohan Chandraker, Yu-Xiang Wang

Active Adversarial Domain Adaptation

An active learning framework by integrating domain adversarial learning and importance sampling for continuous semi-supervised domain adaptation, applied to object recognition and detection.

Authors: Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, Manmohan Chandraker

DAVID: Dual-Attentional Video Deblurring

Video deblurring with an internal attention module to select optimal temporal scales for restoring the sharp center frame and an external attention module to aggregate such estimates across different blur levels.

Authors: Junru Wu, Xiang Yu, Ding Liu, Manmohan Chandraker, Zhangyang Wang

Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones

Action recognition on drone videos without using labeled data, through adversarial domain adaptation to labeled Internet video datasets.

Authors: Jinwoo Choi, Gaurav Sharma, Manmohan Chandraker, Jia-Bin Huang

Adversarial Learning of Privacy-Preserving and Task-Oriented Representations

Learning deep features that protect against model inversion attacks by an adversary with access to large-scale public data, with application to facial attribute recognition while protecting identity.

Authors: T. Xiao, Y.-H. Tsai, K. Sohn, M.K. Chandraker, M.-H. Yang

EcSeg: Semantic Segmentation of Metaphase Images Containing Extrachromosomal DNA

EcSeg: Segmentation of Metaphase Images Containing Extrachromosomal DNA

Semantic segmentation for automatic microscopy image analysis to quantify oncogenes located on extrachromosomal DNA.

Authors: Utkrisht Rajkumar, Kristen Turner, Jens Luebeck, Viraj Deshpande, Manmohan Chandraker, Paul Mischel, Vineet Bafna

Universal Semi-Supervised Semantic Segmentation

Learning fair representations that perform well across both data-rich and data-poor domains, applied to semantic segmentation that minimizes annotation and deployment costs.

Authors: T. Kalluri, G. Varma, M.K. Chandraker and C.V. Jawahar

Domain Adaptation for Structured Output via Disentangled Patch Representations

Unsupervised domain adaptation for semantic segmentation by learning discriminative feature representations of patches in the source domain that correspond to modes of the output space distribution.

Authors: Y.-H. Tsai, K. Sohn, S. Schulter and M.K. Chandraker

A Parametric Top-View Representation of Complex Road Scenes

A parametric representation for 3D scene understanding of complex road scenes that is intuitive for human visualization and interpretable for higher-level decision making.

Authors: Z. Wang, B. Liu, S. Schulter and M.K. Chandraker

Feature Transfer Learning for Face Recognition with Under-Represented Data

Mitigating bias against under-represented categories in face recognition by augmenting the feature space of under-represented subjects using the distribution of features for subjects that have sufficiently diverse samples.

Authors: X. Yin, X. Yu, K. Sohn, X. Liu and M.K. Chandraker

Learning Structure-And-Motion-Aware Rolling Shutter Correction

Theoretical limits on SFM with a rolling-shutter camera and leveraging data-driven priors through a network that learns camera motion and scene stucture to undistort a single rolling shutter image.

Authors: B. Zhuang, Q.-H. Tran, P. Ji, L.F. Cheong and M.K. Chandraker

Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild

Unsupervised domain adaptation that combines insights from semi-supervised learning for feature-level adaptation and 3D geometry-guided image synthesis for pixel-level adaptation.

Authors: L. Tran, K. Sohn, X. Yu, X. Liu, and M.K. Chandraker

Learning to Simulate

A reinforcement learning-based method for automatically adjusting the parameters of any non-differentiable simulator, thereby controlling the distribution of synthesized data in order to maximize the accuracy of a model trained on that data.

Authors: N. Ruiz, S. Schulter and M.K. Chandraker

Unsupervised Domain Adaptation for Distance Metric Learning

Making face recognition fair across ethnicities through unsupervised adaptation across domains with non-overlapping label spaces, retaining identification power on all ethnicities while keeping representations for all identities well-separated.

Authors: K. Sohn, W. Shang, X. Yu and M.K. Chandraker

$Single-Shot Analysis of Refractive Shapes Using Convolutional Neural Networks$

Single-Shot Analysis of Refractive Shapes Using Convolutional Neural Networks

Single-image depth map and normal estimation for transparent shapes using a network trained on a new synthetic dataset.

Authors: J. Stets, Z. Li, J. Frisvad and M.K. Chandraker

AutoNUE: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments

AutoNUE: A Dataset for Autonomous Navigation in Unconstrained Environments

Taking a step towards self-driving on Indian roads through a novel large-scale dataset that provides instance segmentation and object detection labels.

Authors: G. Varma, A. Subramanian, A. Namboodiri, M.K. Chandraker and C.V. Jawahar

Memory Warps for Learning Long-Term Online Video Representations

A memory-based online video representation that achieves efficiency and accuracy through spatiotemporal warping to compensate for motions, while enabling prediction of feature representations in future frames.

Authors: T.-H. Vu, S. Schulter, W. Choi and M.K. Chandraker

Learning to Reconstruct Shape and Spatially-Varying Reflectance with a Single Image

Learning Single-Image Reconstruction of Shape and Spatially-Varying Reflectance

A differential rendering layer with global illumination to train a network that recovers shape and complex, spatially-varying material using a single image acquired with a mobile phone camera.

Authors: Z. Li, Z. Xu, R. Ramamoorthi, K. Sunkavalli and M.K. Chandraker

Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image

A differentiable rendering layer that models image formation with a complex spatially-varying BRDF to recover high-quality material information using a single mobile phone image.

Authors: Z. Li, K. Sunkavalli and M.K. Chandraker

Learning to Look around Objects for Top-View Representations of Outdoor Scenes

Predict occluded portions of the scene layout by looking around foreground objects like cars or pedestrians, with learned priors and rules about typical road layouts from simulated or, if available, map data.

Authors: S. Schulter, M. Zhai, N. Jacobs and M.K. Chandraker

Hierarchical Metric Learning and Matching for 2D and 3D Geometric Correspondences

Metric learning and deep supervision to leverage feature hierarchies in a deep convolutional network for 2D and 3D geometric matching.

Authors: M. Fathy, Q.-H. Tran, Z. Zia, P. Vernaza and M.K. Chandraker

Learning to Adapt Structured Output Space for Semantic Segmentation

Unsupervised domain adaptation for semantic segmentation that aligns spatial similarities across domains through a structured output space.

Authors: Y.-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang and M.K. Chandraker

Learning to See through Turbulent Water

Learning appearance and geometric priors for single-image undistortion of an image observed through a dynamic refractive medium such as water waves.

Authors: Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi and M.K. Chandraker

Learning Efficient Object Detection Models with Knowledge Distillation

Fast and accurate object detection network through knowledge distillation that transfers insights to a compact student model from a higher-capacity teacher model.

Authors: G. Chen, W. Choi, X. Yu, T. Han and M.K. Chandraker

Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos

Video face recognition that utilizes large-scale unlabeled video data for adversarial domain alignment while transferring discriminative knowledge from large-scale labeled still images.

Authors: K. Sohn, S. Liu, G. Zhong, X. Yu, M.-H. Yang and M.K. Chandraker

Towards Large-Pose Face Frontalization

A GAN that utilizes 3D geometric priors to frontalize a profile view of a face, achieving high quality as well as maintaining identity.

Authors: X. Yin, X. Yu, K. Sohn, X. Liu and M.K. Chandraker

Feature Reconstruction-Based Disentanglement for Pose-Invariant Face Recognition

Feature Reconstruction Disentanglement for Pose-Invariant Face Recognition

Metric learning to explicitly disentangle identity and pose, by demanding alignment between feature reconstructions through various combinations of identity and pose features.

Authors: X. Peng, X. Yu, K. Sohn, D. Metaxas and M.K. Chandraker

Weakly Supervised Generative Adversarial Networks for 3D Reconstruction

Weakly supervised 3D reconstruction using silhouettes through a raytrace pooling layer that enables perspective projection and backpropagation, along with an adversarial constraint.

Authors: J. Gwak, C. Choy, A. Garg, M.K. Chandraker and S. Savarese

Robust Energy Minimization for BRDF-Invariant Shape from Light Fields

A variational energy minimization framework for robust recovery of shape in multiview stereo with unknown BRDF, with applications to light field cameras.

Authors: Z. Li, Z. Xu, R. Ramamoorthi and M.K. Chandraker

Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing

Deep supervision to sequentially infer intermediate concepts associated with the final task allows better generalization, demonstrated with an application to 3D semantic parsing from 2D images.

Authors: C. Li, Z. Zia, Q.-H. Tran, X. Yu, G. Hager and M.K. Chandraker

DESIRE: Distant Future Prediction in Dynamic Scenes with Multiple Interacting Agents

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

Predicting future trajectories in complex 3D scenes by accounting for the multimodaility of future behaviors, scene semantics and interactions among objects.

Authors: N. Lee, W. Choi, P. Vernaza, C. Choy, P. Torr and M.K. Chandraker

Deep Network Flow for Multi-Object Tracking

Learning differentiable cost functions for multi-target tracking by association, with a bilevel optimization to minimize a loss defined on the solution of a liner program.

Authors: S. Schulter, P. Vernaza, W. Choi and M.K. CHandraker

Person Re-Identification in the Wild

The PRW dataset, methods and metrics for jointly performing person detection and re-identification in videos.

Authors: L. Zheng, H. Zhang, S. Sun, M.K. Chandraker, Y. Yang and Q. Tian

Universal Correspondence Network

Deep metric learning to learn a feature space for geometric and semantic correspondence, with novel architectures such as a convolutional spatial transformer to handle local variations.

Authors: C. Choy, J. Gwak, S. Savarese and M.K. Chandraker

Deep Deformation Network for Object Landmark Localization

Landmark localization for non-rigid objects by incorporating geometric insights such as shape bases and thin-plate spline deformations in a deep convolutional network.

Authors: X. Yu, F. Zhou and M.K. Chandraker

A 4D Light-Field Dataset and CNN Architectures for Material Recognition

A dataset of light field images and novel 4D convolutional architectures to improve material recognition by exploiting multiple sub-aperture views and view-dependent reflectance effects.

Authors: T.-C. Wang, E. Hiroaki, J. Zhu, M.K. Chandraker, A. Efros and R. Ramamoorthi

WarpNet: Weakly Supervised Matching for Single-View Reconstruction

Single-view reconstruction of non-rigid objects like birds without using part annotations, through a deformation-aware synthetic data generation and spatial priors in a deep network.

Authors: A. Kanazawa, D. Jacobs and M.K. Chandraker

SVBRDF-Invariant Shape and Reflectance Recovery from Light Fields

Shape and spatially-varying material recovery from a single light field image of an object, using theoretical invariants from differential stereo with general reflectance.

Authors: T.-C. Wang, M.K. Chandraker, A. Efros and R. Ramamoorthi

A Continuous Occlusion Model for Road Scene Understanding

Interpretable occlusion reasoning for 3D scene understanding that assigns points to objects by modeling reflection or transmission probabilities for the corresponding camera ray.

Authors: V. Dhiman, Q.-H. Tran, J. Corso and M.K. Chandraker

Atomic Scenes for Scalable Traffic Scene Recognition

Scalable framework for monocular 3D scene understanding with a hierarchical model that captures co-occurence and mutual exclusion relationships while incorporating both low-level trajectory features and high-level scene features.

Authors: C.-Y. Chen, W. Choi and M.K. Chandraker

The Information Available to a Moving Observer on Shape with Unknown, Isotropic BRDFs

The Information Available to a Moving Observer on Shape with Unknown BRDFs

Theoretical inavriants that eliminate BRDF from a system of differential stereo equations to yield PDE constraints that delineate the extent of shape recovery.

Authors: M.K. Chandraker

High Accuracy Monocular SFM and Scale Correction for Autonomous Driving

Robust monocular SFM in road scenes through novel designs for long-term feature tracking and scale drift correction.

Authors: S. Song, M.K. Chandraker and C. Guest

Joint SFM and Detection Cues in 3D Object Localization for Autonomous Driving

Single-camera 3D localization of objects in traffic scenes by combining cues from SFM point tracks and object detection bounding boxes.

Authors: S. Song and M.K. Chandraker

On Joint Shape and Material Recovery from Motion Cues

Differential invariants under object or camera motion that constrain shape recovery or material estimation under various conditions on camera, material or lighting.

Authors: M.K. Chandraker

What Camera Motion Reveals About Shape with Unknown BRDF

Specification of the extent of shape recovery through differential stereo invariants obtained using images of an object with unknown material observed under a small camera motion.

Authors: M.K. Chandraker

Robust Scale Estimation in Real-Time Monocular SFM for Autonomous Driving

Scale drift correction in monocular SFM using cues such as sparse features, dense stereo and object bounding boxes, by relating observation covariances of cues to error behaviors of their underlying variables.

Authors: S. Song and M.K. Chandraker

What Motion Reveals About Shape with Unknown BRDF and Lighting

Theoretical analysis of shape recovery using differential flow invariants obtained under small motions of an object relative to its environment.

Authors: M.K. Chandraker, D. Reddy, Y. Wang and R. Ramamoorthi

Dense Object Reconstruction with Semantic Priors

Semantic 3D reconstruction using shape priors consisting of a mean shape for the commonality of shapes across a category and weighted anchor points to encode similarities between instances in the form of appearance and spatial consistency.

Authors: Y. Bao, M.K. Chandraker, Y. Lin and S. Savarese

Parallel, Real-Time Monocular Visual Odometry

The first demonstration of practical monocular visual odometry in extended traffic scenes, through novel multithreaded design for robust feature tracking and scale drift correction.

Authors: S. Song and M.K. Chandraker

On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs

A photometric flow relation that specifies theoretical extent of shape recovery possible with differential motion of a light source.

Authors: M.K. Chandraker, J. Bai and R. Ramamoorthi

What An Image Reveals About Material Reflectance

A semiparametric regression for single-image material estimation that achieves better generalization and interpretability, enabling applications such as relighting and material editing.

Authors: M.K. Chandraker and R. Ramamoorthi

A Theory of Photometric Reconstruction for Unknown Isotropic Reflectances

Differential invariants under light source motion that determine the extent of shape recovery and prior information needed for it, using images of an object with unknown material.

Authors: M.K. Chandraker, J. Bai and R. Ramamoorthi

On the Duality of Forward and Inverse Light Transport

Fast computational methods for light transport inversion through a duality with forward rendering that allows analogues to radiosity, Monte Carlo and wavelet-based methods.

Authors: M.K. Chandraker, J. Bai, T.-T. Ng and R. Ramamoorthi

Globally Optimal Algorithms for Stratified Autocalibration

Tight convex relations in a branch and bound framework for globally optimal estimation of the plane at infinity with chirality constraints and dual image of the absolute conic with semidefinite constraints, allowing metric upgrade of a projective reconsturction.

Authors: M.K. Chandraker, S. Agarwal, D.J. Kriegman and S. Belongie

A Dual Theory of Inverse and Forward Light Transport

Establishes light transport inversion as the dual problem of forward rendering, enabling efficient new methods for projector radiometric compensation and separation of bounces of global illumination.

Authors: J. Bai, M.K. Chandraker, T.-T. Ng and R. Ramamoorthi

Moving in Stereo: Efficient Structure and Motion Using Lines

A fast solution for structure and motion from line correspondences through an overdetermined polynomial system, used in a robust RANSAC framework for line-based SFM.

Authors: M.K. Chandraker, J. Lim and D.J. Kriegman

Globally Optimal Bilinear Programming for Computer Vision Applications

Globally optimal solutions to bilinear problems through convex relations in a branch and bound framework, demonstrated for face 3D morphable model fitting and non-rigid SFM.

Authors: M.K. Chandraker and D.J. Kriegman

Practical Global Optimization for Multiview Geometry

Insights from fractional programming and the theory of convex underestimators for globally optimal solutions in multiview geometry under the standard L2-norm of reprojection errors and the L1-norm which is less sensitive to outliers.

Authors: F. Kahl, S. Agarwal, M.K. Chandraker, D.J. Kriegman and S. Belongie

Globally Optimal Affine and Metric Upgrades in Stratified Autocalibration

Globally optimal metric upgrade of a projective reconstruction, through efficient convex relaxations for estimating the plane at infinity and camera intrinsic parameters.

Authors: M.K. Chandraker, S. Agarwal, D.J. Kriegman and S. Belongie

High Precision Multi-touch Sensing on Surfaces using Overhead Cameras

Enable multi-touch interactions on an arbitrary flat surface using a pair of cameras mounted above the surface, by robustly identifying fingertips and precisely detecting touch.

Authors: A. Agarwal, S. Izadi, M.K. Chandraker and A. Blake

ShadowCuts: Photometric Stereo with Shadows

An algorithm for performing Lambertian photometric stereo in the presence of shadows. It reduces the low frequency bias inherent to the normal integration process and ensures that the recovered surface is consistent with the shadowing configuration.

Authors: M.K. Chandraker, S. Agarwal and D.J. Kriegman

Autocalibration via Rank-Constrained Estimation of the Absolute Quadric

An autocalibration algorithm for upgrading a projective reconstruction to a metric reconstruction by estimating the absolute dual quadric.

Authors: M.K. Chandraker, S. Agarwal, F. Kahl, D. Nistér and D.J. Kriegman

Practical Global Optimization for Multiview Geometry

A practical method for finding the provably globally optimal solution to numerous problems in projective geometry including multiview triangulation, camera resectioning and homography estimation.

Authors: S. Agarwal, M.K. Chandraker, F. Kahl, D.J. Kriegman and S. Belongie

Reflections on the Generalized Bas-Relief Ambiguity

For general nonconvex surfaces, interreflections completely resolve the GBR ambiguity. The full Euclidean geometry can be recovered from uncalibrated photometric stereo for which the light source directions and strengths are unknown.

Authors: M.K. Chandraker, F. Kahl and D.J. Kriegman

Real-Time Camera Pose in a Room

A new, fully mobile, purely vision-based pose tracking system that works indoors in a prepared room, using artificial landmarks.

Authors: M.K. Chandraker, C. Stock and A. Pinz

Subpixel Corner Detection for Tracking Applications using CMOS Camera Technology

Subpixel Corner Detection for Tracking using CMOS Camera Technology

A multistage approach to gray-level corner detection that detects corners as the intersection points of the involved edges only by using a small neighborhood of the estimated corner position.

Authors: C. Stock, U. Mühlmann, M.K. Chandraker and A. Pinz