Learning to Reconstruct Shape and Spatially-Varying Reflectance from a Single Image

Learning to Reconstruct Shape and Spatially-Varying Reflectance from a Single Image

Siggraph Asia 2018

Zhengqin Li1 Zexiang Xu1 Ravi Ramamoorthi1 Kalyan Sunkavalli2 Manmohan Chandraker1
1UC San Diego 2Adobe


Reconstructing shape and reflectance properties from images is a highly under-constrained problem, and has previously been addressed by using specialized hardware to capture calibrated data or by assuming known (or highly constrained) shape or reflectance. In contrast, we demonstrate that we can recover non-Lambertian, spatially-varying BRDFs and complex geometry belonging to any arbitrary shape class, from a single RGB image captured under a combination of unknown environment illumination and flash lighting. We achieve this by training a deep neural network to regress shape and reflectance from the image. Our network is able to address this problem because of three novel contributions: first, we build a large-scale dataset of procedurally generated shapes and real-world complex SVBRDFs that approximate real world appearance well. Second, single image inverse rendering requires reasoning at multiple scales, and we propose a cascade network structure that allows this in a tractable manner. Finally, we incorporate an in-network rendering layer that aids the reconstruction task by handling global illumination effects that are important for real-world scenes. Together, these contributions allow us to tackle the entire inverse rendering problem in a holistic manner and produce state-of-the-art results on both synthetic and real data.



19.1 MB

Supplementary Material

19.3 MB


128 GB

All Real Examples


Code and models



  title={Learning to reconstruct shape and spatially-varying reflectance from a single image},
  author={Li, Zhengqin and Xu, Zexiang and Ramamoorthi, Ravi and Sunkavalli, Kalyan and Chandraker, Manmohan},
  booktitle={SIGGRAPH Asia 2018 Technical Papers},