Research Projects

	Mathematical and Computational Fundamentals of Visual Appearance Developing new practical rendering algorithms requires a deeper theoretical understanding of light transport. My PhD thesis on A Signal-Processing Framework for Forward and Inverse Rendering included a theory of the reflection operator in terms of a special type of spherical convolution. Since then, I have been working on other theoretical approaches to analyzing the reflection operator, including links to principal component analysis, and efficient algorithms to compute generalized triple product integrals for interactive rendering. These ideas can be used to efficiently compute images with environment maps, cast shadows, and volumetric scattering. They can also lead to an understanding of the fundamental limits of inverse problems in graphics and vision, and novel frequency domain identities or invariants in images. Most recently, I have derived a complete first order or gradient theory of lighting, shading, and shadows , and a theory of locally low-dimensional light transport , with a number of new insights and analytic formulae. Thesis: [ Planar Light Fields [SPIE 01], Irradiance (Lambertian) [JOSA 01], Signal-Processing (Isotropic BRDFs) [SIGGRAPH 01], PhD ] Frequency Analysis: [ PCA [PAMI 02], Fourier Shadows [ECCV 04, PAMI 05], Signal-Processing (general case) [TOG 04], Spherical Harmonic Identities [ECCV 06, PAMI 08], Analysis of BRDF Factorization [EGSR 08], Fourier Motion Blur [SIGGRAPH 09] ] Wavelet Triple Products: Triple Product Integrals [SIGGRAPH 04], Affine Double and Triple Product Integrals [TOG 09] Gradient and Local Light Tranpsort Theory: First Order Analysis [TOG 07], Locally Low Dimensional Light Transport [SIGGRAPH 07]
	Interactive Photorealistic Rendering Global illumination algorithms can produce very realistic images, but interactive applications like games rarely use realistic illumination, materials, or shading effects. We seek to bridge the gap between interactivity and photorealism, combining theoretical analysis with practical algorithms to efficiently compute and represent photorealistic data for interactive rendering. Shadows, Environment, Normal Maps: [ Soft Shadows [SIGGRAPH 00], Irradiance Maps [SIGGRAPH 01], Fequency Env. Maps [SIGGRAPH 02], Normal Maps [SIGGRAPH 07] ] All-Frequency Precomputed Transport: [ All-Frequency Relighting [SIGGRAPH 03], Wavelet Triple Products [SIGGRAPH 04], BRDF Editing [SIGGRAPH 06], Temporal Coherence [EGSR 06], 4D Relighting [I3D 07], BRDF Editing with GI [TOG 08], BRDF Factorization [EGSR 08], Affine Wavelet Double/Triple Products [TOG 09] PRT Survey [FnT in CG&V 07/9] ]
	Acquisition, Representation and Rendering with Data-Driven Geometry, Lighting and Materials Realistic image synthesis requires accurate models for object geometry, illumination and material properties. Today, these are often the limiting factor in realism, and we therefore often acquire them from the real world. The challenges are effective acquisition, compact and usable representation, and efficient rendering. With the size of current datasets, and important new challenge is also scale and multiscale representations. We address these challenges by developing the appropriate mathematical representations and computational methods. Appearance Acquisition: [ Inverse Rendering [SIG 01], Reflectance Sharing [EGSR 05, PAMI 06], Scattering by Dilution [SIG 06], Inhomogeneous Dynamic Media [ECCV 08], Compressive Light Transport Sensing [TOG 09] ] Representation and Editing: [ SVBRDFs [SIGGRAPH 06], BRDF editing [SIGGRAPH 06], TSVBRDFs [SIGGRAPH 06], BSSRDFs [SIGGRAPH 06] ] Rendering by Monte Carlo Sampling: [ Environment Maps [SIGGRAPH 03], BRDFs [SIGGRAPH 04], CDF compression [EGSR 05], Visibility [JGT 06] ] MultiScale Appearance: [ Filtered Normal Maps [SIGGRAPH 07], Multiscale Textures [SIGGRAPH 08] ] Geometry: [ Generative Models [SIGGRAPH 99], Spacetime Stereo [CVPR 03, PAMI 05], Combining Positions and Normals [SIGGRAPH 05], Viewpoint Codes [CVPR 07] ]
	Efficient Methods for Volumetric Scattering The scattering of light in participating media like mist, fog and haze is particularly difficult to simulate numerically in computer graphics. By deriving analytic models for these computationally expensive phenomena, we can render a host of volumetric effects at interactive rates. Most recently, we have also acquired the first database of the scattering properties in media, by dilution in water, used simulations to develop a general empirical BSSRDF model, and developed new realistic models for human skin. [ Inhomogeneous media [EGSR 04], Real Time Single Scattering [SIGGRAPH 05], Acquisition by Dilution [SIGGRAPH 06], Human Skin [SIASIA 08] Empirical BSSRDF Model [SIGGRAPH 09] ]
	Complex Illumination, Materials and Shadows in Computer Vision We are interested in solving many classic computer vision problems like lighting-insensitive recognition under more general assumptions about illumination, material properties and cast shadows. We use the mathematical tools developed in my thesis and subsequent work to analyze and understand complex appearance effects. Most recently, we have derived a new class of frequency domain identities that have implications for relighting and image consistency checking. [ PCA [PAMI 02], Specularities for Recognition [ICCV 03], Fourier Cast Shadows [ECCV 04, PAMI 05], Modeling Illumination [Book chapter 05], Frequency Domain Invariants [ECCV 06, PAMI 08], Duality of Light Transport [ECCV 10] Differential Photometric Stereo [CVPR 11] ]