The study of the computational aspects of reflection, and especially the interaction between reflection and illumination, is of fundamental importance in both computer graphics and vision. In computer graphics, the interaction between the incident illumination and the reflective properties of a surface to give its overall visual appearance is a basic building block in most rendering algorithms. We are interested in the mathematical foundations of reflection and appearance, analyzing the theoretical characteristics of illumination, material properties and cast shadows. This analysis can be used to derive new computational algorithms that are relevant to many areas in graphics and vision. This combination of mathematical analysis and signalprocessing and other computational tools underlies all of our work (therefore this page also includes a few publications which are also classified under the practical applications to which they pertain).
Primary Current Participants 
Reflection as Convolution in Spherical Harmonics 
My PhD dissertation on A SignalProcessing Framework for Forward and Inverse Rendering introduced the basic idea of reflection as a spherical convolution of the incident illumination and the BRDF of the surface, with a product formula in spherical harmonics analogous to the standard convolution formula in the Fourier domain. While many authors had qualitatively described reflection as a convolution over the last 20 years, this is the first time a formal analysis and convolution formula have been derived. In terms of signalprocessing, the incident lighting can be seen as the signal, and the material properties or BRDF of the surface as the filter. There are many applications to both forward and inverse rendering.
Analysis of Planar Light Fields From Homogeneous Convex Curved Surfaces Under Distant Illumination
Proceedings of Human Vision and Electronic Imaging VI (part of Photonics West, 2001), pages 185198
This relatively simple to read paper considers the 2D case using
only Fourier transforms.  

On the relationship between Radiance and Irradiance: Determining the illumination from images of a convex Lambertian object
Journal of the Optical Society of America (JOSA A) Oct 2001, pages 24482459
This paper considers the 3D Lambertian case using spherical harmonics
and derives an analytic formula for the irradiance in terms of the
radiance, including the 9 parameter Lambertian BRDF approximation.

A SignalProcessing Framework for Inverse Rendering
This paper is the most mathematical so far and derives the theory for
the general 3D case with arbitrary isotropic BRDFs.
It also applies the
results to the practical problem of inverse rendering under
complex illumination.  
An Efficient Representation for Irradiance Environment Maps:
Siggraph 01, pages 497500 We use the 9 parameter Lambertian BRDF approximation for analytic rendering of diffuse objects under distant illumination specified by environment maps. Full Paper: gzipped PS (3.4M) PDF (1M) Talk: PPT (1.8M) Video 

Frequency Space Environment Map Rendering:
Siggraph 02, pages 517526 We present a new method for realtime rendering of objects with complex isotropic BRDFs under distant natural illumination, as specified by an environment map. Our approach is based on our signalprocessing framework. Full Paper: gzipped PS (4.2M) PDF (3.3M) 


A SignalProcessing Framework for Reflection ACM Transactions on Graphics (volume 23(4), Oct 2004, pages 10041042) A new theoretical paper presenting a general unified signalprocessing framework for 2D, 3D lambertian, 3D isotropic and 3D anisotropic cases, which includes many new results and significantly generalizes the previous articles. Paper: PDF 
General Applications of Convolution and Frequency Analysis 
The idea of reflection as convolution has more general theoretical and practical relevance. In recent work, we have used convolution to analytically derive principal components for Lambertian surfaces under different illumination conditions, explaining many empirical results on lightinginsensitive recognition for the first time. We have also shown that a convolution result can be derived for cast shadows in natural textures (cast shadows are an element not considered in my PhD thesis at all, which looked only at reflection and illumination). The idea of reflection as convolution has also recently been applied for both subsurface scattering and multiple scattering in volumes. While those methods are not directly related, similar mathematical machinery with Legendre polynomials can be used. Most recently, we have derived a new class of frequency domain identities or invariants , that lead to new BRDF/lighting transfer algorithms for relighting, and a new class of invariants to check image consistency, and detect tampering or image splicing. Finally, we have used frequencybased methods to analyze the accuracy of BRDF factorizations, and to develop a Fourier theory for fast rendering of motion blur.

Analytic PCA Construction for Theoretical Analysis of Lighting Variability, Including Attached Shadows, in a Single Image of a Convex Lambertian Object PAMI Oct 2002,
pp 13221333.
We explain for the first time some classic empirical results on lighting variability, and take a first step toward analyzing many classic vision problems under complex lighting. Paper: PDF (.8M) 

A Fourier Theory for Cast Shadows ECCV 04, pages I 146162 ; PAMI Feb 05 (pages 288295) We show that cast shadows can be mathematically analyzed for many simple configurations, resulting in a standard convolution formula that can be derived analytically in 2D and analyzed numerically in 3D. The results help explain many effects of lighting variability in 3D textures and suggest new bases for that purpose. Paper: ECCV 04 , PAMI 05 

A Theory of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting
Transfer and Image Consistency
ECCV 06, vol IV, pp 4155, PAMI 30(2), pages 197213, Feb 2008. We develop new mathematical results based on the spherical harmonic convolution framework for reflection, deriving novel identities, which are the angular frequency domain analogs to common spatial domain invariants such as reflectance ratios. Paper: PDF (PAMI 08) PDF (ECCV 06) 

An Analysis of the InOut BRDF Factorization
for ViewDependent Relighting
EuroGraphics Symposium on Rendering, 2008. We analyze the popular inout BRDF factorization for viewdependent relighting theoretically. We show that for the Phong BRDFs, the factors or eigenmodes are simply spherical harmonic basis functions, and an exact frequency space analysis is possible. The number of terms needed grows linearly with the Phong exponent or quadratically with the frequency cutoff for reflected or halfangle dependence. The theory allows for setting the number of BRDF terms, and also indicates that high frequency specularity is still out of reach of current algorithms. Paper: PDF Video (18M) 
Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur
SIGGRAPH 09. Motion blur is crucial for highquality rendering, but is also very expensive. Our first contribution is a frequency analysis of motionblurred scenes, including moving objects, specular reflections, and shadows. We show that motion induces a shear in the frequency domain, and that the spectrum of moving scenes is usually contained in a wedge. This allows us to compute adaptive spacetime sampling rates, to accelerate rendering. Our second contribution is a novel sheared reconstruction filter that is aligned to the firstorder direction of motion and enables even lower sampling rates. Paper: PDF 
Generalized ClebschGordan Triple Products 
My dissertation considered only the interaction of reflection and illumination, without cast shadows. When only two quantities are involved, the mathematical analysis is relatively simpler, since orthonormality can be applied to integrals, and simple results like convolution result. However, when three quantities are involved (lighting, BRDF, visibility), we must often consider an integral of a triple product . In spherical harmonics, a machinery of ClebschGordan expansions has been developed, but with a focus on analytic results for low orders rather than efficient evaluation. To our knowledge, there has been very little work on efficiently evaluating triple products in the mathematical literature. We present the first analysis of triple products in a variety of bases, and a very efficient algorithm for Haar wavelets.
Triple Product Wavelet Integrals for AllFrequency Relighting
Siggraph 04, pp 475485. We propose a new mathematical and computational analysis of precomputed light transport. We use factored forms, separately precomputing the effects of visibility and material properties. Rendering then requires computing triple product integrals at each vertex, involving the lighting, visibility and BRDF. Our main contribution is a general analysis of these triple products likely to have broad applicability in computer graphics and numerical analysis. Paper: PDF (5M) Video (17M) 

Affine Double and Triple Product Wavelet Integrals for Rendering
ACM Transactions on Graphics 28(2), Article 14, pages 117, Apr 2009. Many problems in computer graphics involve integrations of products of functions. Double and tripleproduct integrals are commonly used in applications such as allfrequency relighting or importance sampling, but are limited to distant illumination. In contrast, nearfield lighting from planar area lights involves an affine transform of the source radiance at different points in space. Our main contribution is a novel affine double and tripleproduct integral theory. Paper: PDF Video (AVI 16M) 
First Order or Gradient and Local Light Transport Analysis 
Frequency domain or spherical harmonic analysis is by definition global, and often limited to distant illumination or no shadowing. Wavelet methods are fast computationally, but do not permit analytic formulae. First order or gradient analysis is completely local, enables spatial and directional lighting variation, object curvature and bump maps. Analytic formulae enable gradientbased image sampling and interpolation even for soft shadows. A further local analysis of light transport reveals the dimensionality in local patches, and the optimal patch size.

A First Order Analysis of
Lighting, Shading, and Shadows ACM Transactions on
Graphics, 26(1), Article 2, pages 121, Jan 2007. The shading in a scene depends on a combination of many factors, such as how the lighting varies spatially across a surface, how it varies along different directions, the geometric curvature and reflectance properties of objects, and the locations of soft shadows. In this paper, we conduct a complete first order or gradient analysis of lighting, shading and shadows, showing how each factor separately contributes to scene appearance, and when it is important. 

A Theory of Locally Low Dimensional Light Transport
SIGGRAPH 07. We develop a theory of locally low dimensional light transport, to analytically derive the dimensionality of light transport for a local patch. We analyze the eigenvalues for canonical configurations using Szego's eigenvalue theorem. We show mathematically that for symmetric patches of area A, the number of basis functions for glossy reflections increases linearly with A, while for simple cast shadows, it often increases as sqrt(A). There are practical applications to CPCA and other PRT algorithms. Paper: PDF Video (30M) 