Interactive Photorealistic Rendering


While offline global illumination algorithms like ray tracing or radiosity have long been able to produce very realistic images, these effects are often absent in interactive applications like games. Indeed, it is rare to see real-time images with realistic illumination from wide-area light sources such as skylight, realistic materials like velvet, satin, paints, or wood, and shading effects like soft shadows. We seek to bridge the gap between realism and interactivity, allowing for measured illumination and reflectance data , and taking complex light transport effects like soft shadows into account.
Primary Current Participants

Aner Ben-Artzi (Graduated May 2007)
Kevin Egan
Ryan Overbeck
Charles Han
Bo Sun (Graduated Aug 2008)
Sebastian Enrique (Graduated MS Jun 2005)     check out the cool downloadable software for image relighting
Ren Ng (Stanford) (Graduated Jun 2006)
Ravi Ramamoorthi

Shadows, Environment and Normal Maps

Shadow and environment maps are among the oldest techniques to add realism to computer-generated images. However, shadow maps historically have only supported point light sources. We extend them to handle soft shadows from small area sources. Environment maps have historically supported only mirror specular reflections. We use the signal-processing methods in my PhD thesis to develop efficient algorithms and analytic methods for both Lambertian objects and general materials. Recently, we have shown that similar ideas, and more modern mathematical tools like spherical expectation maximization, can be used to solve the long-standing problem of normal map filtering.
Efficient Image-Based Methods for Rendering Soft Shadows: Siggraph 00, pages 375-384
We present two efficient image-based approaches for computation and display of high-quality soft shadows from area light sources. Our methods are related to shadow maps and provide the associated benefits.

Full Paper:     gzipped PS (4M)    PDF (1.7M)    Talk:    PPT (1.8M)
An Efficient Representation for Irradiance Environment Maps: Siggraph 01, pages 497-500
We use the 9 parameter Lambertian BRDF approximation for analytic rendering of diffuse objects under distant illumination specified by environment maps.

Full Paper:     gzipped PS (3.4M)    PDF (1M)    Talk:    PPT (1.8M)    Video
Frequency Space Environment Map Rendering: Siggraph 02, pages 517-526
We present a new method for real-time rendering of objects with complex isotropic BRDFs under distant natural illumination, as specified by an environment map. Our approach is based on our signal-processing framework.

Full Paper:     gzipped PS (4.2M)    PDF (3.3M)   

Frequency Domain Normal Map Filtering SIGGRAPH 07.
Filtering is critical for representing image-based detail, such as textures or normal maps, across a variety of scales. While mipmapping textures is commonplace, accurate normal map filtering remains a challenging problem because of nonlinearities in shading--we cannot simply average nearby surface normals. In this paper, we show analytically that normal map filtering can be formalized as a spherical convolution of the normal distribution function (NDF) and the BRDF, for a large class of common BRDFs such as Lambertian, microfacet and factored measurements. Our practical algorithms leverage a significant body of previous work that has studied lighting-BRDF convolution. We show how spherical harmonics can be used to filter the NDF for Lambertian and low-frequency specular BRDFs, while spherical von Mises-Fisher distributions can be used for high-frequency materials.

Paper:     PDF     Video (103M)
All-Frequency Precomputed Light Transport

More generally, we would be able to simultaneously get the effects of natural lighting and soft shadows at all frequencies. We would like to be able to interactively change the lighting or viewpoint for applications like lighting design or games. Our approach is to precompute information of interest, such as the appearance of a scene under all possible lighting conditions. The challenge is then how to compactly and efficiently represent this information for interactive rendering. We do so by developing new signal-processing methods using wavelets, which are ideally suited to handle all frequencies, from broad area sources to sharp specular materials. Most recently, we have used similar techniques as for precomputation-based relighting for material design and BRDF editing.
All-Frequency Shadows Using Non-Linear Wavelet Lighting Approximation Siggraph 03, pages 376-381
We present a method, based on pre-computed light transport, for real-time rendering of objects under all-frequency, time-varying illumination represented as a high-resolution environment map. For accurate rendering, using non-linear wavelets is an order of magnitude faster than using linear spherical harmonics, the current best technique.

PDF (1M)     Video (42MB)    
Triple Product Wavelet Integrals for All-Frequency Relighting Siggraph 04, pages 475-485
We propose a new mathematical and computational analysis of pre-computed light transport. We use factored forms, separately pre-computing the effects of visibility and material properties. Rendering then requires computing triple product integrals at each vertex, involving the lighting, visibility and BRDF. Our main contribution is a general analysis of these triple products likely to have broad applicability in computer graphics and numerical analysis.

PDF (5M)
Real-Time BRDF Editing in Complex Lighting Siggraph 06, pages 945-954
We develop the theory and algorithms to for the first time allow users to edit BRDFs in real time to design materials in their final placement in a scene with complex natural illumination and cast shadows.

PDF (20M)     Video (59M)    

Exploiting Temporal Coherence for Incremental All-Frequency Relighting EGSR 06, pages 151-160.
Current PRT methods exploit spatial coherence of the lighting (such as with wavelets) and of light transport (such as with CPCA). We consider a significant, yet unexplored form of coherence, temporal coherence of the lighting from frame to frame. We achieve speedups of 3x-4x over conventional PRT with minimal implementation effort, and can trivially be added to almost any existing PRT algorithm.

Paper:     PDF

4D Compression and Relighting with High-Resolution Light Transport Matrices ACM Symposium on Interactive 3D graphics 2007, pages 81-88.
We use a 4D wavelet transform for relighting with all-frequency illumination. A key observation is that a standard 4D wavelet transform can actually inflate portions of the light transport matrix. Therefore, we present an adaptive 4D wavelet transform that terminates at a level that avoids inflation and maximizes sparsity in the matrix data. Finally, we present an algorithm for fast relighting from adaptively compressed transport matrices.

Paper:     PDF     Video

A Precomputed Polynomial Representation for Interactive BRDF Editing with Global Illumination ACM Transactions on Graphics 27(2), Article 13, pages 1-13. Presented at SIGGRAPH 2008.
We develop a mathematical framework and practical algorithms to edit BRDFs with global illumination in a complex scene. A key challenge is that light transport for multiple bounces is non-linear in the scene BRDFs, while current PRT frameworks assume linear light transport. We address this by developing a new bilinear representation of the reflection operator, deriving a polynomial multi-bounce tensor precomputed framework, and reducing the complexity of further bounces.

Paper:     PDF     Video (7M)

An Analysis of the In-Out BRDF Factorization for View-Dependent Relighting EuroGraphics Symposium on Rendering, 2008.
We analyze the popular in-out BRDF factorization for view-dependent relighting theoretically. We show that for the Phong BRDFs, the factors or eigenmodes are simply spherical harmonic basis functions, and an exact frequency space analysis is possible. The number of terms needed grows linearly with the Phong exponent or quadratically with the frequency cutoff for reflected or half-angle dependence. The theory allows for setting the number of BRDF terms, and also indicates that high frequency specularity is still out of reach of current algorithms.

Paper:     PDF     Video (18M)
Affine Double and Triple Product Wavelet Integrals for Rendering ACM Transactions on Graphics 28(2), Article 14, pages 1-17, Apr 2009.
Many problems in computer graphics involve integrations of products of functions. Double- and triple-product integrals are commonly used in applications such as all-frequency relighting or importance sampling, but are limited to distant illumination. In contrast, near-field lighting from planar area lights involves an affine transform of the source radiance at different points in space. Our main contribution is a novel affine double- and triple-product integral theory.

Paper:     PDF     Video (AVI 16M)
Precomputation-Based Rendering Foundations and Trends in Computer Graphics and Vision 3(4), 281-369.
Precomputation-based relighting and radiance transfer has a long history with a spurt of renewed interest, including adoption in commercial video games, due to recent mathematical developments and hardware advances. In this survey, we describe the mathematical foundations, history, current research and future directions for precomputation-based rendering.

Paper:     PDF

Last modified: Wed May 20 12:01:16 PDT 2009