Tzu-Mao Li

tzli@ucsd.edu
CV

I am an assistant professor at the CSE department of UCSD, working with awesome people at the Center for Visual Computing. I explore the connections between visual computing algorithms and modern data-driven methods and develop programming languages and systems for facilitating the exploration. I did a 2-year postdoc with Jonathan Ragan-Kelley at both MIT CSAIL and UC Berkeley. I did my Ph.D. in the computer graphics group at MIT CSAIL, advised by Frédo Durand. I received my B.S. and M.S. degrees in computer science and information engineering from National Taiwan University in 2011 and 2013, respectively, where I worked with Yung-Yu Chuang at the Communication and Multimedia Lab.

Students

Current students: Yash Belhe (PhD student, with Ravi Ramamoorthi), Shiyang Jia (PhD student), Mohammad Shafiei Rezvani Nezhad (PhD student, with Ravi Ramamoorthi), Xuanda Yang (PhD student), Xuan Tang (PhD student), Wesley Chang (PhD student, with Ravi Ramamoorthi), Trevor Hedstrom (PhD student, with Jürgen P. Schulze), Mallikarjun Swamy (MSc student), Shanbin Ke (MSc student)

Alumni: Andrew Oabel (ERSP student), Maggie Liu (ERSP student), Nabhan Sazzad (ERSP student), Jesus Gutierrez (visiting student, with Ravi Ramamoorthi), Melody Ruth (UCSD undergrad, with Ravi Ramamoorthi), Peiyu Xu (visiting student)

Prospective students/postdocs

See this webpage for information and an overview of my research.

Teaching

CSE 168: Computer Graphics II: Rendering (Spring 2022)

CSE 272: Advanced Image Synthesis (Winter 2022, Winter 2023)

CSE 290: Seminar in Computer Science and Engineering (Fall 2022, Fall 2021)
We will read and discuss differentiable programming this fall. A tentative reading list is up. Let me know if you have any suggested reading!

Publications

Code, slides, video, papers are in the project pages.
Also check out my new Youtube channel for talk videos! I will upload more in the near future.

Differentiable Rendering of Neural SDFs through Reparameterization
Sai Praveen Bangaru, Michaél Gharbi, Tzu-Mao Li, Fujun Luan, Kalyan Sunkavalli, Miloš Hašan, Sai Bi, Zexiang Xu, Gilbert Bernstein, Frédo Durand
SIGGRAPH Asia 2022 (conference track full paper)
unbiased differentiable SDF rendering. concurrent with Vicini et al.'s work. some extra memory saving tricks are introduced.
Designing Perceptual Puzzles by Differentiating Probabilistic Programs
Kartik Chandra, Tzu-Mao Li, Joshua Tenenbaum, Jonathan Ragan-Kelley
SIGGRAPH 2022 (conference track full paper)
synthesizing optical illusion by differentiating through Bayesian vision models
Searching for Fast Demosaicking Algorithms
Karima Ma, Michael Gharbi, Andrew Adams, Shoaib Kamil, Tzu-Mao Li, Connelly Barnes, Jonathan Ragan-Kelley
ACM Transaction on Graphics (Presented at SIGGRAPH 2022)
systematically exploring the Pareto frontier of efficient and high-quality data-driven image processing algorithms
Efficient Automatic Scheduling of Imaging and Vision Pipelines for the GPU
Luke Anderson, Andrew Adams, Karima Ma, Tzu-Mao Li, Tian Jin, Jonathan Ragan-Kelley
Proceedings of the ACM on Programming Languages (OOPSLA 2021)
a scalable and data-driven Halide autoscheduler that can process large pipelines and output GPU schedules
Learning to Cluster for Rendering with Many Lights
Yu-Chen Wang, Yu-Ting Wu, Tzu-Mao Li, Yung-Yu Chuang
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2021)
adapting both the clustering and sampling distribution in lightcuts rendering using a data-driven method built on reinforcement lightcuts learning
Multi-Resolution Shared Representative Filtering for Real-Time Depth Completion
Yu-Ting Wu, Tzu-Mao Li, I-Chao Shen, Hong-Shiang Lin, Yung-Yu Chuang
High-Performance Graphics (HPG) 2021
fast sampling-based cross bilateral filtering for depth completion with large missing regions
Systematically Differentiating Parametric Discontinuities
Sai Praveen Bangaru*, Jesse Michel*, Kevin Mu, Gilbert Bernstein, Tzu-Mao Li, Jonathan Ragan-Kelley
*equal contribution
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2021)
a programming language perspective on the automatic differention of integrals with discontinuous integrands, and its applications in physics and rendering
Unbiased Warped-Area Sampling for Differentiable Rendering
Sai Praveen Bangaru, Tzu-Mao Li, Frédo Durand
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2020)
solving the differentiable rendering boundary integral with standard area sampling in an unbiased way using divergence theorem
Differentiable Vector Graphics Rasterization for Editing and Learning
Tzu-Mao Li, Michal Lukáč, Michaël Gharbi, Jonathan Ragan-Kelley
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2020)
design choices and applications of differentiable vector graphics rasterization
Physics-Based Differentiable Rendering: A Comprehensive Introduction
Shuang Zhao, Wenzel Jakob, Tzu-Mao Li
SIGGRAPH 2020 Course
our differentiable rendering tutorial!
DiffTaichi: Differentiable Programming for Physical Simulation
Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, Frédo Durand
International Conference on Learning Representation (ICLR) 2020
automatic differentiated Taichi and applications in model-based reinforcement learning
Taichi: A Language for High-Performance Computation on Spatially Sparse Data Structures
Yuanming Hu, Tzu-Mao Li, Luke Anderson, Jonathan Ragan-Kelley, Frédo Durand
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2019)
a data-oriented compiler that decouples hierarchical sparse data structures design from computation
Learning to Optimize Halide with Tree Search and Random Programs
Andrew Adams, Karima Ma, Luke Anderson, Riyadh Baghdadi, Tzu-Mao Li, Michaël Gharbi, Benoit Steiner, Steven Johnson, Kayvon Fatahalian, Frédo Durand, Jonathan Ragan-Kelley
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2019)
first Halide autoscheduler that produces faster code comparing to human experts on average
Sample-based Monte Carlo Denoising using a Kernel-Splatting Network
Michaël Gharbi, Tzu-Mao Li, Miika Aittala, Jaakko Lehtinen, Frédo Durand
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2019).
permutation invariant mapping from Monte Carlo samples to an image through splatting
Differentiable Visual Computing [slides (Keynote)] [slides (Powerpoint)]
Tzu-Mao Li
MIT PhD Dissertation
ACM SIGGRAPH 2020 Outstanding Doctoral Dissertation Award (announcement)
a coherent view of my PhD research, with some new discussions regarding previous work, and some background reviews
Inverse Path Tracing for Joint Material and Lighting Estimation
Dejan Azinović, Tzu-Mao Li, Anton Kaplanyan, Matthias Nießner
Conference on Computer Vision and Pattern Recognition (CVPR), 2019 (oral presentation)
applying differentiable rendering for material and lighting reconstruction
Differentiable Monte Carlo Ray Tracing through Edge Sampling
Tzu-Mao Li, Miika Aittala, Frédo Durand, Jaakko Lehtinen
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2018)
computing gradients of the light transport equation through an explicit sampling of Dirac delta functions on triangle edges
Differentiable Programming for Image Processing and Deep Learning in Halide
Tzu-Mao Li, Michaël Gharbi, Andrew Adams, Frédo Durand, Jonathan Ragan-Kelley
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2018)
Halide meets automatic differentiation, and a new way to think about data-driven image processing
Aether: An Embedded Domain Specific Sampling Language for Monte Carlo Rendering
Luke Anderson, Tzu-Mao Li, Jaakko Lehtinen, Frédo Durand
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2017)
a programming language for Monte Carlo rendering that automatically computes the probability density of a light path sample
Anisotropic Gaussian Mutations for Metropolis Light Transport through Hessian-Hamiltonian Dynamics
Tzu-Mao Li, Jaakko Lehtinen, Ravi Ramamoorthi, Wenzel Jakob, Frédo Durand
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2015)
a Metropolis light transport algorithm that makes use of automatically differentiated Hessian matrix of light path contribution
Dual-Matrix Sampling for Scalable Translucent Material Rendering
Yu-Ting Wu, Tzu-Mao Li, Yu-Hsun Lin, and Yung-Yu Chuang
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2015
subsurface scattering with many-lights using matrix sampling
SURE-based Optimization for Adaptive Sampling and Reconstruction
Tzu-Mao Li, Yu-Ting Wu, Yung-Yu Chuang
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2012)
Stein's unbiased risk estimator for sampling and denoising in Monte Carlo rendering

Word cloud

Some keywords extracted from the publications above. They might give you some sense of my research.

Misc

lajolla
An educational renderer for UCSD CSE 272
Deriving Radiative Backpropagation using the Recursive Form of Path Tracing
A short note about the paper "Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering". I show that you can derive a similar algorithm using traditional automatic differentiation.
diffvg
A differentiable vector graphics rasterizer with PyTorch and Tensorflow interfaces.
redner
A differentiable Monte Carlo ray tracer with PyTorch and Tensorflow interfaces.
Graphics bibtex
A mega bibtex file containing many graphics-related literatures.
Joint Stein’s Unbiased Risk Estimation for Adaptive Sampling and Reconstruction
A short note on a generalized formulation of our SURE-based rendering method.
dpt
My prototypical renderer.
smallgdpt
Gradient-Domain Path Tracing in ~450 lines.