Abstract:

With the introduction of consumer light field cameras, light field imaging has recently become widespread. However, there is an inherent trade-off between the angular and spatial resolution, and thus, these cameras often sparsely sample in either spatial or angular domain. In this paper, we use machine learning to mitigate this trade-off. Specifically, we propose a novel learning-based approach to synthesize new views from a sparse set of input views. We build upon existing view synthesis techniques and break down the process into disparity and color estimation components. We use two sequential convolutional neural networks to model these two components and train both networks simultaneously by minimizing the error between the synthesized and ground truth images. We show the performance of our approach using only four corner sub-aperture views from the light fields captured by the Lytro Illum camera. Experimental results show that our approach synthesizes high-quality images that are superior to the state-of-the-art techniques on a variety of challenging real-world scenes. We believe our method could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase.

The Video

Download

	Paper 18.6 MB		Lo-res PDF 0.8 MB		Supplementary material 75.7 MB		Video 293 MB
	Test set 5.3 GB		Training set 17.5 GB		Training and test code 14.3 MB		External URL

© held by Owner/Author 2016. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM TOG and can be accessed through External URL.

Bibtex

@article{LearningViewSynthesis,
    author  = {Nima Khademi Kalantari and Ting-Chun Wang and Ravi Ramamoorthi},
    title   = {Learning-Based View Synthesis for Light Field Cameras},
    journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2016)},
    volume  = {35},
    number  = {6},
    year    = {2016},
}