Sparsity in the discrete vs. continuous Fourier domain, and our reconstruction results: (a) The discrete Fourier transform(top) of a particular 2D angular slice wu, wv of the crystal ball's light field, and its reconstructed continuous version (bottom). (b) A grid showing the original images from the Stanford light field archive. The images used by our algorithm are highlighted (courtesy of [Stanford 2008]); (c) and (d) Two examples of reconstructed viewpoints showing successful reconstruction of this highly non-Lambertian scene which exhibits caustics, specularities, and nonlinear parallax. The uv locations of (c) and (d) are shown as blue and green boxes in (b).


Abstract

Sparsity in the Fourier domain is an important property that enables the dense reconstruction of signals, such as 4D light fields, from a small set of samples. The sparsity of natural spectra is often derived from continuous arguments, but reconstruction algorithms typically work in the discrete Fourier domain. These algorithms usually assume that sparsity derived from continuous principles will hold under discrete sampling. This paper makes the critical observation that sparsity is much greater in the continuous Fourier spectrum than in the discrete spectrum. This difference is caused by a windowing effect. When we sample a signal over a finite window, we convolve its spectrum by an infinite sinc, which destroys much of the sparsity that was in the continuous domain. Based on this observation, we propose an approach to reconstruction that optimizes for sparsity in the continuous Fourier spectrum. We describe the theory behind our approach and discuss how it can be used to reduce sampling requirements and improve reconstruction quality. Finally, we demonstrate the power of our approach by showing how it can be applied to the task of recovering non- Lambertian light fields from a small number of 1D viewpoint trajectories.



Results





Paper

Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain
Lixin Shi, Haitham Hassanieh, Abe Davis, Dina Katabi, Fredo Durand
ACM Transactions on Graphics Volume:34, No:1, November 2014.
[PAPER]   [SLIDES]


Code

Code: LightField_SFFT.zip

Documentation: Documentation.pdf

Note:
  • This code is provided for research purposes only. It is under the MIT License.
  • Please read the above paper and the documentation before running the code.
  • If you have any questions or problems running the code, please email sparselightfields@csail.mit.edu



  • People

    Lixin Shi (CSAIL MIT)

    Haitham Hassanieh (CSAIL MIT)

    Abe Davis (CSAIL MIT)

    Dina Katabi (CSAIL MIT)

    Fredo Durand (CSAIL MIT)

    Tianwei Huang (HKUST)