Compressive light field photography

ABSTRACT

Light field cameras, e.g. [Lytro 2012; Veeraraghavan et al. 2007], have ushered a new direction in photography allowing consumers to synthesize photographs with novel viewpoints or varying focus after the actual recording. Unfortunately, current light field camera designs impose a fixed tradeoff between spatial and angular resolution --- spatial resolution is reduced to capture angular light variation on the sensor. We introduce a principled computational framework and a new camera design to acquire and reconstruct light fields at full spatial and angular resolution from a single exposure. Our framework introduces a high-dimensional sparse basis for light fields learned from millions of light fields patches. The same optimization procedure also allows for the synthesis of optimal mask patterns that are mounted at a slight offset in front of the sensor and optically attenuate the light field before it is recorded by the sensor. Finally, a weighted compressive sensing-style reconstruction is performed to recover the light field. We demonstrate, in theory and with simulations, how our compressive approach to light field photography outperforms state-of-art techniques.

References

1. E. Candes and Y. Eldar and D. Needell and P. Randall. 20010. Compressed sensing with coherent and redundant dictionaries. Harmonic Analysis.
2. Lytro, I., 2012. Lytro Light Field Camera.
3. Mairal, J., Bach, F., Ponce, J., and Sapiro, G. 2009. Online dictionary learning for sparse coding. International Conference on Machine Learning.
4. Veeraraghavan, A., Raskar, R., Agarwal, A., Mohan, A., and Tumblin, J. 2007. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Siggraph.