Abstract
We present a camera with switchable primaries using shiftable layers of color filter arrays (CFAs). By layering a pair of CMY CFAs in this novel manner we can switch between multiple sets of color primaries (namely RGB, CMY and RGBCY) in the same camera. In contrast to fixed color primaries (e.g. RGB or CMY), which cannot provide optimal image quality for all scene conditions, our camera with switchable primaries provides optimal color fidelity and signal to noise ratio for multiple scene conditions.
Next, we show that the same concept can be used to layer two RGB CFAs to design a camera with switchable low dynamic range (LDR) and high dynamic range (HDR) modes. Further, we show that such layering can be generalized as a constrained satisfaction problem (CSP) allowing to constrain a large number of parameters (e.g. different operational modes, amount and direction of the shifts, placement of the primaries in the CFA) to provide an optimal solution.
We investigate practical design options for shiftable layering of the CFAs. We demonstrate these by building prototype cameras for both switchable primaries and switchable LDR/HDR modes.
To the best of our knowledge, we present, for the first time, the concept of shiftable layers of CFAs that provides a new degree of freedom in photography where multiple operational modes are available to the user in a single camera for optimizing the picture quality based on the nature of the scene geometry, color and illumination.
Supplemental Material
- Alter, F., Matsushita, Y., and Tang, X. 2006. An intensity similarity measure in low-light conditions. In ECCV, 267--280. Google Scholar
- Baone, G. A., and Qi, H. 2006. Demosaicking methods for multispectral cameras using mosaic focal plane array technology. Proc. SPIE 6062.Google Scholar
- Bayer, B. Color imaging array. US Patent 3,971,065.Google Scholar
- Cao, H., and Kot, A. C. 2008. A generalized model for detection of demosaicing characteristics. In ICME, 1513--1516.Google Scholar
- Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum 22, 419--426.Google ScholarCross Ref
- Freeman, T. W. 1988. Median filter for reconstructing missing color samples. US Patent 4,724,395.Google Scholar
- Gindele, E., and Gallagher, A. Sparsely sampled image sensing device with color and luminance photosites. US Patent 6,476,865.Google Scholar
- Gu, J., Wolfe, P. J., and Hirakawa, K. 2010. Filterbank-based universal demosaicking. 1981--1984.Google Scholar
- Gunturk, B., Member, S., Altunbasak, Y., Member, S., and Mersereau, R. M. 2002. Color plane interpolation using alternating projections. IEEE Trans. Image Processing 11, 997--1013. Google ScholarDigital Library
- Gunturk, B., Glotzbach, J., Altunbasak, Y., Schafer, R., and Mersereau, R. 2005. Demosaicking: color filter array interpolation. IEEE Signal Processing Magazine.Google Scholar
- Hamilton, J., and Adams, J. 1997. Adaptive color plane interpolation in single sensor color electronic camera. US Patent 5,629,734.Google Scholar
- Hirakawa, K., and Wolfe, P. 2008. Spatio-spectral color filter spatio-spectral color filter array design for optimal image recovery. IEEE TIP 17, 10. Google Scholar
- Imai, F. H., Berns, R. S., and Tzeng, D.-Y. 2000. A comparative analysis of spectral reflectance estimated in various spaces using a trichromatic camera system. Journal of Imaging Science and Technology 44, 280--287.Google Scholar
- Kumar, M., Morales, E., Adams, J., and Hao, W. 2009. New digital camera sensor architecture for low light imaging. IEEE ICIP, 2681--2684. Google Scholar
- Langfelder, G., Zaraga, F., and Longoni, A. 2009. Tunable spectral responses in a color-sensitive cmos pixel for imaging applications. Electron Devices, IEEE Transactions on.Google ScholarCross Ref
- Li, X. 2005. Demosaicing by successive approximation. IEEE Trans. Image Process. 14, 3, 370--379. Google ScholarDigital Library
- Lu, W., and Tan, Y.-P. 2003. Color filter array demosaicking: new method and performance measures. Image Processing, IEEE Transactions on. Google Scholar
- Lukac, R. 2008. Single-Sensor Imaging: Methods and Applications for Digital Cameras. CRC Press. Google Scholar
- Mohan, A., Raskar, R., and Tumblin, J. 2008. Agile spectrum imaging: Programmable wavelength modulation for cameras and projectors. Computer Graphics Forum 27, 2, 709--717.Google ScholarCross Ref
- Ratner, N., and Schechner, Y. Y. 2007. Illumination multiplexing within fundamental limits. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on 0, 1--8.Google Scholar
- Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2007. Multiplexing for Optimal Lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 8, 1339--1354. Google ScholarDigital Library
- Shogenji, R., Kitamura, Y., Yamada, K., Miyatake, S., and Tanida, J. 2004. Multispectral imaging using compact compound optics. Opt. Exp., 16431655.Google Scholar
- Susanu, G., Peterescu, S., Nanu, F., Capata, A., and Corcoran, P. 2009. Rgbw sensor array. US Patent 2009/0,167,893.Google Scholar
- Yamagami, T., Sasaki, T., and Suga, A. Image signal processing apparatus having a color filter with offset luminance filter elements. US Patent 5,323,233.Google Scholar
- Yasuma, F., Mitsunaga, T., Iso, D., and Nayar, S. Generalized Assorted Pixel Camera: Post-Capture Control of Resolution, Dynamic Range and Spectrum. Tech. rep.Google Scholar
Index Terms
Switchable primaries using shiftable layers of color filter arrays
Recommendations
Switchable primaries using shiftable layers of color filter arrays
SIGGRAPH '11: ACM SIGGRAPH 2011 papersWe present a camera with switchable primaries using shiftable layers of color filter arrays (CFAs). By layering a pair of CMY CFAs in this novel manner we can switch between multiple sets of color primaries (namely RGB, CMY and RGBCY) in the same ...
Extracting depth and matte using a color-filtered aperture
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papersThis paper presents a method for automatically extracting a scene depth map and the alpha matte of a foreground object by capturing a scene through RGB color filters placed in the camera lens aperture. By dividing the aperture into three regions through ...
Extracting depth and matte using a color-filtered aperture
This paper presents a method for automatically extracting a scene depth map and the alpha matte of a foreground object by capturing a scene through RGB color filters placed in the camera lens aperture. By dividing the aperture into three regions through ...
Comments