Mix-and-match holography

Abstract

Computational caustics and light steering displays offer a wide range of interesting applications, ranging from art works and architectural installations to energy efficient HDR projection. In this work we expand on this concept by encoding several target images into pairs of front and rear phase-distorting surfaces. Different target holograms can be decoded by mixing and matching different front and rear surfaces under specific geometric alignments. Our approach, which we call mix-and-match holography, is made possible by moving from a refractive caustic image formation process to a diffractive, holographic one. This provides the extra bandwidth that is required to multiplex several images into pairing surfaces.

We derive a detailed image formation model for the setting of holographic projection displays, as well as a multiplexing method based on a combination of phase retrieval methods and complex matrix factorization. We demonstrate several application scenarios in both simulation and physical prototypes.

References

1. Se Hyun Ahn and L Jay Guo. 2009. Large-area roll-to-roll and roll-to-plate nanoimprint lithography: a step toward high-throughput application of continuous nanoimprinting. ACS Nano 3, 8 (2009), 2304--2310.
2. Ilya Baran, Philipp Keller, Derek Bradley, Stelian Coros, Wojciech Jarosz, Derek Nowrouzezahrai, and Markus Gross. 2012. Manufacturing layered attenuators for multiple prescribed shadow images. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 603--610.
3. Floraine Berthouzoz and Raanan Fattal. 2012. Resolution enhancement by vibrating displays. ACM Trans. Graph 31, 2 (2012), 15.
4. Matthew Brand. 2011. Specular holography. Applied Optics 50, 25 (2011), 5042--5046.
5. Edward Buckley. 2010. Holographic projector using one lens. Optics Letters 35, 20 (2010), 3399--3401.
6. Stephen Y Chou, Peter R Krauss, and Preston J Renstrom. 1996. Nanoimprint lithography. Journal of Vacuum Science & Technology B: Microelectronics and Nanometer Structures Processing, Measurement, and Phenomena 14, 6 (1996), 4129--4133.
7. Andrzej Cichocki and Rafal Zdunek. 2007. Multilayer nonnegative matrix factorization using projected gradient approaches. Int. J. of Neural Systems 17, 06 (2007), 431--446.
8. Gerwin Damberg, James Gregson, and Wolfgang Heidrich. 2016. High Brightness HDR Projection Using Dynamic Freeform Lensing. ACM Trans. Graph. 35, 3 (2016), 24.
9. Gerwin Damberg and Wolfgang Heidrich. 2015. Efficient freeform lens optimization for computational caustic displays. Optics Express 23, 8 (2015), 10224--10232.
10. James R Fienup. 1982. Phase retrieval algorithms: a comparison. Applied optics 21, 15 (1982), 2758--2769.
11. Manuel Finckh, Holger Dammertz, and Hendrik PA Lensch. 2010. Geometry construction from caustic images. In Proc. ECCV. 464--477.
12. Martin Fuchs, Ramesh Raskar, Hans-Peter Seidel, and Hendrik Lensch. 2008. Towards passive 6D reflectance field displays. ACM Transactions on Graphics (TOG) 27, 3 (2008), 58.
13. Rw W Gerchberg and WO Saxton. 1971. Phase determination from image and diffraction plane pictures in electron-microscope. Optik 34, 3 (1971), 275--+.
14. Tim D Gerke and Rafael Piestun. 2010. Aperiodic volume optics. Nature Photonics 4, 3 (2010), 188--193.
15. Daniel Glasner, Todd Zickler, and Anat Levin. 2014. A reflectance display. ACM Trans. Graph. 33, 4 (2014), 61.
16. Joseph W Goodman. 2005. Introduction to Fourier optics. Roberts and Company Publishers.
17. Felix Heide, Qiang Fu, Yifan Peng, and Wolfgang Heidrich. 2016. Encoded diffractive optics for full-spectrum computational imaging. Scientific Reports 6 (2016).
18. Felix Heide, James Gregson, Gordon Wetzstein, Ramesh Raskar, and Wolfgang Heidrich. 2014a. Compressive multi-mode superresolution display. Optics Express 22, 12 (2014), 14981--14992.
19. Felix Heide, Douglas Lanman, Dikpal Reddy, Jan Kautz, Kari Pulli, and David Luebke. 2014b. Cascaded displays: spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. 33, 4 (2014), 60.
20. Ngoc-Diep Ho. 2008. Nonnegative matrix factorization algorithms and applications. Ph.D. Dissertation. ÉCOLE POLYTECHNIQUE.
21. Reynald Hoskinson, Boris Stoeber, Wolfgang Heidrich, and Sidney Fels. 2010. Light reallocation for high contrast projection using an analog micromirror array. ACM Trans. Graph. 29, 6 (2010), 165.
22. Rafael Hostettler, Ralf Habel, Markus Gross, and Wojciech Jarosz. 2015. Dispersion-based Color Projection using Masked Prisms. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 329--338.
23. Matthias B Hullin, Ivo Ihrke, Wolfgang Heidrich, Tim Weyrich, Gerwin Damberg, and Martin Fuchs. 2012. Computational fabrication and display of material appearance. In Eurographics 2013-State of the Art Reports. 137--153.
24. Ganghun Kim, José A Domínguez-Caballero, and Rajesh Menon. 2012. Design and analysis of multi-wavelength diffractive optics. Optics Express 20, 3 (2012), 2814--2823.
25. Thomas Kiser, Michael Eigensatz, Minh Man Nguyen, Philippe Bompas, and Mark Pauly. 2013. Architectural Caustics---Controlling Light with Geometry. Springer.
26. Douglas Lanman, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich, and Ramesh Raskar. 2011. Polarization fields: dynamic light field display using multi-layer LCDs. ACM Trans. Graph. 30, 6 (2011), 186.
27. Seungjae Lee, Changwon Jang, Seokil Moon, Jaebum Cho, and Byoungho Lee. 2016. Additive light field displays: realization of augmented reality with holographic optical elements. ACM Tran. Graph. 35, 4 (2016), 60.
28. LB Lesem, PM Hirsch, and JA Jordan. 1969. The kinoform: a new wavefront reconstruction device. IBM Journal of Research and Development 13, 2 (1969), 150--155.
29. Anat Levin, Daniel Glasner, Ying Xiong, Frédo Durand, William Freeman, Wojciech Matusik, and Todd Zickler. 2013. Fabricating BRDFs at high spatial resolution using wave optics. ACM Trans. Graph. 32, 4 (2013), 144.
30. Chuan-bi Lin. 2007. Projected gradient methods for nonnegative matrix factorization. Neural Computation 19, 10 (2007), 2756--2779.
31. Adoph W Lohmann and DP Paris. 1967. Binary Fraunhofer holograms, generated by computer. Applied Optics 6, 10 (1967), 1739--1748.
32. XF Meng, LZ Cai, YR Wang, XL Yang, XF Xu, GY Dong, XX Shen, H Zhang, and XC Cheng. 2007. Hierarchical image encryption based on cascaded iterative phase retrieval algorithm in the Fresnel domain. Journal of Optics A: Pure and Applied Optics 9, 11 (2007), 1070.
33. Jason Mitchell, Moby Francke, and Dhabih Eng. 2007. Illustrative rendering in team fortress 2. In Proceedings of the 5th international symposium on Non-photorealistic animation and rendering. ACM, 71--76.
34. Fai H Mok. 1993. Angle-multiplexed storage of 5000 holograms in lithium niobate. Optics Letters 18, 11 (1993), 915--917.
35. Yuki Nagahama, Tomoyoshi Shimobaba, Tetsuya Kawashima, Takashi Kakue, and Tomoyoshi Ito. 2016. Holographic multi-projection using the random phase-free method. Applied Optics 55, 5 (2016), 1118--1123.
36. Moni Naor and Adi Shamir. 1994. Visual cryptography. In Workshop on the Theory and Application of of Cryptographic Techniques. Springer, 1--12.
37. Naohisa Okada, Tomoyoshi Shimobaba, Yasuyuki Ichihashi, Ryutaro Oi, Kenji Yamamoto, Minoru Oikawa, Takashi Kakue, Nobuyuki Masuda, and Tomoyoshi Ito. 2013. Band-limited double-step Fresnel diffraction and its application to computer-generated holograms. Optics Express 21, 7 (2013), 9192--9197.
38. Marios Papas, Thomas Houit, Derek Nowrouzezahrai, Markus Gross, and Wojciech Jarosz. 2012. The magic lens: refractive steganography. ACM Trans. Graph. 31, 6 (2012), 186.
39. Marios Papas, Wojciech Jarosz, Wenzel Jakob, Szymon Rusinkiewicz, Wojciech Matusik, and Tim Weyrich. 2011. Goal-based Caustics. In Computer Graphics Forum, Vol. 30. 503--511.
40. Yifan Peng, Qiang Fu, Felix Heide, and Wolfgang Heidrich. 2016. The diffractive achromat full spectrum computational imaging with diffractive optics. ACM Transactions on Graphics (TOG) 35, 4 (2016), 31.
41. Demetri Psaltis and Geoffrey W Burr. 1998. Holographic data storage. Computer 31, 2 (1998), 52--60.
42. Demetri Psaltis, Kevin Curtis, Michael Levene, Allen Pu, and George Barbastathis. 1995. Holographic storage using shift multiplexing. Optics letters 20, 7 (1995), 782--784.
43. Weidong Qu, Huarong Gu, Hao Zhang, and Qiaofeng Tan. 2015. Image magnification in lensless holographic projection using double-sampling Fresnel diffraction. Applied Optics 54, 34 (2015), 10018--10021.
44. Behzad Sajadi, M Gopi, and Aditi Majumder. 2012. Edge-guided resolution enhancement in projectors via optical pixel sharing. ACM Trans. Graph 31, 4 (2012), 79.
45. Yuliy Schwartzburg, Romain Testuz, Andrea Tagliasacchi, and Mark Pauly. 2014. High-contrast computational caustic design. ACM Trans. Graph. 33, 4 (2014), 74.
46. Yishi Shi, Guohai Situ, and Jingjuan Zhang. 2007. Multiple-image hiding in the Fresnel domain. Optics letters 32, 13 (2007), 1914--1916.
47. Tomoyoshi Shimobaba, Michal Makowski, Takashi Kakue, Minoru Oikawa, Naohisa Okada, Yutaka Endo, Ryuji Hirayama, and Tomoyoshi Ito. 2013. Lensless zoomable holographic projection using scaled Fresnel diffraction. Optics Express 21, 21 (2013), 25285--25290.
48. G Tricoles. 1987. Computer generated holograms: an historical review. Applied Optics 26, 20 (1987), 4351--4360.
49. Gordon Wetzstein, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar. 2012. Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting. ACM Trans. Graph. 31, 4 (2012), 80.
50. Gaolei Xue, Juan Liu, Xin Li, Jia Jia, Zhao Zhang, Bin Hu, and Yongtian Wang. 2014. Multiplexing encoding method for full-color dynamic 3D holographic display. Optics Express 22, 15 (2014), 18473--18482.
51. Genzhi Ye, Sundeep Jolly, V Michael Bove Jr, Qionghai Dai, Ramesh Raskar, and Gordon Wetzstein. 2014. Toward BxDF display using multilayer diffraction. ACM Trans. Graph. 33, 6 (2014), 191.
52. Yonghao Yue, Kei Iwasaki, Bing-Yu Chen, Yoshinori Dobashi, and Tomoyuki Nishita. 2012. Pixel art with refracted light by rearrangeable sticks. Computer Graphics Forum 31, 2pt3 (2012), 575--582.
53. Y. Yue, K. Iwasaki, B.-Y. Chen, Y. Dobashi, and T. Nishita. 2014. Poisson-based Continuous Surface Generation for Goal-based Caustics. ACM Trans. Graph. (2014).

Index Terms

• Computing methodologies

  • Computer graphics

• Hardware

  • Communication hardware, interfaces and storage

    • Displays and imagers