Flexibles: Deformation-Aware 3D-Printed Tangibles for Capacitive Touchscreens

ABSTRACT

We introduce Flexibles: 3D-printed flexible tangibles that are deformation-aware and operate on capacitive touchscreens. Flexibles add expressive deformation input to interaction with on-screen tangibles. Based on different types of deformation mapping, we contribute a set of 3D-printable mechanisms that capture pressing, squeezing, and bending input with multiple levels of intensities. They can be integrated into 3D printed objects with custom geometries and on different locations. A Flexible is printed in a single pass on a consumer-level 3D printer without requiring further assembly. Through a series of interactive prototypes, example applications and a technical evaluation, we show the technical feasibility and the wide applicability of Flexibles.

References

1. Moritz Bächer, Benjamin Hepp, Fabrizio Pece, Paul G. Kry, Bernd Bickel, Bernhard Thomaszewski, and Otmar Hilliges. 2016. DefSense: Computational Design of Customized Deformable Input Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems CHI '16. ACM Press, NY, NY, USA, 3806--3816. DOI: http://dx.doi.org/10.1145/2858036.2858354
2. Patrick Baudisch, Torsten Becker, and Frederik Rudeck. 2010. Lumino: Tangible Blocks for Tabletop Computers Based on Glass Fiber Bundles. In Proceedings of the 28th international conference on Human factors in computing systems CHI '10. ACM Press, NY, NY, USA, 1165. DOI: http://dx.doi.org/10.1145/1753326.1753500
3. L. K. Baxter. 1996. Capacitive Sensors: Design and Applications, vol. 1 of IEEE Press Series on Electronics Technology. (1996).
4. Bernd Bickel, Moritz Bächer, Miguel a. Otaduy, Hyunho Richard Lee, Hanspeter Pfister, Markus Gross, and Wojciech Matusik. 2010. Design and Fabrication of Materials with Desired Deformation Behavior. ACM Transactions on Graphics 29, 4 (jul 2010), 1. DOI: http://dx.doi.org/10.1145/1833351.1778800
5. Eric Brockmeyer, Ivan Poupyrev, and Scott Hudson. 2013. PAPILLON: Designing Curved Display Surfaces with Printed Optics. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press, NY, NY, USA, 457--462. DOI: http://dx.doi.org/10.1145/2501988.2502027
6. Jesse Burstyn, Nicholas Fellion, and Paul Strohmeier. 2015. PrintPut: Resistive and Capacitive Input Widgets for Interactive 3D Prints. Lecture Notes in Computer Science, Vol. 9296. Springer International Publishing, Cham. 332--339 pages. DOI: http://dx.doi.org/10.1007/978-3-319-22701-6
7. Liwei Chan, Stefanie Müller, Anne Roudaut, and Patrick Baudisch. 2012. CapStones and ZebraWidgets: sensing stacks of building blocks, dials and sliders on capacitive touch screens. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems CHI '12. ACM Press, NY, NY, USA, 2189--2192. DOI: http://dx.doi.org/10.1145/2207676.2208371
8. Kiran Dandekar, Balasundar I Raju, and Mandayam a Srinivasan. 2003. 3-D Finite-Element Models of Human and Monkey Fingertips to Investigate the Mechanics of Tactile Sense. Journal of Biomechanical Engineering 125, 5 (2003), 682. DOI: http://dx.doi.org/10.1115/1.1613673
9. Kentaro Go, Katsutoshi Nonaka, Koji Mitsuke, and Masayuki Morisawa. 2012. Object shape and touch sensing on interactive tables with optical fiber sensors. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction - TEI '12. ACM Press, NY, NY, USA, 123. DOI: http://dx.doi.org/10.1145/2148131.2148158
10. Nan-wei Gong, Jürgen Steimle, Simon Olberding, Steve Hodges, Nicholas Edward Gillian, Yoshihiro Kawahara, and Joseph A. Paradiso. 2014. PrintSense: A Versatile Sensing Technique to Support Multimodal Flexible Surface Interaction. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems CHI '14. ACM Press, NY, NY, USA, 1407--1410. DOI: http://dx.doi.org/10.1145/2556288.2557173
11. Daniel Gröger, Elena Chong Loo, and Jürgen Steimle. 2016. HotFlex: Post-print Customization of 3D Prints Using Embedded State Change. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems CHI '16. ACM Press, NY, NY, USA, 420--432. DOI: http://dx.doi.org/10.1145/2858036.2858191
12. Fabian Hennecke, Franz Berwein, and Andreas Butz. 2011. Optical Pressure Sensing for Tangible User Interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces ITS '11. ACM Press, NY, NY, USA, 45. DOI: http://dx.doi.org/10.1145/2076354.2076362
13. Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. Proceedings of the ACM CHI'15 Conference on Human Factors in Computing Systems 1 (2015), 3011--3014. DOI: http://dx.doi.org/10.1145/2702123.2702518
14. Jonathan Hook, Thomas Nappey, Steve Hodges, Peter Wright, and Patrick Olivier. 2014. Making 3D Printed Objects Interactive Using Wireless Accelerometers. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems CHI EA '14. ACM Press, NY, NY, USA, 1435--1440. DOI: http://dx.doi.org/10.1145/2559206.2581137
15. Scott E. Hudson. 2014. Printing Teddy Bears: A Technique for 3D Printing of Soft Interactive Objects. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems CHI '14. ACM Press, NY, NY, USA, 459--468. DOI: http://dx.doi.org/10.1145/2556288.2557338
16. Alexandra Ion, Johannes Frohnhofen, Ludwig Wall, Robert Kovacs, Mirela Alistar, Jack Lindsay, Pedro Lopes, Hsiang-ting Chen, and Patrick Baudisch. 2016. Metamaterial Mechanisms. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST '16. ACM Press, NY, NY, USA, 529--539. DOI: http://dx.doi.org/10.1145/2984511.2984540
17. Yoshio Ishiguro and Ivan Poupyrev. 2014. 3D Printed Interactive Speakers. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems CHI '14. ACM Press, NY, NY, USA, 1733--1742. DOI: http://dx.doi.org/10.1145/2556288.2557046
18. Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms. In Proceedings of the SIGCHI conference on Human factors in computing systems CHI '97, Vol. 39. ACM Press, NY, NY, USA, 234--241. DOI: http://dx.doi.org/10.1145/258549.258715
19. Kunihiro Kato and Homei Miyashita. 2015. ExtensionSticker: A Proposal for a Striped Pattern Sticker to Extend Touch Interfaces and its Assessment. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15, Vol. 1. ACM Press, NY, NY, USA, 1851--1854. DOI: http://dx.doi.org/10.1145/2702123.2702500
20. Kunihiro Kato and Homei Miyashita. 2016. 3D Printed Physical Interfaces that can Extend Touch Devices. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST '16 Adjunct. ACM Press, NY, NY, USA, 47--49. DOI: http://dx.doi.org/10.1145/2984751.2985700
21. Sven Kratz, Tilo Westermann, Michael Rohs, and Georg Essl. 2011. CapWidgets: Tangible Widgets versus Multi-Touch Controls on Mobile Devices. In Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems CHI EA '11. ACM Press, NY, NY, USA, 1351. DOI: http://dx.doi.org/10.1145/1979742.1979773
22. Gierad Laput, Eric Brockmeyer, Scott E. Hudson, and Chris Harrison. 2015. Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15. ACM Press, NY, NY, USA, 1935--1944. DOI: http://dx.doi.org/10.1145/2702123.2702414
23. Simon J. Leigh, Robert J. Bradley, Christopher P. Purssell, Duncan R. Billson, and David a. Hutchins. 2012. A Simple, Low-Cost Conductive Composite Material for 3D Printing of Electronic Sensors. PLoS ONE 7, 11 (nov 2012), e49365. DOI: http://dx.doi.org/10.1371/journal.pone.0049365
24. Rong-Hao Liang, Kai-Yin Cheng, Liwei Chan, Chuan-Xhyuan Peng, Mike Y. Chen, Rung-Huei Liang, De-Nian Yang, and Bing-Yu Chen. 2013. GaussBits: Magnetic Tangible Bits for Portable and Occlusion-Free Near-Surface Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '13. ACM Press, NY, NY, USA, 1391. DOI: http://dx.doi.org/10.1145/2470654.2466185
25. Rong-Hao Liang, Han-Chih Kuo, Liwei Chan, De-Nian Yang, and Bing-Yu Chen. 2014. GaussStones: Shielded Magnetic Tangibles for Multi-Token Interactions on Portable Displays. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14. ACM Press, NY, NY, USA, 365--372. DOI: http://dx.doi.org/10.1145/2642918.2647384
26. S. Moscato, R. Bahr, T. Le, M. Pasian, M. Bozzi, L. Perregrini, and M. M. Tentzeris. 2016. Infill-Dependent 3-D-Printed Material Based on NinjaFlex Filament for Antenna Applications. IEEE Antennas and Wireless Propagation Letters 15 (2016), 1506--1509. DOI: http://dx.doi.org/10.1109/LAWP.2016.2516101
27. Tamotsu Murakami and Naomasa Nakajima. 1994. Direct and intuitive input device for 3-D shape deformation. In Conference companion on Human factors in computing systems CHI '94. ACM Press, NY, NY, USA, 233--236. DOI: http://dx.doi.org/10.1145/259963.260449
28. Simon Olberding, Sergio Soto Ortega, Klaus Hildebrandt, and Jürgen Steimle. 2015. Foldio: Digital Fabrication of Interactive and Shape-Changing Objects With Foldable Printed Electronics. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST '15. ACM Press, NY, NY, USA, 223--232. DOI: http://dx.doi.org/10.1145/2807442.2807494
29. Makoto Ono, Buntarou Shizuki, and Jiro Tanaka. 2013. Touch & Activate: Adding Interactivity to Existing Objects using Active Acoustic Sensing. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press, NY, NY, USA, 31--40. DOI: http://dx.doi.org/10.1145/2501988.2501989
30. M. Pakanen, A. Colley, and J. Häkkilä. 2014. Squeezy Bracelet - Designing a Wearable Communication Device for Tactile Interaction. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational - NordiCHI '14. ACM Press, NY, NY, USA, 305--314. DOI: http://dx.doi.org/10.1145/2639189.2639238
31. Julian Panetta, Qingnan Zhou, Luigi Malomo, Nico Pietroni, Paolo Cignoni, and Denis Zorin. 2015. Elastic Textures for Additive Fabrication. ACM Transactions on Graphics 34, 4 (2015), 135:1--135:12. DOI: http://dx.doi.org/10.1145/2766937
32. Jesús Pérez, Bernhard Thomaszewski, Stelian Coros, Bernd Bickel, José A. Canabal, Robert Sumner, and Miguel A. Otaduy. 2015. Design and Fabrication of Flexible Rod Meshes. ACM Transactions on Graphics 34, 4 (jul 2015), 138:1--138:12. DOI: http://dx.doi.org/10.1145/2766998
33. Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato. 2015. Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture. Virtual Reality 19, 1 (mar 2015), 45--56. DOI: http://dx.doi.org/10.1007/s10055-014-0256-y
34. Jun Rekimoto. 2002. SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems Changing our world, changing ourselves CHI '02. ACM Press, NY, NY, USA, 113. DOI: http://dx.doi.org/10.1145/503376.503397
35. Christian Rendl, Patrick Greindl, Michael Haller, Martin Zirkl, Barbara Stadlober, and Paul Hartmann. 2012. PyzoFlex: Printed Piezoelectric Pressure Sensing Foil. In Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST '12. ACM Press, NY, NY, USA, 509. DOI: http://dx.doi.org/10.1145/2380116.2380180
36. Christian Rendl, Michael Haller, Shahram Izadi, David Kim, Sean Fanello, Patrick Parzer, Christoph Rhemann, Jonathan Taylor, Martin Zirkl, Gregor Scheipl, and Thomas Rothländer. 2014. FlexSense: A Transparent Self-Sensing Deformable Surface. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14. ACM Press, NY, NY, USA, 129--138. DOI: http://dx.doi.org/10.1145/2642918.2647405
37. Munehiko Sato, Ivan Poupyrev, and Chris Harrison. 2012. Touché: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems CHI '12. ACM Press, NY, NY, USA, 483. DOI: http://dx.doi.org/10.1145/2207676.2207743
38. Valkyrie Savage, Colin Chang, and Björn Hartmann. 2013. Sauron: Embedded Single-camera Sensing of Printed Physical User Interfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press, NY, NY, USA, 447--456. DOI: http://dx.doi.org/10.1145/2501988.2501992
39. Valkyrie Savage, Ryan Schmidt, Tovi Grossman, George Fitzmaurice, and Björn Hartmann. 2014. A Series of Tubes: Adding Interactivity to 3D Prints Using Internal Pipes. In Proceedings of the 27th annual ACM symposium on User interface software and technology UIST '14. ACM Press, NY, NY, USA, 3--12. DOI: http://dx.doi.org/10.1145/2642918.2647374
40. Martin Schmitz, Mohammadreza Khalilbeigi, Matthias Balwierz, Roman Lissermann, Max Mühlhäuser, and Jürgen Steimle. 2015. Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects. In Proceedings of the 28th annual ACM symposium on User interface software and technology - UIST '15. ACM Press, NY, NY, USA, 253--258. DOI: http://dx.doi.org/10.1145/2807442.2807503
41. Martin Schmitz, Andreas Leister, Niloofar Dezfuli, Jan Riemann, Florian Müller, and Max Mühlhäuser. 2016. Liquido: Embedding Liquids into 3D Printed Objects to Sense Tilting and Motion. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems CHI EA '16. ACM Press, NY, NY, USA, 2688--2696. DOI: http://dx.doi.org/10.1145/2851581.2892275
42. Bertrand Schneider, Megan Strait, Laurence Muller, Sarah Elfenbein, Orit Shaer, and Chia Shen. 2012. Phylo-Genie: Engaging Students in Collaborative 'TreeThinking' through Tabletop Techniques. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems CHI '12. ACM Press, NY, NY, USA, 3071--3080. DOI: http://dx.doi.org/10.1145/2207676.2208720
43. Christian Schumacher, Bernd Bickel, Jan Rys, Steve Marschner, Chiara Daraio, and Markus Gross. 2015. Microstructures to Control Elasticity in 3D Printing. ACM Transactions on Graphics 34, 4 (2015), 136:1--136:13. DOI: http://dx.doi.org/10.1145/2766926
44. Ronit Slyper, Ivan Poupyrev, and Jessica Hodgins. 2011. Sensing Through Structure: Designing Soft Silicone Sensors. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction - TEI '11. ACM Press, NY, NY, USA, 213--220. DOI: http://dx.doi.org/10.1145/1935701.1935744
45. Jürgen Steimle, Andreas Jordt, and Pattie Maes. 2013. Flexpad: Highly Flexible Bending Interactions for Projected Handheld Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '13. ACM Press, NY, NY, USA, 237--246. DOI: http://dx.doi.org/10.1145/2470654.2470688
46. Yuta Sugiura, Masahiko Inami, and Takeo Igarashi. 2012. A Thin Stretchable Interface for Tangential Force Measurement. In Proceedings of the 25th annual ACM symposium on User interface software and technology UIST '12. ACM Press, NY, NY, USA, 529--536. DOI: http://dx.doi.org/10.1145/2380116.2380182
47. Yuta Sugiura, Gota Kakehi, Anusha Withana, Calista Lee, Daisuke Sakamoto, Maki Sugimoto, Masahiko Inami, and Takeo Igarashi. 2011. Detecting Shape Deformation of Soft Objects Using Directional Photorefiectivity Measurement. In Proceedings of the 24th annual ACM symposium on User interface software and technology UIST '11. ACM Press, NY, NY, USA, 509. DOI: http://dx.doi.org/10.1145/2047196.2047263
48. Giovanni Maria Troiano, Esben Warming Pedersen, and Kasper Hornbæk. 2015. Deformable Interfaces for Performing Music. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15. ACM Press, NY, NY, USA, 377--386. DOI: http://dx.doi.org/10.1145/2702123.2702492
49. Karen Vanderloock, Vero Vanden Abeele, Johan A.K. Suykens, and Luc Geurts. 2013. The Skweezee System: Enabling the Design and the Programming of Squeeze Interactions. In Proceedings of the 26th annual ACM symposium on User interface software and technology UIST '13. ACM Press, NY, NY, USA, 521--530. DOI: http://dx.doi.org/10.1145/2501988.2502033
50. Marynel Vázquez, Eric Brockmeyer, Ruta Desai, Chris Harrison, and Scott E. Hudson. 2015. 3D Printing Pneumatic Device Controls with Variable Activation Force Capabilities. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15. ACM Press, NY, NY, USA, 1295--1304. DOI: http://dx.doi.org/10.1145/2702123.2702569
51. Simon Voelker, Christian Cherek, Jan Thar, Thorsten Karrer, Christian Thoresen, Kjell Ivar Øvergrd, and Jan Borchers. 2015. PERCs: Persistently Trackable Tangibles on Capacitive Multi-Touch Displays. In Proceedings of the 28th annual ACM symposium on User interface software and technology - UIST '15. ACM Press, NY, NY, USA, 351--356. DOI: http://dx.doi.org/10.1145/2807442.2807466
52. Simon Voelker, Kosuke Nakajima, Christian Thoresen, Yuichi Itoh, Kjell Ivar Øvergrd, and Jan Borchers. 2013. PUCs: Detecting Transparent, Passive Untouched Capacitive Widgets on Unmodified Multi-touch Displays. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces ITS '13. ACM Press, NY, NY, USA, 101--104. DOI: http://dx.doi.org/10.1145/2512349.2512791
53. Johnty Wang, Nicolas Alessandro, Sidney Fels, and Bob Pritchard. 2011. SQUEEZY: Extending a Multi-touch Screen with Force Sensing Objects for Controlling Articulatory Synthesis. In Proceedings of the International Conference on New Interfaces for Musical Expression. Oslo, Norway, 531--532. http://www.nime.org/proceedings/2011/nime2011.531.pdf
54. Chihiro Watanabe, Alvaro Cassinelli, Yoshihiro Watanabe, and Masatoshi Ishikawa. 2014. Generic Method for Crafting Deformable Interfaces to Physically Augment Smartphones. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems CHI EA '14. ACM Press, NY, NY, USA, 1309--1314. DOI: http://dx.doi.org/10.1145/2559206.2581307
55. Cary Williams, Xing Dong Yang, Grant Partridge, Joshua Millar-Usiskin, Arkady Major, and Pourang Irani. 2011. TZee: Exploiting the Lighting Properties of Multi-touch Tabletops for Tangible 3D Interactions. In Proceedings of the 2011 annual conference on Human factors in computing systems CHI '11. ACM Press, NY, NY, USA, 1363. DOI: http://dx.doi.org/10.1145/1978942.1979143
56. Karl Willis, Eric Brockmeyer, Scott Hudson, and Ivan Poupyrev. 2012. Printed Optics: 3D Printing of Embedded Optical Elements for Interactive Devices. In Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST '12. ACM Press, NY, NY, USA, 589--598. DOI: http://dx.doi.org/10.1145/2380116.2380190
57. Neng-Hao Yu, Polly Huang, Yi-ping Hung, Li-Wei Chan, Seng-yong Yong Lau, Sung-Sheng Tsai, I.-Chun Hsiao, Dian-je Tsai, Fang-i. Hsiao, Lung-pan Cheng, and Mike Chen. 2011. TUIC: Enabling Tangible Interaction on Capacitive Multi-touch Display. In Proceedings of the 2011 annual conference on Human factors in computing systems CHI '11. ACM Press, NY, NY, USA, 2995. DOI: http://dx.doi.org/10.1145/1978942.1979386
58. Thomas G. Zimmerman, Joshua R. Smith, Joseph a. Paradiso, David Allport, and Neil Gershenfeld. 1995. Applying Electric Field Sensing to Human-Computer Interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems CHI '95. ACM Press, NY, NY, USA, 280--287. DOI: http://dx.doi.org/10.1145/223904.223940

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)