Memory Stones: An Intuitive Information Transfer Technique between Multi-touch Computers

ABSTRACT

Owners of multiple personal computing devices, such as mobile phones, tablet PCs, laptops, or desktop PCs, may frequently want to transfer information from one device to another.Whereas a drag-and-drop function on the same computing device is easy to achieve, it becomes cumbersome in an environment with multiple computing devices. We have to first locate and then select the target device from a list of devices on a network, even when the device is right in front of us. In this paper, a novel direct manipulation technique for executing drag-and-drop operations between multi-touch devices is proposed.Under our interface concept, dubbed "Memory Stones," a user can "pick up" a data object displayed on one device screen, "carry" it to another device screen, and "put it down" on that device using only their fingers.During this drag-and-drop operation, the user is invited to pantomime the act of carrying a tangible object (a "stone") while keeping their fingertip positions unchanged. The system identifies both the source and target devices by matching the shape of the polygon formed by the fingertips when touching each respective screen. We have developed a prototype system for small-to-large sized multi-touch computers including smartphones, tablet PCs, laptops, and desktop PCs, and have carried out a preliminary evaluation of its feasibility.

References

1. Ayatsuka, Y., M. N., and Rekimoto, J. Gaze-link: A new metaphor of real-world oriented user interface. IPSJ Journal 42, 6 (2011), 1330--1337.(in Japanese).
2. Ayatsuka, Y., and Rekimoto, J. transticks: physically manipulatable virtual connections. CHI '05, ACM (2005), 251--260.
3. Bragdon, A., DeLine, R., Hinckley, K., and Morris, M. R. Code space: Touch + air gesture hybrid interactions for supporting developer meetings. ITS'11, ACM (2011), 212--221.
4. Hardy, R., and Rukzio, E. Touch & interact: Touch-based interaction of mobile phones with displays. MobileHCI '08, ACM (2008), 245--254.
5. Hassan, N., Rahman, M. M., Irani, P., and Graham, P. Chucking: A one-handed document sharing technique. INTERACT '09, Springer-Verlag (2009), 264--278.
6. Hinckley, K. Synchronous gestures for multiple persons and computers. UIST '03, ACM (2003), 149--158.
7. Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., and Smith, M. Stitching: Pen gestures that span multiple displays. AVI '04, ACM (2004), 23--31.
8. Holmquist, L. E., Mattern, F., Schiele, B., Alahuhta, P., Beigl, M., and Gellersen, H.-W. Smart-its friends: A technique for users to easily establish connections between smart artefacts. UbiComp '01, Springer-Verlag (2001), 116--122.
9. Lee, H., Jeong, H., Lee, J., Yeom, K.-W., Shin, H.-J., and Park, J.-H. Select-and-point: A novel interface for multi-device connection and control based on simple hand gestures. CHI EA '08, ACM (2008), 3357--3362.
10. Mayrhofer, R., and Gellersen, H. Shake well before use: authentication based on accelerometer data. PERVASIVE'07, Springer-Verlag (2007), 144--161.
11. McCune, J. M., Perrig, A., and Reiter, M. K. Seeing-is-believing: using camera phones for human-verifiable authentication. Int. J. Secur. Netw. 4, 1/2 (Feb. 2009), 43--56.
12. Ogata, M., Sugiura, Y., Osawa, H., and Imai, M. Flashtouch: data communication through touchscreens. CHI '13, ACM (New York, NY, USA, 2013), 2321--2324.
13. Peng, C., Shen, G., Zhang, Y., and Lu, S. Point&connect: intention-based device pairing for mobile phone users. MobiSys '09, ACM (2009), 137--150.
14. Rekimoto, J. Pick-and-drop: a direct manipulation technique for multiple computer environments. UIST'97, ACM (New York, NY, USA, 1997), 31--39.
15. Rekimoto, J. Synctap: synchronous user operation for spontaneous network connection. Personal Ubiquitous Comput. 8, 2 (May 2004), 126--134.
16. Shneiderman, B. Designing the user interface 2nd edition.
17. Swindells, C., Inkpen, K. M., Dill, J. C., and Tory, M. That one there! pointing to establish device identity. UIST '02, ACM (2002), 151--160.
18. Wilson, A. D., and Benko, H. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. UIST '10, ACM (2010), 273--282.
19. Yatani, K., Tamura, K., Hiroki, K., Sugimoto, M., and Hashizume, H. Toss-it: intuitive information transfer techniques for mobile devices. CHI EA '05, ACM (2005), 1881--1884.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)