atmoSphere: designing cross-modal music experiences using spatial audio with haptic feedback

ABSTRACT

We use cross-Modal correspondence -the interaction between two or more sensory modalities- to create an engaging user experience. We present atmoSphere, a system that provides users immersive music experiences using spatial audio and haptic feedback. We focused on cross-modality of auditory and haptic sensations to augment the sound environment. The atmoSphere consists of a spatialized music and a sphere shaped device which provides haptic feedback. It provides users imagination of large sound environment although they feel haptic sensation in their hands. First user feedback is very encouraging. According to participants, atmoSphere creates an engaging experience.

References

1. Kevin Fan, Hideyuki Izumi, Yuta Sugiura, Kouta Minamizawa, Sohei Wakisaka, Masahiko Inami, Naotaka Fujii, and Susumu Tachi. 2013. Reality jockey: lifting the barrier between alternate realities through audio and haptic feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2557--2566.
2. Waka Fujisaki, Midori Tokita, and Kenji Kariya. 2015. Perception of the material properties of wood based on vision, audition, and touch. Vision research 109 (2015), 185--200.
3. Heng Gu, Susana Sanchez, Kai Kunze, and Masahiko Inami. 2015. An augmented e-reader for multimodal literacy. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. ACM, 353--356.
4. Shoya Ishimaru, Kai Kunze, Katsuma Tanaka, Yuji Uema, Koichi Kise, and Masahiko Inami. 2015. Smart eyewear for interaction and activity recognition. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 307--310.
5. Ali Israr, Seung-Chan Kim, Jan Stec, and Ivan Poupyrev. 2012. Surround haptics: tactile feedback for immersive gaming experiences. In CHI'12 Extended Abstracts on Human Factors in Computing Systems. ACM, 1087--1090.
6. Ali Israr, Siyan Zhao, Kyna McIntosh, JaeKyun Kang, Zachary Schwemler, Eric Brockmeyer, Mark Baskinger, and Moshe Mahler. 2015. Po2: augmented haptics for interactive gameplay. In ACM SIGGRAPH 2015 Emerging Technologies. ACM, 21.
7. Yasutoshi Makino, Yoshikazu Furuyama, Seki Inoue, and Hiroyuki Shinoda. 2016. HaptoClone (haptic-optical clone) for mutual tele-environment by real-time 3D image transfer with midair force feedback. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1980--1990.

Index Terms

• Hardware

  • Communication hardware, interfaces and storage

    • Tactile and hand-based interfaces

      • Haptic devices

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices