ABSTRACT
We present a novel, paired, wearable system for combining the kinesthetic experiences of two persons. These devices allow users to sense and combine muscle contraction and joint rigidity bi-directionally. This is achieved through kinesthetic channels based on electromyogram (EMG) measurement and electrical muscle stimulation (EMS). We developed a pair of wearable kinesthetic input-output (I/O) devices called bioSync that uses specially designed electrodes to perform biosignal measurement and stimulation simultaneously on the same electrodes.
In a user study, participants successfully evaluated the strength of their partners' muscle contractions while exerting their own muscles. We confirmed that the pair of devices could help participants synchronize their hand movements through tapping, without visual and auditory feedback. The proposed interpersonal kinesthetic communication system can be used to enhance interactions such as clinical gait rehabilitation and sports training, and facilitate sharing of physical experiences with Parkinson's patients, thereby enhancing understanding of the physical challenges they face in daily life.
Supplemental Material
- Jun Nishida, Kanako Takahashi, and Kenji Suzuki. 2015. A wearable stimulation device for sharing and augmenting kinesthetic feedback. In Proceedings of the 6th Augmented Human International Conference (AH '15). ACM, NY, NY, USA, 211--212. DOI: http://dx.doi.org/10.1145/2735711.2735775 Google ScholarDigital Library
- Jun Nishida, Kenji Suzuki. 2016. bioSync: Wearable haptic I/O device for synchronous kinesthetic interaction. In Proceedings of IEEE Virtual Reality (VR), pp.243--244. DOI: http://dx.doi.org/10.1109/VR.2016.7504744 Google ScholarCross Ref
- Jun Nishida and Kenji Suzuki. 2016. bioSync: Synchronous Kinesthetic Experience among People. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, NY, NY, USA, 3742--3745. DOI: http://dx.doi.org/10.1145/2851581.2890244 Google ScholarDigital Library
- Shunichi Kasahara and Jun Rekimoto. 2014. JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation. In Proceedings of the 5th Augmented Human International Conference (AH '14). ACM, NY, NY, USA, , Article 46 , 8 pages. DOI: http://dx.doi.org/10.1145/2582051.2582097 Google ScholarDigital Library
- Shunichi Kasahara, Mitsuhito Ando, Kiyoshi Suganuma, and Jun Rekimoto. 2016. Parallel Eyes: Exploring Human Capability and Behaviors with Paralleled First Person View Sharing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 1561--1572. DOI: http://dx.doi.org/10.1145/2858036.2858495 Google ScholarDigital Library
- Hiroyuki Iizuka, Daisuke Kondo, Hiroki Kawasaki, Hideyuki Ando, and Taro Maeda. 2011. Coordinated behavior between visually coupled dyads. In Proceedings of the 2nd Augmented Human International Conference (AH '11). ACM, NY, NY, USA, , Article 23 , 4 pages. DOI: http://dx.doi.org/10.1145/1959826.1959849 Google ScholarDigital Library
- H. Kawasaki, H. Iizuka, S. Okamoto, H. Ando and T. Maeda. 2010. Collaboration and skill transmission by first-person perspective view sharing system, 19th International Symposium in Robot and Human Interactive Communication, Viareggio, pp. 125--131. DOI: http://dx.doi.org/10.1109/ROMAN.2010.5598668 Google ScholarCross Ref
- Scott Brave and Andrew Dahley. 1997. inTouch: a medium for haptic interpersonal communication. In CHI '97 Extended Abstracts on Human Factors in Computing Systems (CHI EA '97). ACM, NY, NY, USA, 363--364. DOI: http://dx.doi.org/10.1145/1120212.1120435 Google ScholarDigital Library
- Ken Nakagaki, Chikara Inamura, Pasquale Totaro, Thariq Shihipar, Chantine Akikyama, Yin Shuang, and Hiroshi Ishii. 2015. Linked-Stick: Conveying a Physical Experience using a Shape-Shifting Stick. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, NY, NY, USA, 1609--1614. DOI: http://dx.doi.org/10.1145/2702613.2732712 Google ScholarDigital Library
- Jeremy N. Bailenson, Nick Yee, Scott Brave, Dan Merget and David Koslow. 2007. Virtual Interpersonal Touch: Expressing and Recognizing Emotions Through Haptic Devices. Journal of Human Computer Interaction. vol.22, No.3, 325--353. DOI: http://dx.doi.org/10.1080/07370020701493509Google ScholarCross Ref
- Daniel Leithinger, Sean Follmer, Alex Olwal, and Hiroshi Ishii. 2014. Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). ACM, NY, NY, USA, 461--470. DOI: http://dx.doi.org/10.1145/2642918.2647377 Google ScholarDigital Library
- Hideyuki Nakanishi, Kazuaki Tanaka, and Yuya Wada. 2014. Remote handshaking: touch enhances video-mediated social telepresence. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2143--2152. DOI: http://dx.doi.org/10.1145/2556288.2557169 Google ScholarDigital Library
- Scott Brave, Clifford Nass and Erenee Sirinian. 2001. Force-Feedback in computer-mediated communication. Proceedings of HCI International '2001 (the 9th International Conference on Human-Computer Interaction), New Orleans, USA, Volume 3, 145--149.Google Scholar
- N. Igarashi, K. Suzuki, H. Kawamoto and Y. Sankai. 2010. bioLights: Light emitting wear for visualizing lower-limb muscle activity, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 6393--6396. DOI: http://dx.doi.org/10.1109/IEMBS.2010.5627306 Google ScholarCross Ref
- Y. Tsubouchi and K. Suzuki. 2010. BioTones: A wearable device for EMG auditory biofeedback. In Proc of Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, 6543--6546. DOI: http://dx.doi.org/10.1109/IEMBS.2010.5627097 Google ScholarCross Ref
- Dozza, M., Horak, F.B. and Chiari, L. 2007. Auditory biofeedback substitutes for loss of sensory information in maintaining stance. Experimental Brain Research, Vol.178, No.1, 37--48. DOI: http://dx.doi.org/10.1007/s00221-006-0709-y Google ScholarCross Ref
- Raafat Shalaby, Thomas Schauer, Wolfgang Liedecke and Jorg Raisch. 2010. Amplifier design for EMG recording from stimulation electrodes during functional electrical stimulation leg cycling ergometry, Biomedizinische Technik/Biomedical Engineering. Vol. 56, Issue 1, 23--33, DOI: https://doi.org/10.1515/bmt.2010.055Google Scholar
- Pedro Lopes, Alexandra Ion, Willi Mueller, Daniel Hoffmann, Patrik Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 939--948. DOI: http://dx.doi.org/10.1145/2702123.2702461 Google ScholarDigital Library
- Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, NY, NY, USA, 543--552. DOI: http://dx.doi.org/10.1145/1978942.1979018 Google ScholarDigital Library
- Pedro Lopes, Patrik Jonell, and Patrick Baudisch. 2015. Affordance++: Allowing Objects to Communicate Dynamic Use. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 2515--2524. DOI: http://dx.doi.org/10.1145/2702123.2702128 Google ScholarDigital Library
- Pedro Lopes and Patrick Baudisch. 2013. Muscle-propelled force feedback: bringing force feedback to mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 2577--2580. DOI: http://dx.doi.org/10.1145/2470654.2481355 Google ScholarDigital Library
- Pedro Lopes, Alexandra Ion, and Patrick Baudisch. 2015. Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). ACM, NY, NY, USA, 11--19. DOI: http://dx.doi.org/10.1145/2807442.2807443 Google ScholarDigital Library
- Y. Muraoka. 2001. Development of an emg recording device from stimulation electrodes for functional electrical stimulation. Frontiers of Medical and Biological Engineering: The International Journal of the Japan Society of Medical Electronics and Biological Engineering, vol.11, No.4, 323--333. DOI: http://dx.doi.org/10.1163/156855701321138969 Google ScholarCross Ref
- C. M. Harris and D. M. Wolpert. 1998. Signal-dependent noise determines motor planning. Nature, 394, pp.780--784 Google ScholarCross Ref
- Manami Katoh, Narihiro Nishimura, Maki Yokoyama, Taku Hachisu, Michi Sato, Shogo Fukushima, and Hiroyuki Kajimoto. 2013. Optimal selection of electrodes for muscle electrical stimulation using twitching motion measurement. In Proceedings of the 4th Augmented Human International Conference (AH '13). ACM, NY, NY, USA, 237--238. DOI: http://dx.doi.org/10.1145/2459236.2459279 Google ScholarDigital Library
- S. Iwarsson. 1999. The housing enabler: An objective tool for assessing accessibility, The British Journal of Occupational Therapy, vol.62, No.11, pp.491--497. Google ScholarCross Ref
- G. P. Rosati Papini, M. Fontana and M. Bergamasco. 2016. Desktop Haptic Interface for Simulation of Hand-Tremor. IEEE Transactions on Haptics, vol. 9, no. 1, 33--42, Jan.-March 1. DOI: http://dx.doi.org/10.1109/TOH.2015.2504971 Google ScholarDigital Library
- Jun Nishida, Hikaru Takatori, Kosuke Sato, and Kenji Suzuki. 2015. CHILDHOOD: wearable suit for augmented child experience. In ACM SIGGRAPH 2015 Emerging Technologies (SIGGRAPH '15). ACM, NY, NY, USA, Article 7, DOI: http://dx.doi.org/10.1145/2782782.2792501 Google ScholarDigital Library
- Suzuki, K., Mito, G., Kawamoto, H., Hasegawa, Y. and Sankai, Y. 2007. Intention-based walking support for paraplegia patients with Robot Suit HAL, Advanced Robotics, Vol.21, pp.1441--1469. DOI: http://dx.doi.org/10.1163/156855307781746061Google ScholarCross Ref
- Bertrand Philippe, Daniel Gonzalez-Franco, Arthur Pointeau, and Christian Cherene. 2014. The Machine to be Another - Embodied Telepresence using human performers. Prix Ars Electronica (2014).Google Scholar
Index Terms
- bioSync: A Paired Wearable Device for Blending Kinesthetic Experience
Recommendations
bioSync: Synchronous Kinesthetic Experience among People
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsThis paper presents synchronous kinesthetic interaction among people. The users are able to perceive muscle activity bi-directionally, such as muscle contraction or rigidity of joints, through somatosensory channels in a realistic manner. We have ...
Controlling Maximal Voluntary Contraction of the Upper Limb Muscles by Facial Electrical Stimulation
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsIn this paper, we propose to use facial electrical stimulation to control maximal voluntary contraction (MVC) of the upper limbs. The method is based on a body mechanism in which the contraction of the masseter muscles enhances MVC of the limb muscles. ...
sEMG-based continuous estimation of joint angles of human legs by using BP neural network
In this paper, we propose an mth order nonlinear model to describe the relationship between the surface electromyography (sEMG) signals and the joint angles of human legs, in which a simple BP neural network is built for the model estimation. The inputs ...
Comments