M. Iftekhar Tanveer
ABSTRACT
The inability to access non-verbal cues is a setback for people who are blind or visually impaired. A visual-to-auditory Sensory Substitution Device (SSD) may help improve the quality of their lives by transforming visual cues into auditory cues. In this paper, we describe the design and development of a robust and real-time SSD called iFEPS -- improved Facial Expression Perception through Sound. The implementation of the iFEPS evolved over time through a participatory design process. We conducted both subjective and objective experiments to quantify the usability of the system. Evaluation with 14 subjects (7 blind + 7 blind-folded) shows that the users were able to perceive the facial expressions in most of the time. In addition, the overall subjective usability of the system was found to be scoring 4.02 in a 5 point Likert scale.
- M. Argyle, V. Salter, H. Nicholson, M. Williams, and P. Burgess. The communication of inferior and superior attitudes by verbal and non-verbal signals*. British journal of social and clinical psychology, 9(3):222--231, 1970.Google Scholar
- D. Astler, H. Chau, K. Hsu, A. Hua, A. Kannan, L. Lei, M. Nathanson, E. Paryavi, M. Rosen, H. Unno, et al. Increased accessibility to nonverbal communication through facial and expression recognition technologies for blind/visually impaired subjects. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility, pages 259--260. ACM, 2011. Google ScholarDigital Library
- P. Bach-y Rita, C. Collins, F. Saunders, B. White, and L. Scadden. Vision substitution by tactile image projection. 1969.Google Scholar
- H. Beyer and K. Holtzblatt. Contextual design: defining customer-centered systems. Morgan Kaufmann, 1997. Google ScholarDigital Library
- P. Ekman and W. Friesen. Facial action coding system. Consulting Psychologists Press, Stanford University, Palo Alto, 1977.Google Scholar
- N. A. Giudice, H. P. Palani, E. Brenner, and K. M. Kramer. Learning non-visual graphical information using a touch-based vibro-audio interface. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility, ASSETS '12, pages 103--110, 2012. Google ScholarDigital Library
- L. H. Goldish and H. E. Taylor. The optacon: A valuable device for blind persons. New Outlook for the Blind, 68(2):49--56, 1974.Google Scholar
- S. K. e. a. Kane. Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities. In ACM SIGACCESS conference on Computers and accessibility, pages 115--122. ACM, 2009. Google ScholarDigital Library
- J. Loomis, R. Klatzky, and N. Giudice. Sensory substitution of vision: importance of perceptual and cognitive processing. Assistive technology for blindness and low vision. CRC Press, Boca Raton, pages 161--193, 2012.Google Scholar
- P. Lucey, J. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In CVPR Workshops (CVPRW), 2010, pages 94--101. IEEE, 2010.Google ScholarCross Ref
- P. Meijer. An experimental system for auditory image representations. Biomedical Engineering, IEEE Transactions on, 39(2):112--121, 1992.Google Scholar
- M. Olivetti Belardinelli, S. Federici, F. Delogu, and M. Palmiero. Sonification of spatial information: audio-tactile exploration strategies by normal and blind subjects. Universal Access in Human-Computer Interaction. Intelligent and Ubiquitous Interaction Environments, pages 557--563, 2009. Google ScholarDigital Library
- G. Radvansky. Human memory. Recherche, 67:02, 2005.Google Scholar
- A. Rahman, M. Tanveer, and M. Yeasin. A spatio-temporal probabilistic framework for dividing and predicting facial action units. In ACII-Volume Part II, pages 598--607. Springer-Verlag, 2011. Google ScholarDigital Library
- B. S. Reddy and B. Chatterji. An fft-based technique for translation, rotation, and scale-invariant image registration. Image Processing, IEEE Transactions on, 5(8):1266--1271, 1996. Google ScholarDigital Library
- S. Réhman and L. Liu. ifeeling: Vibrotactile rendering of human emotions on mobile phones. Mobile Multimedia Processing, pages 1--20, 2010.Google ScholarCross Ref
- J. Saragih, S. Lucey, and J. Cohn. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 91(2):200--215, 2011. Google ScholarDigital Library
- M. I. Tanveer, A. I. Anam, A. M. Rahman, S. Ghosh, and M. Yeasin. Feps: a sensory substitution system for the blind to perceive facial expressions. In 14th ACM SIGACCESS conference on Computers and accessibility, ASSETS '12, pages 207--208, 2012. Google ScholarDigital Library
Index Terms
- Do you see what I see?: designing a sensory substitution device to access non-verbal modes of communication
Recommendations
"I Always Wanted to See the Night Sky": Blind User Preferences for Sensory Substitution Devices
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing SystemsSensory Substitution Devices (SSDs) convert visual information into another sensory channel (e.g. sound) to improve the everyday functioning of blind and visually impaired persons (BVIP). However, the range of possible functions and options for ...
Synchronized Expressions: An Auditory Interface for Naturally Harmonizing Facial Expressions between People with Visual Impairment and Sighted People
ISS Companion '23: Companion Proceedings of the 2023 Conference on Interactive Surfaces and SpacesWe propose a sound design concept that supports empathic communication between people with visual impairment (PVI) and interlocutors. The combination of sounds with short notes that repeat short notes (riff) representing the facial expressions of the ...
Should I Trust It When I Cannot See It?: Credibility Assessment for Blind Web Users
ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and AccessibilityAs users become increasingly more reliant on online resources to satisfy their information needs, care is needed to ensure that these resources are credible in nature, especially if a decision is to be taken based upon the information accessed. The ...
Comments