FingerReader: A Wearable Device to Explore Printed Text on the Go

ABSTRACT

Accessing printed text in a mobile context is a major challenge for the blind. A preliminary study with blind people reveals numerous difficulties with existing state-of-the-art technologies including problems with alignment, focus, accuracy, mobility and efficiency. In this paper, we present a finger-worn device, FingerReader, that assists blind users with reading printed text on the go. We introduce a novel computer vision algorithm for local-sequential text scanning that enables reading single lines, blocks of text or skimming the text with complementary, multimodal feedback. This system is implemented in a small finger-worn form factor, that enables a more manageable eyes-free operation with trivial setup. We offer findings from three studies performed to determine the usability of the FingerReader.

References

1. Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., and Yeh, T. Vizwiz: Nearly real-time answers to visual questions. In Proc. of UIST, ACM (2010), 333--342.
2. Black, A. W., and Lenzo, K. A. Flite: a small fast run-time synthesis engine. In 4th ISCA Tutorial and Research Workshop (ITRW) on Speech Synthesis (2001).
3. d'Albe, E. F. On a type-reading optophone. Proceedings of the Royal Society of London. Series A 90, 619 (1914), 373--375.
4. Ezaki, N., Bulacu, M., and Schomaker, L. Text detection from natural scene images: towards a system for visually impaired persons. In Proc. of ICPR, vol. 2 (2004), 683--686.
5. Hanif, S. M., and Prevost, L. Texture based text detection in natural scene images-a help to blind and visually impaired persons. In CVHI (2007).
6. Harada, S., Sato, D., Adams, D. W., Kurniawan, S., Takagi, H., and Asakawa, C. Accessible photo album: Enhancing the photo sharing experience for people with visual impairment. In Proc. of CHI, ACM (2013), 2127--2136.
7. Kane, S. K., Frey, B., and Wobbrock, J. O. Access lens: a gesture-based screen reader for real-world documents. In Proc. of CHI, ACM (2013), 347--350.
8. Levenshtein, V. I. Binary codes capable of correcting deletions, insertions and reversals. In Soviet physics doklady, vol. 10 (1966), 707.
9. Lévesque, V. Blindness, technology and haptics. Center for Intelligent Machines (2005), 19--21.
10. Linvill, J. G., and Bliss, J. C. A direct translation reading aid for the blind. Proceedings of the IEEE 54, 1 (1966), 40--51.
11. Mattar, M., Hanson, A., and Learned-Miller, E. Sign classification using local and meta-features. In CVPR Workshops, IEEE (June 2005), 26--26.
12. Nanayakkara, S., Shilkrot, R., Yeo, K. P., and Maes, P. EyeRing: a finger-worn input device for seamless interactions with our surroundings. In Proc. of Augmented Human (2013), 13--20.
13. Pavey, S., Dodgson, A., Douglas, G., and Clements, B. Travel, transport, and mobility of people who are blind and partially sighted in the uk. Royal National Institute for the Blind (RNIB), April 2009.
14. Pazio, M., Niedzwiecki, M., Kowalik, R., and Lebiedz, J. Text detection system for the blind. In 15th european signal processing conference EUSIPCO (2007), 272--276.
15. Peters, J.-P., Thillou, C., and Ferreira, S. Embedded reading device for blind people: a user-centered design. In Proc. of ISIT, IEEE (2004), 217--222.
16. Rissanen, M. J., Vu, S., Fernando, O. N. N., Pang, N., and Foo, S. Subtle, natural and socially acceptable interaction techniques for Ringterfaces-Finger-Ring shaped user interfaces. In Distributed, Ambient, and Pervasive Interactions. Springer, 2013, 52--61.
17. Sears, A., and Hanson, V. Representing users in accessibility research. In Proc. of CHI, ACM (2011), 2235--2238.
18. Shen, H., and Coughlan, J. M. Towards a real-time system for finding and reading signs for visually impaired users. In Proc. of ICCHP, Springer (2012), 41--47.
19. Shilkrot, R., Huber, J., Liu, C., Maes, P., and Nanayakkara, S. C. Fingerreader: A wearable device to support text reading on the go. In CHI EA, ACM (2014), 2359--2364.
20. Shinohara, K., and Tenenberg, J. A blind person's interactions with technology. Commun. ACM 52, 8 (Aug. 2009), 58--66.
21. Shinohara, K., and Wobbrock, J. O. In the shadow of misperception: Assistive technology use and social interactions. In Proc. of CHI, ACM (2011), 705--714.
22. Smith, R. An overview of the tesseract OCR engine. In Proc. of ICDAR, vol. 2, IEEE (2007), 629--633.
23. Stearns, L., Du, R., Oh, U., Wang, Y., Findlater, L., Chellappa, R., and Froehlich, J. E. The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind. In Workshop on Assistive Computer Vision and Robotics, ECCV, Springer (2014).
24. Yi, C., and Tian, Y. Assistive text reading from complex background for blind persons. In Camera-Based Document Analysis and Recognition. Springer, 2012, 15--28.

Index Terms

• Hardware

  • Communication hardware, interfaces and storage

    • Sound-based input / output

• Social and professional topics