poster

Anywhere touchtyping: text input on arbitrary surface using depth sensing

Authors:

Adiyan Mujibiya,

Takashi Miyaki,

Jun RekimotoAuthors Info & Claims

UIST '10: Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology

Pages 443 - 444

https://doi.org/10.1145/1866218.1866262

Published: 03 October 2010 Publication History

Get Access

Abstract

In this paper, touch typing enabled virtual keyboard system using depth sensing on arbitrary surface is proposed. Keystroke event detection is conducted using 3-dimensional hand appearance database matching combined with fingertip's surface touch sensing. Our prototype system acquired hand posture depth map by implementing phase shift algorithm for Digital Light Processor (DLP) fringe projection on arbitrary flat surface. The system robustly detects hand postures on the sensible surface with no requirement of hand position alignment on virtual keyboard frame. The keystroke feedback is the physical touch to the surface, thus no specific hardware must be worn. The system works real-time in average of 20 frames per second.

Supplementary Material

JPG File (pp138-mujibiya.jpg)

Download
5.32 KB

MOV File (pp138-mujibiya.mov)

Download
23.61 MB

References

[1]

}}Mathias Kolsch and Matthew Turk. Keyboards without keyboards: A survey of virtual keyboards. In Proc. of Sensing and Input for Media-centric Systems, 2002.

Google Scholar

[2]

}}Du H., Oggier T., Lustenberger F., and Charbon E. A virtual keyboard based on true-3d optical ranging. In Proc. of British Machine Vision Conference, 2005.

Crossref

Google Scholar

[3]

}}P. S. Huang, C. Zhang, and F. P. Chiang. High-speed 3-d shape measurement based on digital fringe projection. Optical Engineering, 42(1):163--168, 2003.

Crossref

Google Scholar

[4]

}}Blindtype. {Online}. http://www.blindtype.com.

Google Scholar

Cited By

View all

Maman BBermano A(2022)TypeNet: Towards Camera Enabled Touch Typing on Flat Surfaces through Self-Refinement2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV51458.2022.00064(567-576)Online publication date: Jan-2022
https://doi.org/10.1109/WACV51458.2022.00064
Zhang SZhang S(2019)A novel Human-3DTV Interaction System Based on Free Hand Gestures and a Touch-based Virtual InterfaceIEEE Access10.1109/ACCESS.2019.2953798(1-1)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2953798
Wu HWang J(2018)A visual attention-based method to address the midas touch problem existing in gesture-based interactionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-014-1060-032:1(123-136)Online publication date: 28-Dec-2018
https://dl.acm.org/doi/10.1007/s00371-014-1060-0
Show More Cited By

Index Terms

Anywhere touchtyping: text input on arbitrary surface using depth sensing
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces

Recommendations

Visual touchpad: a two-handed gestural input device
ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

This paper presents the Visual Touchpad, a low-cost vision-based input device that allows for fluid two-handed interactions with desktop PCs, laptops, public kiosks, or large wall displays. Two downward-pointing cameras are attached above a planar ...
LightBeam: nomadic pico projector interaction with real world objects
CHI EA '12: CHI '12 Extended Abstracts on Human Factors in Computing Systems

Pico projectors have lately been investigated as mobile display and interaction devices. We propose to use them as 'light beams': Everyday objects sojourning in a beam are turned into dedicated projection surfaces and tangible interaction devices. While ...
Toward Designing a New Virtual Keyboard When All Finger Movements Are Known
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

Compared with the physical QWERTY keyboards, the virtual keyboards are slow, inaccurate, and inconvenient because they simply imitate the traditional QWERTY keyboard. To improve the virtual keyboards, we focus on two observations. First, all alphabetic ...

Comments

Information & Contributors

Information

Published In

UIST '10: Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology

October 2010

126 pages

ISBN:9781450304627

DOI:10.1145/1866218

General Chair:
Ken Perlin
New York University
,
Program Chairs:
Mary Czerwinski
Microsoft Research
,
Rob Miller
MIT CSAIL

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

UIST '10

Sponsor:

UIST '10: The 23nd Annual ACM Symposium on User Interface Software and Technology

October 3 - 6, 2010

New York, New York, USA

Acceptance Rates

Overall Acceptance Rate 355 of 1,733 submissions, 20%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
522
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Maman BBermano A(2022)TypeNet: Towards Camera Enabled Touch Typing on Flat Surfaces through Self-Refinement2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV51458.2022.00064(567-576)Online publication date: Jan-2022
https://doi.org/10.1109/WACV51458.2022.00064
Zhang SZhang S(2019)A novel Human-3DTV Interaction System Based on Free Hand Gestures and a Touch-based Virtual InterfaceIEEE Access10.1109/ACCESS.2019.2953798(1-1)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2953798
Wu HWang J(2018)A visual attention-based method to address the midas touch problem existing in gesture-based interactionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-014-1060-032:1(123-136)Online publication date: 28-Dec-2018
https://dl.acm.org/doi/10.1007/s00371-014-1060-0
Wu HWang JZhang X(2016)User-centered gesture development in TV viewing environmentMultimedia Tools and Applications10.1007/s11042-014-2323-575:2(733-760)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s11042-014-2323-5
Brown JWilson JGossage SHack CBiddle R(2013)Surface Computing and Collaborative Analysis WorkSynthesis Lectures on Human-Centered Informatics10.2200/S00492ED1V01Y201303HCI0196:4(1-168)Online publication date: 31-Aug-2013
https://doi.org/10.2200/S00492ED1V01Y201303HCI019

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

Supplementary Material

References

Cited By

Index Terms

Recommendations

Visual touchpad: a two-handed gestural input device

LightBeam: nomadic pico projector interaction with real world objects

Toward Designing a New Virtual Keyboard When All Finger Movements Are Known

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations