skip to main content
10.1145/1394445.1394457acmconferencesArticle/Chapter ViewAbstractPublication PagesdisConference Proceedingsconference-collections
research-article

Exploring true multi-user multimodal interaction over a digital table

Published: 25 February 2008 Publication History

Abstract

True multi-user, multimodal interaction over a digital table lets co-located people simultaneously gesture and speak commands to control an application. We explore this design space through a case study, where we implemented an application that supports the KJ creativity method as used by industrial designers. Four key design issues emerged that have a significant impact on how people would use such a multi-user multimodal system. First, parallel work is affected by the design of multimodal commands. Second, individual mode switches can be confusing to collaborators, especially if speech commands are used. Third, establishing personal and group territories can hinder particular tasks that require artefact neutrality. Finally, timing needs to be considered when designing joint multimodal commands. We also describe our model view controller architecture for true multi-user multimodal interaction.

References

[1]
Aliakseyeu, D., Subramanian, S., Lucero, A., and Gutwin, C. (2006) Interacting with piles of artifacts on digital tables. Proc. AVI '06, ACM Press, 159--162.
[2]
Bederson, B., Grosjean, J., & Meyer, J. (2004). Toolkit Design for Interactive Structured Graphics, IEEE Transactions on Software Engineering, 30 (8), pp. 535--546.
[3]
Boyle, M. and Greenberg, S. (2005) Rapidly Prototyping Multimedia Groupware. Proc. Distributed Multimedia Systems '05, Knowledge Systems Institute.
[4]
Buxton, W. (2007) Sketching User Experiences: Getting the Design Right and the Right Design, Morgan Kaufmann, ISBN-13: 978-0123740373.
[5]
Cox, D. and Greenberg, S. (2000) Supporting Collaborative Interpretation in Distributed Groupware. Proc. CSCW 00, ACM Press, 289--298.
[6]
Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L. and Clow, J., QuickSet: Multimodal interaction for distributed applications. Proc. ACM Multimedia, 1997, 31--40.
[7]
Foster, G. and Stefik, M. (1986) Cognoter: theory and practice of a collaborative tool. Proc. CSCW '86, ACM Press, 7--15.
[8]
Greenberg, S., Gutwin, C., and Roseman, M. (1996). Semantic Telepointers for Groupware. Proc OzCHI '96, IEEE Computer Society Press. 54--61.
[9]
Gutwin, C., and Greenberg, S. (2004) The importance of awareness for team cognition in distributed collaboration. In E. Salas, S. Fiore (Eds) Team Cognition: Understanding the Factors that Drive Process and Performance, APA Press, 177--201.
[10]
Hall, E. (1966) The Hidden Dimension. Anchor Books
[11]
Klemmer, S. R., Newman, M. W., Farrell, R., Bilezikjian, M., and Landay, J. A. (2001) The designers' outpost: a tangible interface for collaborative web site. Proc. UIST '01, ACM Press, 1--10.
[12]
Kruger, R., Carpendale, M. S. T., Scott, S. D., Greenberg, S.: Roles of Orientation in Tabletop Collaboration: Comprehension, Coordination and Communication. In Journal of Computer Supported Collaborative Work, 13(5--6), 2004, pp. 501--537.
[13]
Lucero, A., Martens, J. (2005) Mood Boards: Industrial Designers' Perception of Using Mixed Reality. Proc. SIGCHI. NL Conference 2005, 13--16.
[14]
Rogers, Y. and Lindley, S. (2004) Collaborating around vertical and horizontal large interactive displays: which way is best? Interacting with Computers (16), 1133--1152. Elsevier.
[15]
Scott, S. D., Carpendale, M. S. T, & Inkpen, K. M. (2004). Territoriality in Collaborative Tabletop Workspaces. Proc. CSCW '04, 294--303.
[16]
Somer, R., (1969) Personal Space: The Behavioural Basis of Design, Spectrum, ISBN 0-13-657577-3.
[17]
Tang, A., Tory, M., Po, B., Neumann, P., and Carpendale, M. S. T. (2006). Collaborative Coupling over Tabletop Displays. Proc. CHI '06. (April 24--27, Montreal, Quebec). pp: 1181--1190. ACM Press.
[18]
Tang, A., Neustaedter, C. and Greenberg, S. (2006) VideoArms: Embodiments for Mixed Presence Groupware. Proc. the 20th BCS-HCI British HCI 2006 Group Conference (Sept 11--15, Queen Mary, University of London, UK).
[19]
Tatar, D. G., Foster, G., and Bobrow, D. G. (1991) Design for conversation: lessons from Cognoter. Int. J. Man-Mach. Stud. 34, 2 (Feb. 1991), 185--209.
[20]
Tse, E., Shen, C., Greenberg, S. and Forlines, C. (2006) Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop. Proc. AVI'06, 336--343, ACM Press.
[21]
Tse, E., Shen, C., Greenberg, S., and Forlines, C. (2007) How pairs interact over a multimodal digital table. Proc. CHI 07, ACM Press, 215--218.
[22]
Tse, E., Greenberg, S. and Shen, C. (2006) GSI DEMO: Multiuser Gesture / Speech Interaction over Digital Tables by Wrapping Single User Applications. Proc. ICMI'06, ACM Press.
[23]
Tse, E., Greenberg, S., Shen, C., Barnwell, J., Shipman, S. and Leigh, D. (2007) Multimodal Split View Tabletop Interaction Over Existing Applications. Report 2007-869-21, Dept. of Computer Science, University of Calgary, Canada.
[24]
Vogel, D. and Baudisch, P. (2007) Shift: A Technique for Operating Pen-Based Interfaces Using Touch. Proc. CHI 2007. p. 657--666.
[25]
Wigdor, D., Balakrishnan, R. (2005). Empirical investigation into the effect of orientation on text readability in tabletop displays. Proc. ECSCW '05.
[26]
Yuizono, T., Munemori, J., and Nagasawa, Y. (1998) GUNGEN: Groupware for a New Idea Generation Consistent Support System. Proc. APCHI '98, IEEE Computer Society, 357--363.

Cited By

View all
  • (2025)Enhancing Group Ideation with a Distributed KJ Method Support SystemJournal of Information Processing10.2197/ipsjjip.33.6633(66-77)Online publication date: 2025
  • (2024)Talk to the Wall: The Role of Speech Interaction in Collaborative Visual AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345633531:1(941-951)Online publication date: 9-Sep-2024
  • (2023)Design-based implementation research: milestones and trade-offs in designing a collaborative representation tool for engineering classroomsEducational technology research and development10.1007/s11423-023-10288-z71:6(2457-2481)Online publication date: 9-Oct-2023
  • Show More Cited By

Index Terms

  1. Exploring true multi-user multimodal interaction over a digital table

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    DIS '08: Proceedings of the 7th ACM conference on Designing interactive systems
    February 2008
    487 pages
    ISBN:9781605580029
    DOI:10.1145/1394445
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 February 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. computer supported cooperative work
    2. multimodal speech and gesture interfaces
    3. tabletop interaction

    Qualifiers

    • Research-article

    Conference

    DIS08
    Sponsor:
    DIS08: Designing Interactive Systems Conference 2008
    February 25 - 27, 2008
    Cape Town, South Africa

    Acceptance Rates

    Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

    Upcoming Conference

    DIS '25
    Designing Interactive Systems Conference
    July 5 - 9, 2025
    Funchal , Portugal

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)34
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Enhancing Group Ideation with a Distributed KJ Method Support SystemJournal of Information Processing10.2197/ipsjjip.33.6633(66-77)Online publication date: 2025
    • (2024)Talk to the Wall: The Role of Speech Interaction in Collaborative Visual AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345633531:1(941-951)Online publication date: 9-Sep-2024
    • (2023)Design-based implementation research: milestones and trade-offs in designing a collaborative representation tool for engineering classroomsEducational technology research and development10.1007/s11423-023-10288-z71:6(2457-2481)Online publication date: 9-Oct-2023
    • (2021)ADQDAProceedings of the ACM on Human-Computer Interaction10.1145/34885345:ISS(1-19)Online publication date: 5-Nov-2021
    • (2021)Challenges and Opportunities in the Design of Digital Distributed Affinity Diagramming ToolsProceedings of the 32nd European Conference on Cognitive Ergonomics10.1145/3452853.3452871(1-5)Online publication date: 26-Apr-2021
    • (2021)Emerging ApplicationsTouch-Based Human-Machine Interaction10.1007/978-3-030-68948-3_7(179-229)Online publication date: 26-Mar-2021
    • (2020)Browser Based Digital Sticky Notes for Design ThinkingCompanion Publication of the 2020 ACM Designing Interactive Systems Conference10.1145/3393914.3395824(349-352)Online publication date: 6-Jul-2020
    • (2020)A Virtual Mouse Interface for Supporting Multi-user InteractionsHuman-Computer Interaction. Multimodal and Natural Interaction10.1007/978-3-030-49062-1_33(497-508)Online publication date: 10-Jul-2020
    • (2019)Modeling Multimodal-Multiuser Interactions in Declarative Multimedia LanguagesProceedings of the ACM Symposium on Document Engineering 201910.1145/3342558.3345400(1-10)Online publication date: 23-Sep-2019
    • (2019)Affinity LensProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300628(1-13)Online publication date: 2-May-2019
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media