poster

Using pen input features as indices of cognitive load

Authors:

Fang ChenAuthors Info & Claims

ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces

Pages 315 - 318

https://doi.org/10.1145/1322192.1322246

Published: 12 November 2007 Publication History

Get Access

Abstract

Multimodal interfaces are known to be useful in map-based applications, and in complex, time-pressure based tasks. Cognitive load variations in such tasks have been found to impact multimodal behaviour. For example, users become more multimodal and tend towards semantic complementarity as cognitive load increases. The richness of multimodal data means that systems could monitor particular input features to detect experienced load variations. In this paper, we present our attempt to induce controlled levels of load and solicit natural speech and pen-gesture inputs. In particular, we analyse for these features in the pen gesture modality. Our experimental design relies on a map-based Wizard of Oz, using a tablet PC. This paper details analysis of pen-gesture interaction across subjects, and presents suggestive trends of increases in the degree of degeneration of pen-gestures in some subjects, and possible trends in gesture kinematics, when cognitive load increases.

References

[1]

Baddeley, A., Working Memory. Science, 1992. 255: 556--559.

Google Scholar

[2]

Berthold, A., & Jameson, A. Interpreting Symptoms of Cognitive Load in Speech Input. In J. Kay (Ed.), UM99, User modeling: Proceedings of the Seventh International Conference. Vienna: Springer Wien New York. (1999) 235--244.

Digital Library

Google Scholar

[3]

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81--97

Google Scholar

[4]

Oviatt, S., Coulston, R., and Lunsford, R. When Do We Interact Multimodally? Cognitive Load and Multimodal Communication Pattern. In ICMI, (Penn State University, PA, USA, 2004). (2004), 8.

Digital Library

Google Scholar

[5]

Paas, F., Tuovinen, J. E., Tabbers, H., and Van Gerven, P. W. M. Cognitive Load Measurements as a mean to Advance Cognitive Load Theory. Educational Psychologist, 38, 1 (2003), 63--71.

Crossref

Google Scholar

[6]

Rubine, D. Specifying Gestures by Example. In SIGGRAPH'91: Proceedings of the 18th annual conference on Computer graphics and interactive techniques, (Las Vegas, 4 July 1991). ACM Press, New York, NY, USA, (1991), 329--337.

Digital Library

Google Scholar

[7]

Ruiz, N., Taib, R. and Chen, F. Examining Redundancy in Multimodal Input. In OzCHI06, (University of Technology, Sydney, NSW, Australia, 2006). (2006)

Digital Library

Google Scholar

Cited By

View all

Suryani MSensuse DSantoso HAji RHadi SSuryono RKautsarina (2024)An initial user model design for adaptive interface development in learning management system based on cognitive loadCognition, Technology and Work10.1007/s10111-024-00772-826:4(653-672)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10111-024-00772-8
Kosch TKarolus JZagermann JReiterer HSchmidt AWoźniak P(2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3582272
Sobocinski MDever DWiedbusch MMubarak FAzevedo RJärvelä S(2023)Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal dataBritish Journal of Educational Technology10.1111/bjet.13393Online publication date: 30-Sep-2023
https://doi.org/10.1111/bjet.13393
Show More Cited By

Index Terms

Using pen input features as indices of cognitive load
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence
      1. Cognitive science
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

Multimodal behavior and interaction as indicators of cognitive load
Special issue on highlights of the decade in interactive intelligent systems

High cognitive load arises from complex time and safety-critical tasks, for example, mapping out flight paths, monitoring traffic, or even managing nuclear reactors, causing stress, errors, and lowered performance. Over the last five years, our research ...
Symptoms of cognitive load in interactions with a dialogue system
MCPMD '18: Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data

Humans adapt their behaviour to the perceived cognitive load of their dialogue partner, for example, delaying non-essential information. We propose that spoken dialogue systems should do the same, particularly in high-stakes scenarios, such as emergency ...
Estimating cognitive load using pupil diameter during a spoken dialogue task
AutomotiveUI '13: Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

We explore the feasibility of using pupil diameter to estimate how the cognitive load of the driver changes during a spoken dialogue task with a remote conversant. The conversants play a series of Taboo games, which do not follow a structured turn-...

Comments

Information & Contributors

Information

Published In

ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces

November 2007

402 pages

ISBN:9781595938176

DOI:10.1145/1322192

General Chairs:
Kenji Mase
Nagoya University, Japan
,
Dominic Massaro
UC Santa Cruz, USA
,
Program Chairs:
Kazuya Takeda
Nagoya University, Japan
,
Deb Roy
MIT, USA
,
Alexandros Potamianos
Technical University of Crete, Greece

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICMI07

Sponsor:

ICMI07: International Conference on Multimodal Interface

November 12 - 15, 2007

Aichi, Nagoya, Japan

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)2

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Suryani MSensuse DSantoso HAji RHadi SSuryono RKautsarina (2024)An initial user model design for adaptive interface development in learning management system based on cognitive loadCognition, Technology and Work10.1007/s10111-024-00772-826:4(653-672)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10111-024-00772-8
Kosch TKarolus JZagermann JReiterer HSchmidt AWoźniak P(2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3582272
Sobocinski MDever DWiedbusch MMubarak FAzevedo RJärvelä S(2023)Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal dataBritish Journal of Educational Technology10.1111/bjet.13393Online publication date: 30-Sep-2023
https://doi.org/10.1111/bjet.13393
Altmeyer KBarz MLauer LPeschel MSonntag DBrünken RMalone S(2023)Digital ink and differentiated subjective ratings for cognitive load measurement in middle childhoodBritish Journal of Educational Psychology10.1111/bjep.1259593:S2(368-385)Online publication date: 26-Mar-2023
https://doi.org/10.1111/bjep.12595
Oppelt MFoltyn ADeuschel JLang NHolzer NEskofier BYang S(2022)ADABase: A Multimodal Dataset for Cognitive Load EstimationSensors10.3390/s2301034023:1(340)Online publication date: 28-Dec-2022
https://doi.org/10.3390/s23010340
Wu ZLin TTang N(2020)Explore the Use of Handwriting Information and Machine Learning Techniques in Evaluating Mental WorkloadCognitive Analytics10.4018/978-1-7998-2460-2.ch072(1423-1439)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-2460-2.ch072
Scheiter KAckerman RHoogerheide V(2020)Looking at Mental Effort Appraisals through a Metacognitive Lens: Are they Biased?Educational Psychology Review10.1007/s10648-020-09555-9Online publication date: 1-Aug-2020
https://doi.org/10.1007/s10648-020-09555-9
Moissa BBonnin GBoyer A(2019)Exploiting Wearable Technologies to Measure and Predict Students’ EffortPerspectives on Wearable Enhanced Learning (WELL)10.1007/978-3-319-64301-4_19(411-431)Online publication date: 2-Nov-2019
https://doi.org/10.1007/978-3-319-64301-4_19
Nourbakhsh NChen FWang YCalvo R(2017)Detecting Users’ Cognitive Load by Galvanic Skin Response with Affective InterferenceACM Transactions on Interactive Intelligent Systems10.1145/29604137:3(1-20)Online publication date: 19-Sep-2017
https://dl.acm.org/doi/10.1145/2960413
Vatavu R(2017)Characterizing gesture knowledge transfer across multiple contexts of useJournal on Multimodal User Interfaces10.1007/s12193-017-0247-x11:4(301-314)Online publication date: 29-Aug-2017
https://doi.org/10.1007/s12193-017-0247-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Multimodal behavior and interaction as indicators of cognitive load

Symptoms of cognitive load in interactions with a dialogue system

Estimating cognitive load using pupil diameter during a spoken dialogue task

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations