skip to main content
10.1145/2634317.2634323acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
research-article

Enhancing action recognition through simultaneous semantic mapping from body-worn motion sensors

Published: 13 September 2014 Publication History

Abstract

Locations and actions are interrelated: some activities tend to occur at specific places, for example a person is more likely to twist his wrist when he is close to a door (to turn the knob). We present an unsupervised fusion method that takes advantage of this characteristic to enhance the recognition of location-related actions (e.g., open, close, switch, etc.). The proposed LocAFusion algorithm acts as a post-processing filter: At run-time, it constructs a semantic map of the environment by tagging action recognitions to Cartesian coordinates. It then uses the accumulated information about a location i) to discriminate between identical actions performed at different places and ii) to correct recognitions that are unlikely, given the other observations at the same location. LocAFusion does not require prior statistics about where activities occur, which allows for seamless deployment to new environments. The fusion approach is agnostic to the sensor modalities and methods used for action recognition and localization.
For evaluation, we implemented a fully wearable setup that tracks the user with a foot-mounted motion sensor and the ActionSLAM algorithm. Simultaneously, we recognize hand actions through template matching on the data of a wrist-worn inertial measurement unit. In 10 recordings with 554 performed object interactions, LocAFusion consistently outperformed location-independent action recognition (8--31% increase in F1 score), identified 96% of the objects in the semantic map and overall correctly labeled 82% of the actions in problems with up to 23 classes.

Supplementary Material

MOV File (p99-hardegger.mov)

References

[1]
Berkhin, P. A survey of clustering data mining techniques. In Grouping multidimensional data. Springer, 2006, 25--71.
[2]
Berlin, E., and Van Laerhoven, K. Detecting leisure activities with dense motif discovery. In Ubiquitous Computing, 14th ACM Conference on, ACM (2012), 250--259.
[3]
Berndt, D. J., and Clifford, J. Using dynamic time warping to find patterns in time series. In KDD workshop, vol. 10, Seattle, WA (1994), 359--370.
[4]
Figo, D., Diniz, P. C., Ferreira, D. R., and Cardoso, J. M. Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing 14, 7 (2010), 645--662.
[5]
Grzonka, S., Karwath, A., Dijoux, F., and Burgard, W. Activity-based estimation of human trajectories. Robotics, IEEE Transactions on 28, 1 (2012), 234--245.
[6]
Hardegger, M., Roggen, D., Mazilu, S., and Tröster, G. Actionslam: Using location-related actions as landmarks in pedestrian slam. In Indoor Positioning and Indoor Navigation, Third International Conference on, IEEE (2012), 1--10.
[7]
Hirschberg, D. S. Algorithms for the longest common subsequence problem. Journal of the ACM 24, 4 (1977), 664--675.
[8]
Hoey, J., Plötz, T., Jackson, D., Monk, A., Pham, C., and Olivier, P. Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive and Mobile Computing 7, 3 (2011), 299--318.
[9]
Ko, M. H., West, G., Venkatesh, S., and Kumar, M. Using dynamic time warping for online temporal fusion in multisensor systems. Information Fusion 9, 3 (2008), 370--388.
[10]
Liao, L., Fox, D., and Kautz, H. Location-based activity recognition. Advances in Neural Information Processing Systems 18 (2006), 787.
[11]
Liao, L., Fox, D., and Kautz, H. Extracting places and activities from GPS traces using hierarchical conditional random fields. The International Journal of Robotics Research 26, 1 (2007), 119--134.
[12]
Lu, C.-H., and Fu, L.-C. Robust location-aware activity recognition using wireless sensor network in an attentive home. Automation Science and Engineering, IEEE Transactions on 6, 4 (2009), 598--609.
[13]
Nguyen-Dinh, L.-V., Roggen, D., Calatroni, A., and Tröster, G. Improving online gesture recognition with template matching methods in accelerometer data. In Intelligent Systems Design and Applications, 2012 12th International Conference on, IEEE (2012), 831--836.
[14]
Nüchter, A., and Hertzberg, J. Towards semantic maps for mobile robots. Robotics and Autonomous Systems 56, 11 (2008), 915--926.
[15]
Pham, C., and Olivier, P. Slice&dice: Recognizing food preparation activities using embedded accelerometers. In Ambient Intelligence. Springer, 2009, 34--43.
[16]
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Tröster, G., et al. Collecting complex activity datasets in highly rich networked sensor environments. In Networked Sensing Systems, Seventh International Conference on, IEEE (2010), 233--240.
[17]
Sokolova, M., and Lapalme, G. A systematic analysis of performance measures for classification tasks. Information Processing & Management 45, 4 (2009), 427--437.
[18]
Stiefmeier, T., Ogris, G., Junker, H., Lukowicz, P., and Tröster, G. Combining motion sensors and ultrasonic hands tracking for continuous activity recognition in a maintenance scenario. In Wearable Computers, 10th International Symposium on, IEEE (2006), 97--104.
[19]
Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., and Tröster, G. Wearable activity tracking in car manufacturing. Pervasive Computing 7, 2 (2008), 42--50.
[20]
Wang, L., Gu, T., Tao, X., and Lu, J. A hierarchical approach to real-time activity recognition in body sensor networks. Pervasive and Mobile Computing 8, 1 (2012), 115--130.
[21]
Wilson, D. H., and Atkeson, C. Simultaneous Tracking and Activity Recognition (STAR) using many anonymous, binary sensors. In Pervasive computing. Springer, 2005, 62--79.
[22]
Zheng, V. W., Zheng, Y., Xie, X., and Yang, Q. Collaborative location and activity recommendations with GPS history data. In World wide web, 19th international conference on, ACM (2010), 1029--1038.
[23]
Zinnen, A., Blanke, U., and Schiele, B. An analysis of sensor-oriented vs. model-based activity recognition. In Wearable Computers. 13th International Symposium on, IEEE (2009), 93--100.

Cited By

View all
  • (2024)Tracking daily paths in home contexts with RSSI fingerprinting based on UWB through deep learning modelsMultimedia Tools and Applications10.1007/s11042-024-19914-1Online publication date: 27-Aug-2024
  • (2020)Towards A Personal Mobile Cloud via Generic Device InterconnectionsIEEE Transactions on Mobile Computing10.1109/TMC.2019.292228919:12(2965-2980)Online publication date: 1-Dec-2020
  • (2019)WLCSSLearn: Learning Algorithm for Template Matching-based Gesture Recognition Systems2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR)10.1109/ICIEV.2019.8858539(91-96)Online publication date: May-2019
  • Show More Cited By

Index Terms

  1. Enhancing action recognition through simultaneous semantic mapping from body-worn motion sensors

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ISWC '14: Proceedings of the 2014 ACM International Symposium on Wearable Computers
    September 2014
    154 pages
    ISBN:9781450329699
    DOI:10.1145/2634317
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 September 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. LCSS
    2. SLAM
    3. activity recognition
    4. tracking
    5. wearable

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    UbiComp '14
    UbiComp '14: The 2014 ACM Conference on Ubiquitous Computing
    September 13 - 17, 2014
    Washington, Seattle

    Acceptance Rates

    Overall Acceptance Rate 38 of 196 submissions, 19%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)11
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Tracking daily paths in home contexts with RSSI fingerprinting based on UWB through deep learning modelsMultimedia Tools and Applications10.1007/s11042-024-19914-1Online publication date: 27-Aug-2024
    • (2020)Towards A Personal Mobile Cloud via Generic Device InterconnectionsIEEE Transactions on Mobile Computing10.1109/TMC.2019.292228919:12(2965-2980)Online publication date: 1-Dec-2020
    • (2019)WLCSSLearn: Learning Algorithm for Template Matching-based Gesture Recognition Systems2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR)10.1109/ICIEV.2019.8858539(91-96)Online publication date: May-2019
    • (2017)A Survey on Activity Detection and Classification Using Wearable SensorsIEEE Sensors Journal10.1109/JSEN.2016.262834617:2(386-403)Online publication date: 15-Jan-2017
    • (2017)iType: Using eye gaze to enhance typing privacyIEEE INFOCOM 2017 - IEEE Conference on Computer Communications10.1109/INFOCOM.2017.8057233(1-9)Online publication date: May-2017
    • (2017)Interconnecting heterogeneous devices in the personal mobile cloudIEEE INFOCOM 2017 - IEEE Conference on Computer Communications10.1109/INFOCOM.2017.8057083(1-9)Online publication date: May-2017
    • (2016)Unsupervised Head--Modifier Detection in Search QueriesACM Transactions on Knowledge Discovery from Data10.1145/298823511:2(1-28)Online publication date: 26-Dec-2016
    • (2016)Assignment Problems of Different-Sized Inputs in MapReduceACM Transactions on Knowledge Discovery from Data10.1145/298737611:2(1-35)Online publication date: 3-Dec-2016
    • (2016)ERfair Scheduler with Processor Suspension for Real-Time Multiprocessor Embedded SystemsACM Transactions on Design Automation of Electronic Systems10.1145/294897922:1(1-25)Online publication date: 13-Dec-2016
    • (2016)Probabilistic Model Checking for Uncertain Scenario-Aware Data FlowACM Transactions on Design Automation of Electronic Systems10.1145/291478822:1(1-27)Online publication date: 2-Sep-2016
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media