Ming Zeng
ABSTRACT
Deep neural networks, including recurrent networks, have been successfully applied to human activity recognition. Unfortunately, the final representation learned by recurrent networks might encode some noise (irrelevant signal components, unimportant sensor modalities, etc.). Besides, it is difficult to interpret the recurrent networks to gain insight into the models' behavior. To address these issues, we propose two attention models for human activity recognition: temporal attention and sensor attention. These two mechanisms adaptively focus on important signals and sensor modalities. To further improve the understandability and mean Fl score, we add continuity constraints, considering that continuous sensor signals are more robust than discrete ones. We evaluate the approaches on three datasets and obtain state-of-the-art results. Furthermore, qualitative analysis shows that the attention learned by the models agree well with human intuition.
- Mohammad Abu Alsheikh, Ahmed Selim, Dusit Niyato, Linda Doyle, Shaowei Lin, and Hwee-Pink Tan. 2016. Deep Activity Recognition Models with Triaxial Accelerometers.. In AAAI Workshop: Artificial Intelligence Applied to Assistive Technologies and Smart Environments.Google Scholar
- Marc Bachlin, Daniel Roggen, Gerhard Troster, Meir Plotnik, Noit Inbar, Inbal Meidan, Talia Herman, Marina Brozgol, Eliya Shaviv, Nir Giladi, and others. 2009. Potentials of enhanced context awareness in wearable assistants for Parkinson's disease patients with the freezing of gait syndrome. In ISWC. IEEE, 123--130. Google ScholarDigital Library
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- Ling Bao and Stephen S Intille. 2004. Activity recognition from user-annotated acceleration data. In PerComp. Springer, 1--17.Google Scholar
- Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM CSUR 46, 3 (2014), 33. Google ScholarDigital Library
- Snehal Chennuru, Peng-Wen Chen, Jiang Zhu, and Joy Ying Zhang. 2012. Mobile Lifelogger-Recording, Indexing, and Understanding a Mobile Users Life. In MobiCASE. Springer, 263--281.Google Scholar
- Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. 2015. Attention-based models for speech recognition. In Advances in neural information processing systems. 577--585. Google ScholarDigital Library
- Yu Guan and Thomas Plötz. 2017. Ensembles of deep lstm learners for activity recognition using wearables. IMWUT 1, 2 (2017), 11. Google ScholarDigital Library
- G Gybenko. 1989. Approximation by superposition of sigmoidal functions. Mathematics of Control, Signals and Systems 2, 4 (1989), 303--314.Google ScholarCross Ref
- Nils Yannick Hammerla, James Fisher, Peter Andras, Lynn Rochester, Richard Walker, and Thomas Plötz. 2015. PD Disease State Assessment in Naturalistic Environments Using Deep Learning.. In AAAI. 1742--1748. Google ScholarDigital Library
- Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. IJCAI (2016). Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780. Google ScholarDigital Library
- Tâm Huynh and Bernt Schiele. 2005. Analyzing features for activity recognition. In Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies. ACM, 159--163. Google ScholarDigital Library
- Suyoun Kim and Ian Lane. 2017. End-to-End Speech Recognition with Auditory Attention for Multi-Microphone Distance Speech Recognition. Interspeech (2017), 3867--3871.Google Scholar
- Klas Leino, Linyi Li, Shayak Sen, Anupam Datta, and Matt Fredrikson. 2018. Influence-Directed Explanations for Deep Convolutional Networks. arXiv preprint arXiv:1802.03788 (2018).Google Scholar
- Francisco Javier Ordóñez Morales and Daniel Roggen. 2016. Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations. In ISWC. ACM, 92--99. Google ScholarDigital Library
- Sebastian Münzner, Philip Schmidt, Attila Reiss, Michael Hanselmann, Rainer Stiefelhagen, and Robert Durichen. 2017. CNN-based sensor fusion techniques for multimodal human activity recognition. In ISWC. ACM, 158--165. Google ScholarDigital Library
- Graham Neubig. 2017. Neural machine translation and sequence-to-sequence models: A tutorial. arXiv preprint arXiv.1703.01619 (2017).Google Scholar
- Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors 16, 1 (2016).Google Scholar
- Thomas Plötz, Nils Y Hammerla, and Patrick Olivier. 2011. Feature learning for activity recognition in ubiquitous computing. In IJCAI, Vol. 22. 1729.Google Scholar
- Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In ISWC. IEEE, 108--109. Google ScholarDigital Library
- Charissa Ann Ronao and Sung-Bae Cho. 2015. Deep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors. In NIP. Springer, 46--53.Google Scholar
- Xiao Wang, Tong Yu, Ming Zeng, and Patrick Tague. 2017. XRec: Behavior-Based User Recognition Across Mobile Devices. IMWUT 1, 3 (2017), 111. Google ScholarDigital Library
- Kelvin Xu, Jimmy Ba, Ryan Kiros, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. arXiv preprint arXiv:1502.03044 (2015).Google Scholar
- Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In IJCAI. AAAI Press, 3995--4001. Google ScholarDigital Library
- Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. Deepsense: A unified deep learning framework for time-series mobile sensing data processing. In WWW. 351--360. Google ScholarDigital Library
- Piero Zappi, Clemens Lombriser, Thomas Stiefmeier, Elisabetta Farella, Daniel Roggen, Luca Benini, and Gerhard Tröster. 2008. Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. In Wireless sensor networks. Springer, 17--33. Google ScholarDigital Library
- Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Juyong Zhang. 2014a. Convolutional Neural Networks for human activity recognition using mobile sensors. In MobiCASE. IEEE, 197--205.Google Scholar
- Ming Zeng, Xiao Wang, Le T Nguyen, Pang Wu, Ole J Mengshoel, and Joy Zhang. 2014b. Adaptive activity recognition with dynamic heterogeneous sensor fusion. In MobiCASE. IEEE, 189--196.Google Scholar
- Ming Zeng, Tong Yu, Xiao Wang, Le T Nguyen, Ole J Mengshoel, and Ian Lane. 2017. Semi-supervised convolutional neural networks for human activity recognition. In Big Data. IEEE, 522--529.Google Scholar
- Licheng Zhang, Xihong Wu, and Dingsheng Luo. 2015. Human activity recognition with HMM-DNN model. In ICCI* CC. IEEE, 192--197.Google Scholar
- Understanding and improving recurrent networks for human activity recognition by continuous attention
Recommendations
Benchmarking Reservoir and Recurrent Neural Networks for Human State and Activity Recognition
Advances in Computational IntelligenceAbstractMonitoring of human states from streams of sensor data is an appealing applicative area for Recurrent Neural Network (RNN) models. In such a scenario, Echo State Network (ESN) models from the Reservoir Computing paradigm can represent good ...
Self-Gated Recurrent Neural Networks for Human Activity Recognition on Wearable Devices
Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017This paper develops a self-gated recurrent neural network (SGRNN), and applies it to human activity recognition (HAR), using time-series signals collected from embedded sensors of wearable devices. Recurrent neural networks (RNNs) are very powerful for ...
BP Neural Network for Human Activity Recognition in Smart Home
CSSS '12: Proceedings of the 2012 International Conference on Computer Science and Service SystemIn this paper, BP (back propagation) neural network is applied to represent and recognize human activities from observed sensor sequences. The proper features for activity recognition are selected. Then, the model of BP neural network for activity ...
Comments