HeartSense: Ubiquitous Accurate Multi-Modal Fusion-based Heart Rate Estimation Using Smartphones

Abstract

Heart rate is one of the most important vital signals for personal health tracking. A number of smartphone-based heart rate estimation systems have been proposed over the years. However, they either depend on special hardware sensors or suffer from the high noise due to the weakness of the heart signals, affecting their accuracy in many practical scenarios.

Inspired by medical studies about the heart motion mechanics, we propose the HeartSense heart rate estimation system. Specifically, we show that the gyroscope sensor is the most sensitive sensor for measuring the heart rate. To further counter noise and handle different practical scenarios, we introduce a novel quality metric that allows us to fuse the different gyroscope axes in a probabilistic framework to achieve a robust and accurate estimate.

We have implemented and evaluated our system on different Android phones. Results using 836 experiments on different subjects in practical scenarios with a side-by-side comparison with other systems show that HeartSense can achieve 1.03 bpm median absolute error for heart rate estimation. This is better than the state-of-the-art by more than 147% in median error, highlighting HeartSense promise as a ubiquitous system for medical and personal well-being applications.

References

1. Ahmed H Abdelhafiz. 2002. Heart failure in older people: causes, diagnosis and treatment. Age and ageing 31, 1 (2002), 29--36.
2. Heba Abdelnasser, Khaled A Harras, and Moustafa Youssef. 2015. UbiBreathe: A ubiquitous non-invasive WiFi-based breathing estimator. In Proceedings of the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 277--286.
3. Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C Miller. 2015. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 837--846.
4. Heba Aly and Moustafa Youssef. 2016. Zephyr: Ubiquitous Accurate multi-Sensor Fusion-based Respiratory Rate Estimation Using Smartphones. In Proceedings of IEEE Conference on Computer Communications (INFOCOM).
5. J Bednar and T Watt. 1984. Alpha-trimmed means and their relationship to median filters. IEEE Transactions on acoustics, speech, and signal processing 32, 1 (1984), 145--153.
6. Gerald D Buckberg. 2002. Basic science review: the helix and the heart. The Journal of Thoracic and Cardiovascular Surgery 124, 5 (2002), 863--883.
7. Daniel G Carey. 2009. Quantifying differences in the fat burning zone and the aerobic zone: implications for training. The Journal of Strength 8 Conditioning Research 23, 7 (2009), 2090--2095.
8. The Helical Heart Company. 2005. The helical heart. http://www.helicalheart.com/video_center.htm. (2005). {Online. Last accessed 14-Feb-2017}.
9. Lilian De Greef, Mayank Goel, Min Joon Seo, Eric C Larson, James W Stout, James A Taylor, and Shwetak N Patel. 2014. Bilicam: using mobile phones to monitor newborn jaundice. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 331--342.
10. Lawrence T DeCarlo. 1997. On the meaning and use of kurtosis. Psychological methods 2, 3 (1997), 292.
11. M Di Rienzo, E Vaini, P Castiglioni, G Merati, P Meriggi, G Parati, A Faini, and F Rizzo. 2013. Wearable seismocardiography: Towards a beat-by-beat assessment of cardiac mechanics in ambulant subjects. Autonomic Neuroscience 178, 1 (2013), 50--59.
12. Malcolm Elliott and Alysia Coventry. 2012. Critical care: the eight vital signs of patient monitoring. Br J Nurs 21, 10(2012), 621--625.
13. Matthew W Gillman, William B Kannel, Albert Belanger, and Ralph B D’Agostino. 1993. Influence of heart rate on mortality among persons with hypertension: the Framingham Study. American heart journal 125, 4 (1993), 1148--1154.
14. Javier Hernandez, Daniel J McDuff, and Rosalind W Picard. 2015. Biophone: Physiology monitoring from peripheral smartphone motions. In Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE, 7180--7183.
15. Azumio Inc. 2010. Instant Heart Rate {Mobile application software}. (2010). Retrieved from https://play.google.com.
16. E Jonathan and Martin Leahy. 2010. Investigating a smartphone imaging unit for photoplethysmography. Physiological measurement 31, 11 (2010), N79.
17. C Omar F Kamlin, Colm PF O’Donnell, Neil J Everest, Peter G Davis, and Colin J Morley. 2006. Accuracy of clinical assessment of infant heart rate in the delivery room. Resuscitation 71, 3 (2006), 319--321.
18. William B Kannel, Craig Kannel, Ralph S Paffenbarger, and L Adrienne Cupples. 1987. Heart rate and cardiovascular mortality: the Framingham Study. American heart journal 113, 6 (1987), 1489--1494.
19. Federica Landreani, Alba Martin-Yebra, Claudia Casellato, Esteban Pavan, Carlo Frigo, Pierre-François Migeotte, and Enrico G Caiani. 2015. Feasibility study for beat-to-beat heart rate detection by smartphone’s accelerometers. In E-Health and Bioengineering Conference (EHB), 2015. IEEE, 1--4.
20. Eric C Larson, Mayank Goel, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, and Shwetak N Patel. 2012. SpiroSmart: using a microphone to measure lung function on a mobile phone. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 280--289.
21. Albert B Levin. 1966. A simple test of cardiac function based upon the heart rate changes induced by the Valsalva maneuver. The American journal of cardiology 18, 1 (1966), 90--99.
22. Wilburta Q Lindh, Marilyn Pooler, Carol D Tamparo, Barbara M Dahl, and Julie Morris. 2013. Delmar’s comprehensive medical assisting: administrative and clinical competencies. Cengage Learning.
23. Jian Liu, Yan Wang, Yingying Chen, Jie Yang, Xu Chen, and Jerry Cheng. 2015. Tracking vital signs during sleep leveraging off-the-shelf wifi. In Proceedings of the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 267--276.
24. NH Mahmood, N Uyop, N Zulkarnain, FK Che Harun, MF Kamarudin, and A Linoby. 2011. LED indicator for heart rate monitoring system in sport application. In Signal Processing and its Applications (CSPA), 2011 IEEE 7th International Colloquium on. IEEE, 64--66.
25. Alison M McManus, Rich SW Masters, Raija MT Laukkanen, CW Clare, Cindy HP Sit, and Fiona CM Ling. 2008. Using heart-rate feedback to increase physical activity in children. Preventive medicine 47, 4 (2008), 402--408.
26. Satoshi Nakatani. 2011. Left ventricular rotation and twist: why should we learn? Journal of Cardiovascular Ultrasound 19, 1 (2011).
27. Alexandros Pantelopoulos and Nikolaos G Bourbakis. 2010. Prognosis: a wearable health-monitoring system for people at risk: Methodology and modeling. IEEE Transactions on Information Technology in Biomedicine 14, 3 (2010), 613--621.
28. N. Patwari, L. Brewer, Q. Tate, O. Kaltiokallio, and M. Bocca. 2014. Breathfinding: A Wireless Network That Monitors and Locates Breathing in a Home. IEEE Journal of Selected Topics in Signal Processing 8, 1 (2014), 30--42.
29. Panagiotis Pelegris, K Banitsas, T Orbach, and Kostas Marias. 2010. A novel method to detect heart beat rate using a mobile phone. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. IEEE, 5488--5491.
30. Rong-Chao Peng, Wen-Rong Yan, Ning-Ling Zhang, Wan-Hua Lin, Xiao-Lin Zhou, and Yuan-Ting Zhang. 2015. Cuffless and Continuous Blood Pressure Estimation from the Heart Sound Signals. Sensors 15, 9 (2015), 23653--23666.
31. Tauhidur Rahman, Alexander T Adams, Ruth Vinisha Ravichandran, Mi Zhang, Shwetak N Patel, Julie A Kientz, and Tanzeem Choudhury. 2015. Dopplesleep: A contactless unobtrusive sleep sensing system using short-range doppler radar. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 39--50.
32. Ltd. Samsung Electronics Co. 2014. Samsung Health {Mobile application software}. (2014). Retrieved from https://play.google.com.
33. Partho P Sengupta, A Jamil Tajik, Krishnaswamy Chandrasekaran, and Bijoy K Khandheria. 2008. Twist mechanics of the left ventricle: principles and application. Journal of the American College of Cardiology 1, 3 (2008), 366--376.
34. Ghufran Shafiq and Kalyana C Veluvolu. 2014. Surface chest motion decomposition for cardiovascular monitoring. Scientific reports 4 (2014).
35. Li Shen, Joshua A Hagen, and Ian Papautsky. 2012. Point-of-care colorimetric detection with a smartphone. Lab on a Chip 12, 21 (2012), 4240--4243.
36. AF Silva, JP Carmo, PM Mendes, and JH Correia. 2011. Simultaneous cardiac and respiratory frequency measurement based on a single fiber Bragg grating sensor. Measurement Science and Technology 22, 7 (2011), 075801.
37. Francisco Torrent-Guasp, Mladen J. Kocica, Antonio F. Corno, Masashi Komeda, Francesc Carreras-Costa, A. Flotats, Juan Cosin-Aguillar, and Han Wen. 2005. Towards new understanding of the heart structure and function. Eurpean Journal of Cardiothoracic Surgery 27, 2 (2005), 191--201.
38. Aly Waninge, Annette AJ Van der Putten, Roy E Stewart, Bert Steenbergen, Ruud Van Wijck, and Cees P Van der Schans. 2013. Heart rate and physical activity patterns in persons with profound intellectual and multiple disabilities. The Journal of Strength 8 Conditioning Research 27, 11 (2013), 3150--3158.
39. Goya Wannamethee, A Gerald Shaper, and Peter W Macfarlane. 1993. Heart rate, physical activity, and mortality from cancer and other noncardiovascular diseases. American journal of epidemiology 137, 7 (1993), 735--748.
40. Hongjiang Wei, Magalie Viallon, Benedicte MA Delattre, Kevin Moulin, Feng Yang, Pierre Croisille, and Yuemin Zhu. 2015. Free-breathing diffusion tensor imaging and tractography of the human heart in healthy volunteers using wavelet-based image fusion. IEEE transactions on medical imaging 34, 1 (2015), 306--316.
41. Lide Zhang, Birjodh Tiwana, Robert P Dick, Zhiyun Qian, Z Morley Mao, Zhaoguang Wang, and Lei Yang. 2010. Accurate online power estimation and automatic battery behavior based power model generation for smartphones. In Hardware/Software Codesign and System Synthesis (CODES+ ISSS), 2010 IEEE/ACM/IFIP International Conference on. IEEE, 105--114.

Index Terms

• Human-centered computing

  • Ubiquitous and mobile computing