Mago: Mode of Transport Inference Using the Hall-Effect Magnetic Sensor and Accelerometer

Abstract

In this paper, we introduce Mago, a novel system that can infer a person's mode of transport (MOT) using the Hall-effect magnetic sensor and accelerometer present in most smart devices. When a vehicle is moving, the motions of its mechanical components such as the wheels, transmission and the differential distort the earth's magnetic field. The magnetic field is distorted corresponding to the vehicle structure (e.g., bike chain or car transmission system), which manifests itself as a strong signal for sensing a person's transportation modality. We utilize this magnetic signal combined with the accelerometer and design a robust algorithm for the MOT detection. In particular, our system extracts frame-based features from the sensor data and can run in nearly real-time with only a few seconds of delay. We evaluated Mago using over 70 hours of daily commute data from 7 participants and the leave-one-out analysis of our cross-user, cross-device model reports an average accuracy of 94.4% among seven classes (stationary, bus, bike, car, train, light rail and scooter). Besides MOT, our system is able to reliably differentiate the phone's in-car position at an average accuracy of 92.9%. We believe Mago could potentially benefit many contextually-aware applications that require MOT detection such as a digital personal assistant or a life coaching application.

References

1. L. Bao and S.S. Intille. 2004. Activity Recognition from User-Annotated Acceleration Data. Lecture Notes in Computer Science. Berlin, Heidelberg.
2. A. Bloch, R. Erdin, S. Meyer, T. Keller, and A. de Spindler. 2015. Battery-Efficient Transportation Mode Detection on Mobile Devices. IEEE International Conference on Mobile Data Management (MDM'15), pp.185-190.
3. C. Bo, X. Jian, X.-Y. Li, X. Mao, Y. Wang, and F. Li. 2013. You're driving and texting: detecting drivers using personal smart phones by leveraging inertial sensors, In Proceedings of ACM International Conference on Mobile Computing and Networking (MobiCom'13), pp. 199--202.
4. N.V. Chawla, K.W. Bowyer, L.O. Hall, and W.P. Kegelmeyer. 2011. SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 2011.
5. K.-Y. Chen, G.A. Cohn, S. Gupta, and S.N. Patel. 2013. uTouch: sensing touch gestures on unmodified LCDs, In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI'13), pp. 2581--2584.
6. K.-Y. Chen, S. Gupta, E. Larson, and S.N. Patel. 2015. DOSE: Detecting User-driven Operating States of Electronic Devices From a Single Sensing Point. IEEE International Conference on Pervasive Computing and Communications (PerCom'15).
7. K.-Y. Chen, M. Harniss, S. Patel, and K. Johnson. 2013. Implementing technology-based embedded assessment in the home and community life of individuals aging with disabilities: a participatory research and development study. Journal of Disability Rehabilitation Assistive Technology, pp. 1--9, 2013.
8. K.-Y. Chen, K. Lyons, S. White, and S. Patel. 2013. uTrack: 3D Input Using Two Magnetic Sensors. In Proceedings of ACM User Interface Software and Technology (UIST'13), pp. 237--244.
9. K.-Y. Chen, S.N. Patel, and S. Keller. 2016. Finexus: Tracking Precise Motions of Multiple Fingertips Using Magnetic Sensing. In Proceedings of ACM International Conference on Human Factors in Computing Systems (CHI'13), pp. 1504--1514.
10. S. Consolvo et al. 2008. Activity sensing in the wild: a field trial of ubifit garden. In Proceedings of ACM International Conference on Human Factors in Computing Systems (CHI'08), pp. 1797--1806.
11. J. Froehlich, T. Dillahunt, P. Klasnja, J. Mankoff, S. Consolvo, B. Harrison, and J. A. Landay. 2009. UbiGreen: investigating a mobile tool for tracking and supporting green transportation habits, In Proceedings of ACM International Conference on Human Factors in Computing Systems (CHI'09), pp. 1043--1052.
12. I. Geis and W. Schulz. 2015. Future role of cost-benefit analysis in intelligent transport system-research. Journal of Intelligent Transport Systems (IET), Vol. 9, Issue 6. (August, 2015), pp. 626--632.
13. A. Glasmeier and S. Christopherson. 2015. Thinking about smart cities. Cambridge Jornal Regions Economy Society, Vol. 8, Issue 1 (March 2015), pp. 3--12.
14. S. Gupta, M.S. Reynolds, and S.N. Patel. 2010. ElectriSense: Single-Point Sensing Using EMI for Electrical Event Detection and Classification in the Home. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'10), pp. 139--148.
15. Z. He and L. Jin. 2009. Activity recognition from acceleration data based on discrete consine transform and SVM. IEEE International Conference on Systems, Man and Cybernetics (SMC'09), pp. 5041--5044.
16. S. Hemminki, P. Nurmi, and S. Tarkoma. 2013. Accelerometer-based transportation mode detection on smartphones. In Proceedings of ACM International Conference on Embedded Networked Sensor Systems (SenSys'13).
17. T. Huynh, M. Fritz, and B. Schiele. 2008. Discovery of activity patterns using topic models. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'08), pp. 10--19.
18. H. Itoh. 1980. Magnetic Field Sensor and Its Application to Automobiles. Sensors for Automotive Systems (SAE) Technical Paper 800123.
19. M.B. Kraichman. 1970. Handbook of electromagnetic propagation in conducting media. (1970).
20. N. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. Campbell. 2010. A survey of mobile phone sensing. IEEE Communication Magazine, Vol. 48, Issue 9 (Sept. 2010), pp. 140--150.
21. G. Lemaitre, F. Nogueira, and C.K. Aridas. 2016. Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning. In arXiv:1609.06570.
22. J. Lenz and S. Edelstein. 2006. Magnetic sensors and their applications. Journal of Sensors, Vol. 6, Issue 3 (June 2006), pp. 631--649.
23. J. Lester, T. Choudhury, and G. Borriello. 2006. A Practical Approach to Recognizing Physical Activities. Lecture Notes in Computer Science. Berlin, Heidelberg.
24. L. Liao, D. Fox, and H. Kautz. 2007. Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields. The International Journal of Robotics Research, Vol. 26, Issue 1 (January 2007), pp. 119--134.
25. L. Liao, D.J. Patterson, D. Fox, and H. Kautz. 2007. Learning and inferring transportation routines. Journal of Artificial Intelligence, Vol. 171, Issue 5-6 (April 2007), p. 311--331.
26. H. Lu, J. Yang, Z. Liu, N.D. Lane, T. Choudhury, and A.T. Campbell. 2010. The Jigsaw continuous sensing engine for mobile phone applications. In Proceedings of ACM International Conference on Embedded Networked Sensor Systems (SenSys'10), pp. 71--84.
27. Magnetic Shields Ltd. 2012. Magnetic shielding and the distortion of magnetic fields. (2012).
28. J. Maitland et al. 2006. Increasing the Awareness of Daily Activity Levels with Pervasive Computing. IEEE Pervasive Health Conference and Workshops, pp. 1--9.
29. V. Markevicius, D. Navikas, M. Zilys, D. Andriukaitis, A. Valinevicius, and M. Cepenas. 2016. Dynamic Vehicle Detection via the Use of Magnetic Field Sensors. Journal of Sensors, Vol. 16, Issue 1 (January 2016).
30. E. Miluzzo et al. 2008. Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application. In Proceedings of ACM International Conference on Embedded Networked Sensor Systems (SenSys'08), pp. 337--350.
31. G. Mone. 2015. The new smart cities. ACM Magazine of Communications, Vol. 58, Issue 7 (June 2015), pp. 20--21.
32. S.N. Patel, T. Robertson, J.A. Kientz, M.S. Reynolds, and G.D. Abowd. 2007. At the flick of a switch: detecting and classifying unique electrical events on the residential power line. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'07), pp. 271--288.
33. N. Ravi, N. Dandekar, P. Mysore, and M.L. Littman. 2005. Activity recognition from accelerometer data. In Proceedings of ACM International Conference on Innovative Applications of Artificial Intelligence (IAAI'05), pp. 1541--1546.
34. S. Reddy, M. Mun, J. Burke, D. Estrin, M. Hansen, and M. Srivastava. 2010. Using mobile phones to determine transportation modes. ACM Transactions on Sensor Networks (TOSN). Vol. 6, Issue 2 (February 2010), pp. 13--27.
35. R.C. Shah, C.-Y. Wan, H. Lu, and L. Nachman. 2014. Classifying the mode of transportation on mobile phones using GIS information. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'14 adjunct), pp. 225--229.
36. T. Sohn et al. 2006. Mobility Detection Using Everyday GSM Traces. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'06), pp. 212--224.
37. L. Stenneth, O. Wolfson, P.S. Yu, and B. Xu. 2011. Transportation mode detection using mobile phones and GIS information. In Proceedings of ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (GIS'11), pp. 54--63.
38. E.J. Wang, T.-J. Lee, A. Mariakakis, M. Goel, S. Gupta, and S.N. Patel. 2015. MagnifiSense: inferring device interaction using wrist-worn passive magneto-inductive sensors. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'15), pp. 15--26.
39. S. Wang, C. Chen, and J. Ma. 2010. Accelerometer Based Transportation Mode Recognition on Mobile Phones. In IEEE Asia-Pacific Conference on Wearable Computing Systems (APWCS'10), pp. 44--46.
40. Y. Wang, J. Yang, H. Liu, Y. Chen, M. Gruteser, and R.P. Martin. 2013. Sensing vehicle dynamics for determining driver phone use, In Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys'13), pp. 41--54.
41. Y. Wang, Y. J. Chen, J. Yang, M. Gruteser, R. Martin, H. Liu, L. Liu and C. Karatas. 2016. Determining Driver Phone Use by Exploiting Smartphone Integrated Sensors. IEEE Trans. on Mobile Computing, Vol. 15, Issue 8 (August 2016), pp. 1965--1981.
42. M. Zhang and A.A. Sawchuk. 2012. A preliminary study of sensing appliance usage for human activity recognition using mobile magnetometer. In Proceedings of ACM International Conference on Ubiquitous Computing (UbiComp'12), pp. 745--748.
43. Y. Zheng, Y. Chen, Q. Li, X. Xie, and W.-Y. Ma. 2010. Understanding transportation modes based on GPS data for web applications. ACM Transactions on the Web (TWEB), Vol. 4, Issue 1 (January 2010).
44. P. Zhou, Y. Zheng, and M. Li. 2013. How Long to Wait? Predicting Bus Arrival Time With Mobile Phone Based Participatory Sensing. IEEE Transaction on Mobile Computing. Vol. 13, Issue 6 (October 2013), pp. 1228--1241.