RadarCat: Radar Categorization for Input & Interaction

ABSTRACT

In RadarCat we present a small, versatile radar-based system for material and object classification which enables new forms of everyday proximate interaction with digital devices. We demonstrate that we can train and classify different types of materials and objects which we can then recognize in real time. Based on established research designs, we report on the results of three studies, first with 26 materials (including complex composite objects), next with 16 transparent materials (with different thickness and varying dyes) and finally 10 body parts from 6 participants. Both leave one-out and 10-fold cross-validation demonstrate that our approach of classification of radar signals using random forest classifier is robust and accurate. We further demonstrate four working examples including a physical object dictionary, painting and photo editing application, body shortcuts and automatic refill based on RadarCat. We conclude with a discussion of our results, limitations and outline future directions.

References

1. IEEE standard letter designations for radar-frequency bands. IEEE Std 521--1984 (Nov 1984), 1--8.
2. Al-Nuaimy, W., Huang, Y., Nakhkash, M., Fang, M., Nguyen, V., and Eriksen, A. Automatic detection of buried utilities and solid objects with gpr using neural networks and pattern recognition. Journal of Applied Geophysics 43, 2 (2000), 157--165.
3. Bahl, P., and Padmanabhan, V. N. RADAR: an in-building rf-based user location and tracking system. In Proceedings IEEE INFOCOM 2000, The Conference on Computer Communications, Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies, Reaching the Promised Land of Communications, Tel Aviv, Israel, March 26-30, 2000 (2000), 775--784.
4. Beaudouin-Lafon, M. Instrumental interaction: An interaction model for designing post-wimp user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '00, ACM (New York, NY, USA, 2000), 446--453.
5. Buettner, M., Prasad, R., Philipose, M., and Wetherall, D. Recognizing daily activities with r'd-based sensors. In Proceedings of the 11th International Conference on Ubiquitous Computing, UbiComp '09, ACM (New York, NY, USA, 2009), 51--60.
6. El-Mahallawy, M. S., and Hashim, M. Material classi'cation of underground utilities from gpr images using dct-based svm approach. Geoscience and Remote Sensing Letters, IEEE 10, 6 (2013), 1542--1546.
7. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. The weka data mining software: An update. SIGKDD Explor. Newsl. 11, 1 (Nov. 2009), 10--18.
8. Harrison, C., and Hudson, S. E. Lightweight material detection for placement-aware mobile computing. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, UIST '08, ACM (New York, NY, USA, 2008), 279--282.
9. Harrison, C., Xiao, R., and Hudson, S. Acoustic barcodes: Passive, durable and inexpensive notched identi'cation tags. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST '12, ACM (New York, NY, USA, 2012), 563--568.
10. Harter, A., Hopper, A., Steggles, P., Ward, A., and Webster, P. The anatomy of a context-aware application. Wirel. Netw. 8, 2/3 (Mar. 2002), 187--197.
11. Ishii, H., and Ullmer, B. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, CHI '97, ACM (New York, NY, USA, 1997), 234--241.
12. Jol, H. M. Ground penetrating radar theory and applications. Elsevier, 2008.
13. Kemp, M. C. Millimetre wave and terahertz technology for detection of concealed threats - a review. In Infrared and Millimeter Waves, 2007 and the 2007 15th International Conference on Terahertz Electronics. IRMMW-THz. Joint 32nd International Conference on (Sept 2007), 647--648.
14. Khan, U. S., Al-Nuaimy, W., and El-Samie, F. E. A. Detection of landmines and underground utilities from acoustic and gpr images with a cepstral approach. Journal of Visual Communication and Image Representation 21, 7 (2010), 731--740.
15. Kharkovsky, S., and Zoughi, R. Microwave and millimeter wave nondestructive testing and evaluation overview and recent advances. IEEE Instrumentation Measurement Magazine 10, 2 (April 2007), 26--38.
16. Langen, B., Lober, G., and Herzig, W. Re'ection and transmission behaviour of building materials at 60 ghz. In Personal, Indoor and Mobile Radio Communications, 1994. Wireless Networks - Catching the Mobile Future., 5th IEEE International Symposium on (Sep 1994), 505--509 vol.2.
17. Laput, G., Yang, C., Xiao, R., Sample, A., and Harrison, C. Em-sense: Touch recognition of uninstrumented, electrical and electromechanical objects. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology, UIST '15, ACM (New York, NY, USA, 2015), 157--166.
18. Le Goc, M., Taylor, S., Izadi, S., and Keskin, C. A low-cost transparent electric 'eld sensor for 3d interaction on mobile devices. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems, CHI '14, ACM (New York, NY, USA, 2014), 3167--3170.
19. Lien, J., Gillian, N., Karagozler, M. E., Amihood, P., Schwesig, C., Olson, E., Raja, H., and Poupyrev, I. Soli: Ubiquitous gesture sensing with millimeter wave radar. ACM Trans. Graph. 35, 4 (July 2016), 142:1--142:19.
20. Maldonado, A., Alvarez, H., and Beetz, M. Improving robot manipulation through 'ngertip perception. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on (Oct 2012), 2947--2954.
21. Matthies, D. J. C., Perrault, S. T., Urban, B., and Zhao, S. Botential: Localizing on-body gestures by measuring electrical signatures on the human skin. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI '15, ACM (New York, NY, USA, 2015), 207--216.
22. Nelson, D. A., Nelson, M. T., Walters, T. J., and Mason, P. A. Skin heating effects of millimeter-wave irradiation-thermal modeling results. IEEE Transactions on Microwave Theory and Techniques 48, 11 (Nov 2000), 2111--2120.
23. Pasolli, E., Melgani, F., and Donelli, M. Automatic analysis of gpr images: A pattern-recognition approach. Geoscience and Remote Sensing, IEEE Transactions on 47, 7 (2009), 2206--2217.
24. Quigley, A., Dix, A., Mackay, W. E., Ishii, H., and Steimle, J. Visions and visioning in chi: Chi 2013 special interest group meeting. In CHI '13 Extended Abstracts on Human Factors in Computing Systems, CHI EA '13, ACM (New York, NY, USA, 2013), 2545--2548.
25. Rekimoto, J., and Ayatsuka, Y. Cybercode: Designing augmented reality environments with visual tags. In Proceedings of DARE 2000 on Designing Augmented Reality Environments, DARE '00, ACM (New York, NY, USA, 2000), 1--10.
26. Roodaki, P. M., Taghian, F., Bashirzadeh, S., and Jalaali, M. A survey of millimeter-wave technologies. In Electrical and Control Engineering (ICECE), 2011 International Conference on (Sept 2011), 5726--5728.
27. Sato, M., Yoshida, S., Olwal, A., Shi, B., Hiyama, A., Tanikawa, T., Hirose, M., and Raskar, R. Spectrans: Versatile material classi'cation for interaction with textureless, specular and transparent surfaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (New York, NY, USA, 2015), 2191--2200.
28. Sharan, L., Liu, C., Rosenholtz, R., and Adelson, E. H. Recognizing materials using perceptually inspired features. International Journal of Computer Vision 103, 3 (2013), 348--371.
29. Skolnik, M. I. Introduction to radar. Radar Handbook 2 (1962).
30. Project Soli. https://www.google.com/atap/project-soli/.
31. Song, J., Wang, S., Lien, J., Poupyrev, I., and Hilliges, O. Interacting with soli: Exploring 'ne-grained dynamic gesture recognition in the radio-frequency spectrum. In Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology, UIST '16, ACM (New York, NY, USA, 2016).
32. Sugama, Y., Murase, T., and Fujii, Y. Projection based virtual tablets system involving robust tracking of rectangular objects and hands. In Proceedings of the 7th Augmented Human International Conference 2016, AH '16, ACM (New York, NY, USA, 2016), 33:1--33:2.
33. Walabot. http://walabot.com/.
34. Weiser, M. The computer for the 21st century. SIGMOBILE Mob. Comput. Commun. Rev. 3, 3 (July 1999), 3--11.
35. Wiese, J., Saponas, T. S., and Brush, A. B. Phoneprioception: Enabling mobile phones to infer where they are kept. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, ACM (New York, NY, USA, 2013), 2157--2166.
36. Willis, K. D. D., and Wilson, A. D. Infrastructs: Fabricating information inside physical objects for imaging in the terahertz region. ACM Trans. Graph. 32, 4 (July 2013), 138:1--138:10.
37. Yilmaz, A., Javed, O., and Shah, M. Object tracking: A survey. ACM Comput. Surv. 38, 4 (Dec. 2006).
38. Zhu, Y., Zhu, Y., Zhao, B. Y., and Zheng, H. Reusing 60ghz radios for mobile radar imaging. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, MobiCom '15, ACM (New York, NY, USA, 2015), 103--116.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)