SensorBench: benchmarking approaches to processing wireless sensor network data

ABSTRACT

Wireless sensor networks enable cost-effective data collection for tasks such as precision agriculture and environment monitoring. However, the resource-constrained nature of sensor nodes, which often have both limited computational capabilities and battery lifetimes, means that applications that use them must make judicious use of these resources. Research that seeks to support data intensive sensor applications has explored a range of approaches and developed many different techniques, including bespoke algorithms for specific analyses and generic sensor network query processors. However, all such proposals sit within a multi-dimensional design space, where it can be difficult to understand the implications of specific decisions and to identify optimal solutions. This paper presents a benchmark that seeks to support the systematic analysis and comparison of different techniques and platforms, enabling both development and user communities to make well informed choices. The contributions of the paper include: (i) the identification of key variables and performance metrics; (ii) the specification of experiments that explore how different types of task perform under different metrics for the controlled variables; and (iii) an application of the benchmark to investigate the behavior of several representative platforms and techniques.

References

1. A. Alfadhly, U. Baroudi, and M. Younis. An effective approach for tolerating simultaneous failures in wireless sensor and actor networks. In MiSeNet, pages 21--26, New York, NY, USA, 2012. ACM.
2. P. Andreou, D. Zeinalipour-Yazti, A. Pamboris, P. K. Chrysanthis, and G. Samaras. Optimized query routing trees for wireless sensor networks. Inf. Syst., 36(2):267--291, 2011.
3. A. Arasu, S. Babu, and J. Widom. The CQL continuous query language: semantic foundations and query execution. VLDB J., 15(2):121--142, 2006.
4. A. Arasu, M. Cherniack, E. F. Galvez, D. Maier, A. Maskey, E. Ryvkina, M. Stonebraker, and R. Tibbetts. Linear road: A stream data management benchmark. In VLDB, pages 480--491, 2004.
5. C. Y. A. Brenninkmeijer, I. Galpin, A. A. A. Fernandes, and N. W. Paton. A semantics for a query language over sensors, streams and relations. In BNCOD, pages 87--99, 2008.
6. P. Corke, T. Wark, R. Jurdak, W. Hu, P. Valencia, and D. Moore. Environmental wireless sensor networks. Proc. of the IEEE, 98(11):1903--1917, 2010.
7. I. Galpin, C. Y. A. Brenninkmeijer, A. J. G. Gray, F. Jabeen, A. A. A. Fernandes, and N. W. Paton. SNEE: a query processor for wireless sensor networks. Distrib. Parallel Dat., 29(1-2):31--85, Nov. 2011.
8. I. Galpin, A. A. A. Fernandes, and N. W. Paton. QoS-aware optimization of sensor network queries. VLDB J., 22(4):495--517, 2013.
9. D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, and D. Culler. The nesC language: A holistic approach to networked embedded systems. SIGPLAN Not., 38(5):1--11, May 2003.
10. J. Gray. Database and transaction processing performance handbook. In The Benchmark Handbook. Morgan Kaurmann, 1993.
11. C. Guestrin, P. Bodik, R. Thibaux, M. Paskin, and S. Madden. Distributed regression: an efficient framework for modeling sensor network data. In IPSN, pages 1--10. IEEE, 2004.
12. C. Haas, J. Wilke, and V. Stöhr. Realistic simulation of energy consumption in wireless sensor networks. In EWSN, pages 82--97, 2012.
13. M. Hempstead, M. Welsh, and D. Brooks. TinyBench: The case for a standardized benchmark suite for TinyOS based wireless sensor network devices. In LCN, pages 585--586, 2004.
14. J. Hill, M. Horton, R. Kling, and L. Krishnamurthy. The platforms enabling wireless sensor networks. Commun. ACM, 47(6):41--46, 2004.
15. J. Hoebeke, I. Moerman, B. Dhoedt, and P. Demeester. An overview of mobile ad hoc networks: Applications and challenges. In European Telecommun. Congr., pages 60--66, November 2004.
16. P. Jackman, A. J. G. Gray, A. Brass, R. Stevens, M. Shi, D. Scuffell, S. Hammersley, and B. Grieve. Processing Online Crop Disease Warning Information via Sensor Networks using ISA Ontologies. Agriculture Engineering International: CIGR Journal, 15(3):243--251, 2013.
17. A. Jain and E. Y. Chang. Adaptive sampling for sensor networks. In DMSN, pages 10--16, New York, NY, USA, 2004. ACM.
18. D. Klan, M. Karnstedt, K. Hose, L. Ribe-Baumann, and K.-U. Sattler. Stream engines meet wireless sensor networks: cost-based planning and processing of complex queries in AnduIN. Distrib. Parallel Data., 29(1-2), 2011.
19. Y. W. Law, J. Doumen, and P. Hartel. Survey and benchmark of block ciphers for wireless sensor networks. TOSN, 2(1):65--93, 2006.
20. Á. Lédeczi, A. Nádas, P. Völgyesi, G. Balogh, B. Kusy, J. Sallai, G. Pap, S. Dóra, K. Molnár, M. Maróti, and G. Simon. Countersniper system for urban warfare. TOSN, 1(2):153--177, 2005.
21. P. Levis, S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer, et al. TinyOS: An operating system for sensor networks. In Ambient intelligence, pages 115--148. Springer, 2005.
22. M. Liu, S. R. Mihaylov, Z. Bao, M. Jacob, Z. G. Ives, B. T. Loo, and S. Guha. SmartCIS: integrating digital and physical environments. SIGMOD Rec., 39(1):48--53, Sept. 2010.
23. Q. Luo, H. Wu, W. Xue, and B. He. Benchmarking in-network sensor query processing. Technical Report HKUST-CS05-09, Department of Computer Science, HKUST, 2005.
24. S. Madden, M. J. Franklin, J. M. Hellerstein, and W. Hong. TinyDB: an acquisitional query processing system for sensor networks. ACM Trans. Database Syst., 30(1):122--173, 2005.
25. A. M. Mainwaring, D. E. Culler, J. Polastre, R. Szewczyk, and J. Anderson. Wireless sensor networks for habitat monitoring. In WSNA, pages 88--97, 2002.
26. I. W. Marshall, M. C. Price, H. Li, N. Boyd, and S. Boult. Multi-sensor cross correlation for alarm generation in a deployed sensor network. In EuroSSC, pages 286--299, 2007.
27. K. Martinez, J. K. Hart, and R. Ong. Environmental sensor networks. Computer, 37(8):50--56, 2004.
28. K. Martinez, R. Ong, and J. Hart. Glacsweb: a sensor network for hostile environments. IEEE Sensor and Ad Hoc Communications and Networks, 2004.
29. S. R. Mihaylov, M. Jacob, Z. G. Ives, and S. Guha. Dynamic join optimization in multi-hop wireless sensor networks. PVLDB, 3(1):1279--1290, 2010.
30. L. Nazhandali, M. Minuth, and T. Austin. SenseBench: toward an accurate evaluation of sensor network processors. In Proc. Int. Workload Characterization Symp., pages 197--203. IEEE, 2005.
31. L. Peng and K. S. Candan. Data-quality guided load shedding for expensive in-network data processing. In ICDE, pages 1325--1328, 2007.
32. U. Raza, A. Camerra, A. L. Murphy, T. Palpanas, and G. P. Picco. What does model-driven data acquisition really achieve in wireless sensor networks? In PerCom, pages 85--94, 2012.
33. V. Shnayder, M. Hempstead, B. rong Chen, G. Werner-Allen, and M. Welsh. Simulating the power consumption of large-scale sensor network applications. In SenSys, pages 188--200, 2004.
34. A. B. Stokes, A. A. A. Fernandes, and N. W. Paton. Resilient sensor network query processing using logical overlays. In MobiDE, pages 45--52, 2012.
35. S. Subramaniam, T. Palpanas, D. Papadopoulos, V. Kalogeraki, and D. Gunopulos. Online outlier detection in sensor data using non-parametric models. In VLDB, pages 187--198, 2006.
36. R. Szewczyk, A. M. Mainwaring, J. Polastre, J. Anderson, and D. E. Culler. An analysis of a large scale habitat monitoring application. In SenSys, pages 214--226, 2004.
37. D. Thain, T. Tannenbaum, and M. Livny. Distributed computing in practice: the Condor experience. Concurrency - Practice and Experience, 17(2-4):323--356, 2005.
38. B. Titzer, D. K. Lee, and J. Palsberg. Avrora: scalable sensor network simulation with precise timing. In IPSN, pages 477--482, 2005.
39. G. Valkanas, A. Kotsifakos, D. Gunopulos, I. Galpin, A. J. G. Gray, A. A. A. Fernandes, and N. W. Paton. Deploying in-network data analysis techniques in sensor networks. In MDM, pages 341--344, 2011.
40. K. Veress and M. Maroti. Linkbench: Benchmark and metric framework for wireless sensor networks. In IPSN, pages 171--172, 2011.
41. G. Werner-Allen, K. Lorincz, J. Johnson, J. Lees, and M. Welsh. Fidelity and yield in a volcano monitoring sensor network. In OSDI, pages 381--396, 2006.
42. G. Werner-Allen, P. Swieskowski, and M. Welsh. Motelab: A wireless sensor network testbed. In IPSN, page 68. IEEE Press, 2005.

Index Terms

• Applied computing

  • Physical sciences and engineering

    • Mathematics and statistics

• Hardware

  • Communication hardware, interfaces and storage

    • Signal processing systems

• Information systems

  • Information systems applications

    • Decision support systems

      • Data analytics

• Networks

  • Network types

    • Mobile networks

    • Wireless access networks