Sampling Simulation Model Profile Data for Analysis

ABSTRACT

The capture of data about the events executed by a discrete event simulation can easily lead to very large trace data files. While disk space is relatively inexpensive and mostly capable of storing these large trace files, the manipulation and analysis of these large trace files can prove difficult. Furthermore, some types of analysis must be performed in-core and they cannot be performed with the trace data exceeds the size of the physical RAM where the analysis is performed. Because of these limits, it is often necessary to strictly limit the simulation run time to satisfy the analysis time memory limits. Experience with the DESMetrics tool suite (a collection of tools to analyze event trace files), demonstrates that our in-memory analysis tools are limited to trace files on the order of 10GB (on a machine with 24GB of RAM). Furthermore, even when it is possible to analyze large trace files, the run time costs of performing this analysis can take several days to complete. While high performance analysis of traces data is not strictly necessary, the results should be available within some reasonably bounded time frame. This paper explores techniques to overcome the limits of analyzing very large event trace files. While explorations for out-out-core analysis have been examined as part of this work, the run time costs for out-of-core processing can increase processing time 10-fold. As a result, the work reported here will focus on an approach to capture and analyze small samples from the event trace file. The work reported in this paper will examine how closely the analysis from sampling matches the analysis from a full trace file. Two techniques for comparison are presented. First a visual comparison of analysis results between the full trace and a trace sample are presented. Second, numerical quantification of the different analysis results (between the full trace and trace sample) will be reported using the Wasserstein, Directed Hausdorff, and Kolmogorov-Smirnov distance metrics. Finally, the ability to process trace samples from a very large trace file of 80GB is demonstrated.

References

1. A. J. Alt and P. A. Wilsey. 2014. Profile Driven Partitioning of Parallel Simulation Models Proceedings of the 2014 Winter Simulation Conference, A. Tolk, S. Y. Diallo, I. O. Ryzhov, L. Yilmaz, S. Buckley, and J. A. Miller (Eds.). Institute of Electrical and Electronics Engineers, Inc., Piscataway, New Jersey, USA, 2750--2761.
2. V. Balakrishnan, R. Radhakrishnan, D. M. Rao, N. B. Abu-Ghazaleh, and P. A. Wilsey. 2001. A Performance and Scalability Analysis Framework for Parallel Discrete Event Simulators. Simulation Practice and Theory Vol. 8 (2001), 529--553.
3. Christopher L. Barrett, Keith R. Bisset, Stephen G. Eubank, Xizhou Feng, and Madhav V. Marathe. 2008. EpiSimdemics: An Efficient Algorithm for Simulating the Spread of Infectious Disease over Large Realistic Social Networks. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing (SC '08). IEEE Press, Piscataway, New Jersey, USA.
4. O. Berry and D. Jefferson. 1985. Critical Path Analysis of Distributed Simulation. In Distributed Simulation. SCS, SCS, San Diego, CA, 57--60.
5. V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre. 2008. Fast Unfolding of Communities in Large Networks. Journal of Statistical Mechanics: Theory and Experiment Vol. 10 (2008), 10008.
6. Christopher D. Carothers, David Bauer, and Shawn Pearce. 2000. ROSS: A High-performance, Low Memory, Modular Time Warp System Proceedings of the Fourteenth Workshop on Parallel and Distributed Simulation (PADS '00). IEEE Computer Society, Washington, DC, USA, 53--60. http://dl.acm.org/citation.cfm?id=336146.336157
7. Patrick Crawford, Stephan J. Eidenbenz, Peter D. Barnes Jr., and Philip A. Wilsey. 2017. Some Properties of Communication Behaviors in Discrete-Event Simulation Models 2017 Winter Simulation Conference (WSC). 1025--1036.
8. A Ferscha and J Johnson. 1996. A Testbed for Parallel Simulation Performance Predictions 1996 Winter Simulation Conference Proceedings, J. M. Charnes andd D. J. Morrice, D. T. Brunner, and J. J. Swain (Eds.). Institute of Electrical and Electronics Engineers, Inc., Piscataway, New Jersey, USA, 637--644.
9. Jean-Baptiste Filippi, Teruhisa Komatsu, and Kyushu Tanaka. 2010. Simulation of drifting seaweeds in East China Sea. Ecological Informatics Vol. 5, 1 (2010), 67--72.
10. R. Fujimoto. 1990. Performance of Time Warp under Synthetic Workloads Proceedings of the SCS Multiconference on Distributed Simulation, David Nicol (Ed.), Vol. Vol. 22. SCS, San Diego, CA, 23--28.
11. R. García-Martínez and H. Flores-Tovar. 1999. Computer modeling of oil spill trajectories with a high accuracy method. Spill Science and Technology Bulletin Vol. 5, 5--6 (1999), 323--330.
12. Sounak Gupta and Philip A. Wilsey. 2017. Quantitative Driven Optimization of a Time Warp Kernel Proceedings of the 2017 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (PADS 17). ACM, New York, NY, USA.
13. John L. Hennessy and David A. Patterson. 2012. Computer Architecture: A Quantitative Approach (5th ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
14. D. Jefferson and P. L. Reiher. 1991. Supercritical Speedup. In Proceedings of the 24$^th$ Annual Simulation Symposium, A. H. Rutan (Ed.). IEEE Computer Society Press, 159--168.
15. Y-B. Lin. 1992. Parallelism Analyzer for Parallel Discrete Event Simulation. ACM Transactions on Modeling and Computer Simulation Vol. 2, 3 (July. 1992), 239--264.
16. M. Livny. 1985. A Study of Parallelism in Distributed Simulation. In Proceedings 1985 SCS Multiconference on Distributed Simulation. SCS, San Diego, CA, 94--98.
17. T. Neudecker, P. Andelfinger, and H. Hartenstein. 2015. A simulation model for analysis of attacks on the Bitcoin peer-to-peer network 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM). 1327--1332.
18. Eun Jung Park, Stephan Eidenbenz, Nandakishore Santhi, Guillaume Chapuis, and Bradley Settlemyer. 2015. Parameterized Benchmarking of Parallel Discrete Event Simulation Systems: Communication, Computation, and Memory. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, I. Moon, T. M. K. Roeder, C. Macal, and M. D. Rossetti (Eds.). Institute of Electrical and Electronics Engineers, Inc., Piscataway, New Jersey, USA, 2836--2847.
19. Mark Plagge, Christopher D. Carothers, and Elsa Gonsiorowski. 2016. NeMo: A Massively Parallel Discrete-Event Simulation Model for Neuromorphic Architectures. In Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (SIGSIM-PADS '16). ACM, New York, NY, USA, 233--244.
20. Doug Weber. 2016. Time Warp Simulation on Multi-core Processors and Clusters. Master's thesis. University of Cincinnati, Cincinnati, OH.
21. RO. Weber. 1991. Toward a comprehensive wildfire spread model. International Journal of Wildland Fire Vol. 1 (1991), 245--253.
22. P. A. Wilsey. 2016. Some Properties of Events Executed in Discrete-Event Simulation Models Workshop on Parallel and Distributed Simulation (PADS 16). ACM, New York, NY, USA.

Index Terms

• Computing methodologies

  • Modeling and simulation

    • Simulation support systems

      • Simulation tools

    • Simulation types and techniques

      • Discrete-event simulation