Green Simulation: Reusing the Output of Repeated Experiments

Abstract

We introduce a new paradigm in simulation experiment design and analysis, called “green simulation,” for the setting in which experiments are performed repeatedly with the same simulation model. Green simulation means reusing outputs from previous experiments to answer the question currently being asked of the simulation model. As one method for green simulation, we propose estimators that reuse outputs from previous experiments by weighting them with likelihood ratios, when parameters of distributions in the simulation model differ across experiments. We analyze convergence of these estimators as more experiments are repeated, while a stochastic process changes the parameters used in each experiment. As another method for green simulation, we propose an estimator based on stochastic kriging. We find that green simulation can reduce mean squared error by more than an order of magnitude in examples involving catastrophe bond pricing and credit risk evaluation.

References

1. Bruce Ankenman, Barry L. Nelson, and Jeremy Staum. 2010. Stochastic kriging for simulation metamodeling. Operat. Res. 58, 2 (2010), 371--382.
2. Russell R. Barton, Barry L. Nelson, and Wei Xie. 2014. Quantifying input uncertainty via simulation confidence intervals. INFORMS J. Comput. 26, 1 (2014), 74--87.
3. Richard J. Beckman and Michael D. McKay. 1987. Monte Carlo estimation under different distributions using the same simulation. Technometrics 29, 2 (1987), 153--160.
4. George D. Birkhoff. 1931. Proof of the ergodic theorem. In Proceedings of the National Academy of Sciences, Vol. 17. National Academy of Sciences, 656--660.
5. Eric S. Blake and Ethan J. Gibney. 2011. The Deadliest, Costliest, and Most Intense United States Tropical Cyclones from 1851 to 2010 (and Other Frequently Requested Hurricane Facts). Retrieved July 25, 2017 from http://www.nhc.noaa.gov/dcmi.shtml.
6. Joshua D. Coval, Jakub W. Jurek, and Erik Stafford. 2009. Economic catastrophe bonds. Amer. Econ. Rev. 99, 3 (June 2009), 628--666.
7. Angelos Dassios and Ji-Wook Jang. 2003. Pricing of catastrophe reinsurance and derivatives using the Cox process with shot noise intensity. Fin. Stochast. 7, 1 (January 2003), 73--95.
8. Mingbin Feng and Jeremy Staum. 2015. Green simulation designs for repeated experiments. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, I. Moon, T. M. K. Roeder, C. Macal, and M. D. Rossetti (Eds.). IEEE Press, Piscataway, NJ, 403--413.
9. Michael Fu and others. 2015. Handbook of Simulation Optimization. Vol. 216. Springer, New York.
10. Paul Glasserman and Xingbo Xu. 2014. Robust risk measurement and model risk. Quant. Fin. 14, 1 (2014), 29--58.
11. Tim Hesterberg. 1988. Advances in Importance Sampling. Ph.D. Dissertation. Stanford University.
12. Tim Hesterberg. 1995. Weighted average importance sampling and defensive mixture distributions. Technometrics 37, 2 (1995), 185--194.
13. Jack P. C. Kleijnen and Reuven Y. Rubinstein. 1996. Optimization and sensitivity analysis of computer simulation models by the score function method. Eur. J. Operat. Res. 88 (1996), 413--427.
14. Stuart A. Klugman, Harry H. Panjer, and Gordon E. Willmot. 2012. Loss Models: From Data to Decisions (4 ed.). John Wiley 8 Sons.
15. Andrea Laforgia. 1991. Bounds for modified Bessel functions. J. Comput. Appl. Math. 34, 3 (1991), 263--267.
16. Pierre L’Ecuyer. 1990. A unified view of the IPA, SF, and LR gradient estimation techniques. Manage. Sci. 36, 11 (1990), 1364--1383.
17. Pierre L’Ecuyer. 1993. Two approaches for estimating the gradient in functional form. In Proceedings of the 25th Conference on Winter Simulation. ACM, New York, 338--346.
18. Yudell L. Luke. 1972. Inequalities for generalized hypergeometric functions. J. Approx. Theory 5, 1 (1972), 41--65.
19. Alvaro Maggiar, Andreas Wächter, Irina S. Dolinskaya, and Jeremy Staum. 2015. A Derivative-Free Algorithm for the Optimization of Functions Smoothed via Gaussian Convolution Using Multiple Importance Sampling. Retrieved May 27, 2017 from http://www.optimization-online.org/DB_HTML/2015/07/5017.html.
20. Luca Martino, Victor Elvira, David Luengo, and Jukka Corander. 2015. An adaptive population importance sampler: Learning from uncertainty. IEEE Trans. Signal Process. 63, 16 (2015), 4422--4437.
21. Alexander J. McNeil, Rüdiger Frey, and Paul Embrechts. 2005. Quantitative Risk Management: Concepts, Techniques, and Tools. Princeton University Press, Princeton, NJ.
22. Robert C. Merton. 1974. On the pricing of corporate debt: The risk structure of interest rates. J. Fin. 29, 2 (1974), 449--470.
23. Sean Meyn and Richard L. Tweedie. 2009. Markov Chains and Stochastic Stability (2 ed.). Cambridge University Press, New York, NY.
24. Munich Re Geo Risks Research. 2015. Loss Events Worldwide 1980--2014, 10 Costliest Events Ordered by Overall Losses. Technical Report. Munich Re.
25. Esa Nummelin. 2004. General Irreducible Markov Chains and Non-Negative Operators. Cambridge Tracts in Mathematics, Vol. 83. Cambridge University Press.
26. Art B. Owen and Yi Zhou. 2000. Safe and effective importance sampling. J. Amer. Statist. Assoc. 95, 449 (2000), 135--143.
27. Reuven Y. Rubinstein and Alexander Shapiro. 1993. Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method. John Wiley 8 Sons, New Jersey.
28. Peter Salemi, Jeremy Staum, and Barry L. Nelson. 2013. Generalized integrated Brownian fields for simulation metamodeling. In Proceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World, R. Pasupathy, S.-H. Kim, R. Hill, A. Tolk, and M. E. Kuhl (Eds.). IEEE Press, 543--554.
29. Jeremy Staum. 2009. Better simulation metamodeling: The why, what, and how of stochastic kriging. In Proceedings of the 2015 Winter Simulation Conference, M. D. Rossetti, B. Johansson R. R. Hill, A. Dunkin, and R. G. Ingalls (Eds.). IEEE Press, Piscataway, NJ, 119--133.
30. Eric Veach. 1997. Robust Monte Carlo Methods for Light Transport Simulation. Ph.D. Dissertation. Stanford University.
31. Eric Veach and Leonidas J. Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95). ACM, New York, NY, 419--428.
32. Wei Xie, Barry L. Nelson, and Russell R. Barton. 2014. A Bayesian framework for quantifying uncertainty in stochastic simulation. Operat. Res. 62, 6 (2014), 1439--1452.

Index Terms

• Computing methodologies

  • Modeling and simulation

    • Model development and analysis

    • Simulation types and techniques