skip to main content
10.5555/1218112.1218154acmconferencesArticle/Chapter ViewAbstractPublication PageswscConference Proceedingsconference-collections
Article

Performance evaluation of spectral procedures for simulation analysis

Published: 03 December 2006 Publication History

Abstract

We summarize an experimental performance evaluation of WASSP and the Heidelberger-Welch (HW) algorithm, two sequential spectral procedures for steady-state simulation analysis. Both procedures approximate the log-smoothed-periodogram of the batch means after suitable data-truncation to eliminate the effects of initialization bias, finally delivering a confidence-interval estimator for the mean response that satisfies user-specified half-length and coverage-probability requirements. HW uses a Cramér-von Mises test for initialization bias based on the method of standardized time series; and then HW fits a quadratic polynomial to the batch-means log-spectrum. In contrast WASSP uses the von Neumann randomness test and the Shapiro-Wilk normality test to obtain an approximately stationary Gaussian batch-means process whose log-spectrum is approximated via wavelets. Moreover, unlike HW, WASSP estimates the final sample size required to satisfy the user's confidence-interval requirements. Regarding closeness of conformance to both confidence-interval requirements, we found that WASSP outperformed HW in the given test problems.

References

[1]
Anderson, T. W., and D. A. Darling. 1952. Asymptotic theory of certain "goodness of fit" criteria based on stochastic processes. The Annals of Mathematical Statistics 23 (2): 193--212.
[2]
Heidelberger, P., and P. D. Welch. 1981a. A spectral method for confidence interval generation and run length control in simulations. Communications of the ACM 24 (4): 233--245.
[3]
Heidelberger, P., and P. D. Welch. 1981b. Adaptive spectral methods for simulation output analysis. IBM Journal of Research and Development 25 (6): 860--876.
[4]
Heidelberger, P., and P. D. Welch. 1983. Simulation run length control in the presence of an initial transient. Operations Research 31 (6): 1109--1144.
[5]
Lada, E. K. 2003. A wavelet-based procedure for steady-state simulation output analysis. Doctoral dissertation, Graduate Program in Operations Research, North Carolina State University, Raleigh, North Carolina. Available online via <www.lib.ncsu.edu/theses/available/etd-04032003-141616/unrestricted/etd.pdf> {accessed July 15, 2006}.
[6]
Lada, E. K., and J. R. Wilson. 2006. A wavelet-based spectral procedure for steady-state simulation analysis. European Journal of Operational Research 174:1769--1801.
[7]
Lada, E. K., J. R. Wilson, and N. M. Steiger. 2003. A wavelet-based spectral method for steady-state simulation analysis. In Proceedings of the 2003 Winter Simulation Conference, ed. S. Chick, P. J. Sánchez, D. Ferrin, and D. J. Morrice, 422--430. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <www.informs-sim.org/wsc03papers/052.pdf> {accessed July 15, 2006}.
[8]
Lada, E. K., J. R. Wilson, N. M. Steiger, and J. A. Joines. 2004a. User's manual for WASSP Version 1 {online}. Edward P. Fitts Department of Industrial and Systems Engineering, NC State University, Raleigh, NC. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/wasspman.pdf> {accessed July 15, 2006}.
[9]
Lada, E. K., J. R. Wilson, N. M. Steiger, and J. A. Joines. 2004b. Performance evaluation of a wavelet-based spectral method for steady-state simulation analysis. In Proceedings of the 2004 Winter Simulation Conference, ed. R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, 694--702. Available online via <www.informs-sim.org/wsc04papers/084.pdf> {accessed July 15, 2006}.
[10]
Lada, E. K., J. R. Wilson, N. M. Steiger, and J. A. Joines. 2005. Performance of a wavelet-based spectral procedure for steady-state simulation analysis. INFORMS Journal on Computing to appear. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/lada05joc.pdf> {accessed July 15, 2006}.
[11]
Lada, E. K., N. M. Steiger, and J. R. Wilson. 2006. Performance evaluation of recent procedures for steady-state simulation analysis. IIE Transactions 38 (9): 711--727. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/lada06iie.pdf> {accessed July 15, 2006}.
[12]
Law, A. M., and J. S. Carson. 1979. A sequential procedure for determining the length of a steady-state simulation. Operations Research 27 (5): 1011--1025.
[13]
Shapiro, S. S., and M. B. Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52 (3--4): 591--611.
[14]
Steiger, N. M., E. K. Lada, J. R. Wilson, J. A. Joines, C. Alexopoulos, and D. Goldsman. 2005. ASAP3: A batch means procedure for steady-state simulation output analysis. ACM Transactions on Modeling and Computer Simulation 15 (1): 39--73. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/steiger05tomacs.pdf> {accessed July 16, 2006}.
[15]
von Neumann, J. 1941. Distribution of the ratio of the mean square successive difference to the variance. The Annals of Mathematical Statistics 12 (4): 367--395.
  1. Performance evaluation of spectral procedures for simulation analysis

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WSC '06: Proceedings of the 38th conference on Winter simulation
      December 2006
      2429 pages
      ISBN:1424405017

      Sponsors

      • IIE: Institute of Industrial Engineers
      • ASA: American Statistical Association
      • IEICE ESS: Institute of Electronics, Information and Communication Engineers, Engineering Sciences Society
      • IEEE-CS\DATC: The IEEE Computer Society
      • SIGSIM: ACM Special Interest Group on Simulation and Modeling
      • NIST: National Institute of Standards and Technology
      • (SCS): The Society for Modeling and Simulation International
      • INFORMS-CS: Institute for Operations Research and the Management Sciences-College on Simulation

      Publisher

      Winter Simulation Conference

      Publication History

      Published: 03 December 2006

      Check for updates

      Qualifiers

      • Article

      Conference

      WSC06
      Sponsor:
      • IIE
      • ASA
      • IEICE ESS
      • IEEE-CS\DATC
      • SIGSIM
      • NIST
      • (SCS)
      • INFORMS-CS
      WSC06: Winter Simulation Conference 2006
      December 3 - 6, 2006
      California, Monterey

      Acceptance Rates

      WSC '06 Paper Acceptance Rate 177 of 252 submissions, 70%;
      Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 96
        Total Downloads
      • Downloads (Last 12 months)0
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 07 Mar 2025

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media