Efficient maximum likelihood estimation of linear models with APL

ABSTRACT

Maximum likelihood estimators used in statistics and econometrics have desirable properties, however, due to the complexity of their solution maximum likelihood techniques are not widely used. This paper examines an APL implementation of one of the most efficient algorithms used to estimate the parameters of the univariate ARMA process. The straight-forward use of APL is found to be unacceptable and so a systematic search for optimizations is made. This search results in an approximate solution for the ML estimator, the use of □NA and FORTRAN, and special matrix techniques to increase the efficiency of the algorithm. The matrix techniques used are implementations of sparse, banded, and block diagonal data structures. Additional matrix techniques involve incremental updating of matrices. The effect of these optimizations is to bring the computational cost from an O(N*3) problem to an O(N*1) problem. This translates into a significant reduction in computer requirements. For example, to estimate a three parameter ARMA model with 5000 observations, memory requirements fall from approximately 95 megabytes to 300 kilobytes. Computer time falls from about 6.4 years to under 1 minute.

REFERENCES

• 1: George Edward Pelham Box , Gwilym Jenkins, Time Series Analysis, Forecasting and Control, Holden-Day, Incorporated, 1990
• 2: Koreislaa, Sergio and Pukkila, Tarmo, Fast Linear Estimation Methods for Vector Autoregressive Moving-Average Models, Journal of Time Series Analysis,(1989),10,325-340.
• 3: Koreisha, Sergio and lhakkfla, Tarmo, A Generalized Least- Squares Approach for Estimation of Autoregressive Moving- Average Models, J. Time Ser. Ana/.,(1990),l 1,139-152.
• 4: Reirtsel, G.C., Basu, S., and Yap, S.F., Maximum Likelihood Estimators in the Multivariate Autoregressive Moving Average Model from a Generalized Least Squares Viewpoint, J. Time Set. Ana/.,(1992),13, 133-146.
• 5: Ansley, Craig F., An algorithm for the exact likelihood of a mixed autoregressive-moving average process, Biometrica (1979),66,1 pp. 59-65.
• 6: Grenander, U., MATHEMATICAL EXPERIMENTS ON THE COMPUTER, Academic Press (1982).

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: APL

Additional Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: FORTRAN

  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

      Nouns: ARMA

General Terms:
Algorithms, Languages