David S. Bunch
Abstract
We present FORTRAN 77 subroutines that solve statistical parameter estimation problems for general nonlinear models, e.g., nonlinear least-squares, maximum likelihood, maximum quasi-likelihood, generalized nonlinear least-squares, and some robust fitting problems. The accompanying test examples include members of the generalized linear model family, extensions using nonlinear predictors (“nonlinear GLIM”), and probabilistic choice models, such as linear-in-parameter multinomial probit models. The basic method, a generalization of the NL2SOL algorithm for nonlinear least-squares, employs a model/trust-region scheme for computing trial steps, exploits special structure by maintaining a secant approximation to the second-order part of the Hessian, and adaptively switches between a Gauss-Newton and an augmented Hessian approximation. Gauss-Newton steps are computed using a corrected seminormal equations approach. The subroutines include variants that handle simple bounds on the parameters, and that compute approximate regression diagnostics.
Supplemental Material
Available for Download
max- and quasi-likelihood estimation in nonlinear regression Gams: l8e1b2, l8e1b4
- 1 The PORT Mathematical Subroutine Library, 3rd ed. AT & T Bell Laboratories, Murray Hill, NJ, 1984.Google Scholar
- 2 BARD, Y. Nonlinear Parameter Estimatton. Academic Press, 1974.Google Scholar
- 3 BELSLEY, D. A., KUH, E., AND WELSCH, R.E. Regression Dtagnostics: Identifying Influential Data and Sources of Collineartty. Wiley, New York, 1980.Google Scholar
- 4 BJ~RCK, h. Stability analysis of the method of seminormal equations for least squares problems. Linear Algebra Appl. 88/89 (Apr. 1987), 31 48.Google Scholar
- 5 BUNCH, D.S. Maximum likelihood estimation of probabilistic choice models. SIAM J. Sci. Stat. Comput. 8, i (Jan. 1987), 56 70. Google Scholar
- 6 BUNCH, D. S. A comparison of algorithms for maximum likelihood estimation of choice models. J. Econometrtcs 38 (1988), 145-167.Google Scholar
- 7 BUNCH, D. S. A comparison of algorithms for maximum likelihood estimation of finite distribution models. Working Paper UCD-GSM-WP1590, Graduate School of Management, Univ. of California, Davis, CA, 1989.Google Scholar
- 8 BUNCH, D.S. MLMNP and MLMNPB: Fortran programs for maximum likelihood estimation of linear-in-parameter multinomial probit models. Rep. UCD-ITS-RR-91-14, Inst. of Transportation Studies, Univ. of California, Davis, CA, 1991.Google Scholar
- 9 BUNCH, D. S., AND KITAMU~, R. Multinomial probit estimation revisited: Testing estimable model specifications, maximum likelihood algorithms, and probit integral approximations for trinomial models of household car ownership. Rep. UCD-TRG-RR-4, Transportation Research Group, Univ. of California, Davis, CA, 1990.Google Scholar
- 10 CARROLL, R. J., AND RUPPERT, D. Transformation and Weighting in Regression. Chapman and Hall, New York, 1988. Google Scholar
- 11 CHATTE~EE, S., AND HAUl, A.S. Sensitivity Analysts in Linear Regresston. John Wiley and Sons, New York, 1988. Google Scholar
- 12 COOK, R. D., AND WEISBERG, S. Residuals and Influence in Regression. Chapman and Hall, London, 1982.Google Scholar
- 13 DAaANzo, C. Multtnomial Probit: The Theory and Its Application to Demand Forecasting. Academic Press, New York, 1979.Google Scholar
- 14 DAWDtAN, M., AND CARROLL, R.J. Variance function estimation. J. Am. Stat. Assoc. 82,400 (Dec. 1987), 1079-1091.Google Scholar
- 15 DENNIS, J. E., JR., GAY, D. M., AND WELSCH, R. E. An adaptive nonhnear least-squares algorithm. ACM Trans Math. Softw. 7, 3 (Sept. 1981), 348 368 Google Scholar
- 16 DENNIS, J. E., JR., GAY, D. M., AND WELSCH, R E. Algorithm 573. NL2SOL--An adaptive nonlinear least-squares algorithm. ACM Trans. Math. Softw. 7, 3 (Sept. 1981), 369 383. Google Scholar
- 17 DENNIS, J. E., JR., AND WALKER, H.F. Convergence theorems for least-change secant update methods SlAM J. Numer. Anal. 18, 6 (Dec 1981), 949-987.Google Scholar
- 18 DENNIS, J E., JR., AND WALKER, H. F. Erratum: Convergence theorems for least-change secant update methods. SIAM J. Numer. Anal. 19, 2 (Apr. 1982), 443.Google Scholar
- 19 Fox, P. A, HALL, A. D., AND SCnRYER, N.L. The PORT mathematical subroutine library. ACM Trans Moth. Softw. 4, 2 (June 1978), 104 126 Google Scholar
- 20 FROME, E.L. Regression methods for binomial and Poisson distributed data. Manuscript, Mathematics and Statistics Research Section, Oak Ridge Natmnal Laboratory, Oak Ridge, TN, 1984. Presented at AAPM Fzrst Mzdyear Topzcal Symposium on Multiple Regression Analysis: Applications in the Health Sciences (Mobfie, AL, Mar. 12 16, 1984).Google Scholar
- 21 GAY, D.M. Computing optimal locally constrained steps SIAM J. Scl. Stat. Comput. 2, 2 (June 1981), 186 197.Google Scholar
- 22 GAY, D. M. ALGORITHM 611 Subroutines for unconstrained minimization using a model/trust-region approach. ACM Trans. Math. Softw. 9, 4 (Dec. 1983), 503-524. Google Scholar
- 23 GAY, D.M. A trust-region approach to linearly constrained optimization. Numerical Analysis Proceedz~lgs (Dundee, 1983) D. F. Griffiths, Ed, Springer-Verlag, 72-105.Google Scholar
- 24 GAY, D. M. Usage summary for selected optimization routines. Computing Science Tech. Rep. No. 153 (1990), AT & T Bell Laboratories, Murray Hill, NJ. Postscript for this report is avafiable by e-maih ask netlib(~,research.att.com to send 153 from research/cstr.Google Scholar
- 25 GAY, D M., AND WELSCH, R. E. Maximum likelihood and quasi-likehhood for nonlinear exponential family regression models. J. Am. Stat Assoc. 83, 404 (Dec. 1988), 990 998.Google Scholar
- 26 HA~IPEL, F. R., RONCnETTI, E. M., ROUSSEEUW, P. J., AND STAHEL, W.A. Robust Statzst~cs. Wiley, 1986.Google Scholar
- 27 HOLLAND, P. W., AND WELSCH, R.E. Robust regression using ~teratively reweighted leastsquares. Commun. Stat. A6, 9 (1977), 813 827.Google Scholar
- 28 HUBER, P.J. Robust StatLstlcs. Wiley, New York, 1981.Google Scholar
- 29 McCULLAGH, P., AND NELDER, J.A. Generahzed Lznear Models. Chapman and Hall, 1983.Google Scholar
- 30 McCULLAGH, P.~ AND NELDER, J.A. Generalized Ltnear Models, 2nd ed. Chapman and Hall, 1989.Google Scholar
- 31 Mog~, J.J. The Levenberg-Marquardt algorithm: Implementation and theory. In Numerical Analysis, Dundee 1977, G. A. Watson, Ed., Sprmger-Verlag, Berlim 1978, 105-116.Google Scholar
- 32 MOR~, J. J., AND SORENSEN, D. C. Computing a trust region step. SIAM J. Sc~. Stat. Comput. 4, 3 (Sept. 1983), 553-572.Google Scholar
- 33 NELDER, J. A., AND PREGIBON, D. An extended quasi-likelihood function. Blornetr~ka 74, 2 (Aug. 1987), 221 232.Google Scholar
- 34 WALKER, H. F, AND GONGLEWSK~, J D. Quasi-Newton methods for maximum-hkelihood estimation. In Advances in Numerzcal Partial Dzfferent~al Equations and Optimization, S. Gdmez, J. P. Hennart and R. A. Tapia, Eds., SIAM, 1991, 332-345. In Proceedings of the Fifth Mexico-United States Workshop.Google Scholar
- 35 WEDDERBURN, R. W. M. Quasi-likelihood functions, generahzed linear models, and the Gauss-Newton method. Biometr~ka 61, 3 (Dec. 1974), 439 447.Google Scholar
Index Terms
- Algorithm 717: Subroutines for maximum likelihood and quasi-likelihood estimation of parameters in nonlinear regression models
Recommendations
Improve maximum likelihood estimation for subband GGD parameters
Statistical model of subband wavelet coefficients fitted with generalized Gaussian density (GGD) has been widely used into image retrieval, classification, segmentation, denoising, and analysis/synthesis. Moment estimation is very simple method for ...
Descent Directions of Quasi-Newton Methods for Symmetric Nonlinear Equations
In general, when a quasi-Newton method is applied to solve a system of nonlinear equations, the quasi-Newton direction is not necessarily a descent direction for the norm function. In this paper, we show that when applied to solve symmetric nonlinear ...
On maximum likelihood estimation in sparse contingency tables
Log-linear and logistic models can be fitted to data in contingency tables either by an iterative proportional fitting algorithm or by an iteratively reweighted Newton-Raphson algorithm. Both algorithms provide maximum likelihood (ML) estimates of the ...
Comments