Sequential Bayesian prediction in the presence of changepoints

ABSTRACT

We introduce a new sequential algorithm for making robust predictions in the presence of changepoints. Unlike previous approaches, which focus on the problem of detecting and locating changepoints, our algorithm focuses on the problem of making predictions even when such changes might be present. We introduce nonstationary covariance functions to be used in Gaussian process prediction that model such changes, then proceed to demonstrate how to effectively manage the hyperparameters associated with those covariance functions. By using Bayesian quadrature, we can integrate out the hyperparameters, allowing us to calculate the marginal predictive distribution. Furthermore, if desired, the posterior distribution over putative changepoint locations can be calculated as a natural byproduct of our prediction algorithm.

References

1. Adams, R. P., & MacKay, D. J. (2007). Bayesian online changepoint detection (Technical Report). University of Cambridge, Cambridge, UK. arXiv:0710.3742v1 {stat.ML}.
2. Basseville, M., & Nikiforov, I. (1993). Detection of abrupt changes: theory and application. Prentice Hall.
3. Brodsky, B., & Darkhovsky, B. (1993). Nonparametric Methods in Change-Point Problems. Springer.
4. Carlin, B. P., Gelfand, A. E., & Smith, A. F. M. (1992). Hierarchical Bayesian analysis of change-point problems. Applied statistics, 41, 389--405.
5. Chen, J., & Gupta, A. (2000). Parametric Statistical Change Point Analysis. Birkhááuser Verlag.
6. Chernoff, H., & Zacks, S. (1964). Estimating the Current Mean of a Normally Distributed Variable Which is Subject to Changes in Time. Annals of Mathematical Statistics, 35, 999--1028.
7. Csorgo, M., & Horvath, L. (1997). Limit theorems in change-point analysis. John Wiley & Sons.
8. Fearnhead, P., & Liu, Z. (2007). On-line inference for multiple changepoint problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69, 589--605.
9. Horváth, L., & Kokoszka, P. (1997). The effect of long-range dependence on change-point estimators. Journal of Statistical Planning and Inference, 64, 57--81.
10. Muller, H. (1992). Change-points in nonparametric regression analysis. Ann. Statist, 20, 737--761.
11. Osborne, M. A., Rogers, A., Ramchurn, S., Roberts, S. J., & Jennings, N. R. (2008). Towards real-time information processing of sensor network data using computationally efficient multi-output Gaussian processes. International Conference on Information Processing in Sensor Networks 2008 (pp. 109--120).
12. Rasmussen, C. E., & Ghahramani, Z. (2003). Bayesian Monte Carlo. In S. Becker and K. Obermayer (Eds.), Advances in neural information processing systems, vol. 15. Cambridge, MA: MIT Press.
13. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press.
14. Ray, B., & Tsay, R. (2002). Bayesian methods for change-point detection in long-range dependent processes. Journal of Time Series Analysis, 23, 687--705.
15. Roberts, S. J. (2000). Extreme value statistics for novelty detection in biomedical data processing. Science, Measurement and Technology, IEE Proceedings- (pp. 363--367).
16. Whitcher, B., Byers, S., Guttorp, P., & Percival, D. (2002). Testing for homogeneity of variance in time series: Long memory, wavelets and the Nile River. Water Resources Research, 38, 10--1029.

Index Terms

• Computing methodologies

  • Machine learning

  • Modeling and simulation

    • Model development and analysis

      • Model verification and validation

      • Modeling methodologies

• Mathematics of computing

  • Probability and statistics

    • Probabilistic algorithms

    • Probabilistic reasoning algorithms

      • Markov-chain Monte Carlo methods

      • Sequential Monte Carlo methods