LIPED: HMM-based life profiles for adaptive event detection

ABSTRACT

In this paper, the proposed LIPED (LIfe Profile based Event Detection) employs the concept of life profiles to predict the activeness of event for effective event detection. A group of events with similar activeness patterns shares a life profile, modeled by a hidden Markov model. Considering the burst-and-diverse property of events, LIPED identifies the activeness status of event. As a result, LIPED balances the clustering precision and recall to achieve better F1 scores than other well known approaches evaluated on the official TDT1 corpus.

References

1. Aggarwal, C. C. A Framework for Diagnosing Changes in Evolving Data Streams. In Proceedings of ACM SIGMOD, 2003, 575--586.
2. Aizen, J., Huttenlocher, D., Kleinberg, J., and Novak, A. Traffic-Based Feedback on the Web. In Proceedings of the National Academy of Sciences: 101, 2004, 5254--5260.
3. Allan, J., Papka, R., and Lavrenko, V. On-Line New Event Detection and Tracking. In Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval, 1998, 37--45.
4. Allan, J., Carbonell, J., Doddington, G., Yamron, J., and Yang, Y. Topic Detection and Tracking Pilot Study: Final Report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1998, 194--218.
5. Baum, L. E., Petrie, T., Soules, G., and Weiss, N. A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains. In Annals of Mathematical Statistics: 41, 1970, 164--171.
6. Dempster, A. P., Laird, N. M., and Rubin, D. B. Maximum likelihood from incomplete data via the EM algorithm. In Journal of the Royal Statistical Society. Series B 39, 1977, 1--38.
7. Kleinberg, J. Bursty and Hierarchical Structure in Streams. In Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining, 2002, 91--101.
8. Ghahramani, S. Fundamentals of Probability. Prentice Hall, 2000.
9. Markov, A. A. An example of statistical investigation in the text of 'Eugene Onyegin' illustrating coupling of 'tests' in chains. In Proceedings of the Academy of Sciences 7, 1913, 153--162.
10. Mitchell, T. M. Machine Learning. McGraw-Hall, 1997.
11. Myers, C., Rabiner, L. R., and Rosenberg, A. E. Performance Tradeoffs in Dynamic Time Warping Algorithms for Isolated Word Recognition. In IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP-28, No. 6, Dec, 1980, 623--635.
12. Rabiner, L. R. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. In Proceedings of the IEEE, 77(2), 1989, 257--286.
13. Salton, G. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, 1989.
14. Viterbi, A. J. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. In IEEE Transactions on Information Theory IT-13, 1967, 1260--1269.
15. Yang, Y., Pierce, T., and Carbonell, J. A Study on Retrospective and On-Line Event Detection. In Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval, 1998, 28--36.

Index Terms

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Clustering and classification

  • Information systems applications

    • Data mining

      • Clustering

• Mathematics of computing

  • Probability and statistics

    • Statistical paradigms

      • Time series analysis