Event threading within news topics

ABSTRACT

With the overwhelming volume of online news available today, there is an increasing need for automatic techniques to analyze and present news to the user in a meaningful and efficient manner. Previous research focused only on organizing news stories by their topics into a flat hierarchy. We believe viewing a news topic as a flat collection of stories is too restrictive and inefficient for a user to understand the topic quickly.

In this work, we attempt to capture the rich structure of events and their dependencies in a news topic through our event models. We call the process of recognizing events and their dependencies <i>event threading</i>. We believe our perspective of modeling the structure of a topic is more effective in capturing its semantics than a flat list of on-topic stories.

We formally define the novel problem, suggest evaluation metrics and present a few techniques for solving the problem. Besides the standard word based features, our approaches take into account novel features such as temporal locality of stories for event recognition and time-ordering for capturing dependencies. Our experiments on a manually labeled data sets show that our models effectively identify the events and capture dependencies among them.

REFERENCES

• 1: J. Allan, J. Carbonell, G. Doddington, J. Yamron, and Y. Yang. Topic detection and tracking pilot study: Final report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, pages 194--218, 1998.
• 2: James Allan , Ao Feng , Alvaro Bolivar, Flexible intrinsic evaluation of hierarchical clustering for TDT, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956914]
• 3: James Allan, Topic detection and tracking: event-based information organization, Kluwer Academic Publishers, Norwell, MA, 2002
• 4: James Allan , Rahul Gupta , Vikas Khandelwal, Temporal summaries of new topics, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.10-18, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383954]
• 5: Regina Barzilay and Lillian Lee. Catching the drift: Probabilistic content models, with applications to generation and summarization. In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics(HLT-NAACL), pages 113--120, 2004.
• 6: D. Lawrie and W. B. Croft. Discovering and comparing topic hierarchies. In Proceedings of RIAO 2000 Conference, pages 314--330, 1999.
• 7: David D. Lewis , Kimberly A. Knowles, Threading electronic mail: a preliminary study, Information Processing and Management: an International Journal, v.33 n.2, p.209-217, March 1997 [doi>10.1016/S0306-4573(96)00063-5 ]
• 8: Juha Makkonen, Investigations on event evolution in TDT, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Proceedings of the HLT-NAACL 2003 student research workshop, p.43-48, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073416.1073424]
• 9: Aixin Sun , Ee-Peng Lim, Hierarchical Text Classification and Evaluation, Proceedings of the 2001 IEEE International Conference on Data Mining, p.521-528, November 29-December 02, 2001
• 10: Yiming Yang , Jaime G. Carbonell , Ralf D. Brown , Thomas Pierce , Brian T. Archibald , Xin Liu, Learning Approaches for Detecting and Tracking News Events, IEEE Intelligent Systems, v.14 n.4, p.32-43, July 1999 [doi>10.1109/5254.784083 ]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
clustering, dependency, event, threading