skip to main content
10.1145/1148170.1148180acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
Article

Thread detection in dynamic text message streams

Published: 06 August 2006 Publication History

Abstract

Text message stream is a newly emerging type of Web data which is produced in enormous quantities with the popularity of Instant Messaging and Internet Relay Chat. It is beneficial for detecting the threads contained in the text stream for various applications, including information retrieval, expert recognition and even crime prevention. Despite its importance, not much research has been conducted so far on this problem due to the characteristics of the data in which the messages are usually very short and incomplete. In this paper, we present a stringent definition of the thread detection task and our preliminary solution to it. We propose three variations of a single-pass clustering algorithm for exploiting the temporal information in the streams. An algorithm based on linguistic features is also put forward to exploit the discourse structure information. We conducted several experiments to compare our approaches with some existing algorithms on a real dataset. The results show that all three variations of the single-pass algorithm outperform the basic single-pass algorithm. Our proposed algorithm based on linguistic features improves the performance relatively by 69.5% and 9.7% when compared with the basic single-pass algorithm and the best variation algorithm in terms of F1 respectively.

References

[1]
J. Allan, J. Carbonell, G. Doddington, J. Yamron, and Y. Yang. Topic detection and tracking pilot study. In Proceedings of DARPA Broadcast News Transcription and Understanding Workshop, pages 194--218, 1998.
[2]
J. Bengel, S. Gauch, E. Mittur, and R. Vijayaraghavan. Chattrack: Chat room topic detection using classification. In 2nd Symposium on Intelligence and Security Informatics (ISI-2004)., page 266-277, Tucson, Arizona., June 2004.
[3]
E. Bingham, A. Kabán, and M. Girolami. Topic identification in dynamical text by complexity pursuit. Neural Process. Lett., 17(1):69--83, 2003.
[4]
E. Elnahrawy. Log-based chat room monitoring using text categorization: A comparative study. In St.Thomas, editor, Proceedings of the IASTED International Conference on Information and Knowledge Sharing (IKS 2002), US Virgin Islands, USA, November 2002.
[5]
V. Hatzivassiloglou, L. Gravano, and A. Maganti. An investigation of linguistic features and clustering algorithms for topical document clustering. In SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 224--231, 2000.
[6]
http://www3.gartner.com/3_consulting_services/marketplace/instMessaging.jsp
[7]
X. Ji and H. Zha. Domain-independent text segmentation using anisotropic diffusion and dynamic programming. In SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 322--329, 2003.
[8]
A. Kabán and M. A. Girolami. A dynamic probabilistic model to visualise topic evolution in text streams. J. Intell. Inf. Syst., 18(2-3):107--125, 2002.
[9]
F. M. Khan, T. A. Fisher, L. Shuler, T. Wu, and W. M. Pottenger. Mining chatroom conversations for social and semantic interactions. Technical Report LU-CSE-02-011, Lehigh University, 2002.
[10]
J. W. Kim, K. S. Candan, and M. E. Dönderler. Topic segmentation of message hierarchies for indexing and navigation support. In WWW '05: Proceedings of the 14th international conference on World Wide Web, pages 322--331, 2005.
[11]
G. Kumaran and J. Allan. Text classification and named entities for new event detection. In SIGIR '04: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, pages 297--304, 2004.
[12]
G. Salton. Automatic text processing: the transformation, analysis, and retrieval of information by computer. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1989.
[13]
G. Salton and M. Smith. On the application of syntactic methodologies in automatic text analysis. In SIGIR '89: Proceedings of the 12th annual international ACM SIGIR conference on Research and development in information retrieval, pages 137--150, 1989.
[14]
A. F. Smeaton. Progress in the application of natural language processing to information retrieval tasks. Comput. J., 35(3):268--278, 1992.
[15]
K. G. Steinbach, M. and V. Kumar. A comparison of document clustering techniques. Technical report 00-034, Department of Computer Science and Engineering, University of Minnesota, 2000.
[16]
V. H. Tuulos and H. Tirri. Combining topic models and social networks for chat data mining. In WI '04: Proceedings of the Web Intelligence, IEEE/WIC/ACM International Conference on (WI'04), pages 206--213, Washington, DC, USA, 2004. IEEE Computer Society.
[17]
R. C. van. Information Retrieval. Butterworths, London, second edition edition, 1979.
[18]
A. Waibel, M. Bett, M. Finke, and R. Stiefelhagen. Meeting brower: Tracking and summarizing meetings. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1998.
[19]
W. Xi, J. Lind, and E. Brill. Learning effective ranking functions for newsgroup search. In SIGIR '04: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, pages 394--401, 2004.
[20]
Y. Yang, T. Pierce, and J. Carbonell. A study of retrospective and on-line event detection. In SIGIR '98: Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 28--36, 1998.

Cited By

View all
  • (2024)Revisiting Conversation Discourse for Dialogue DisentanglementACM Transactions on Information Systems10.1145/369819143:1(1-34)Online publication date: 30-Nov-2024
  • (2024)Toward an end-to-end implicit addressee modeling for dialogue disentanglementMultimedia Tools and Applications10.1007/s11042-024-18428-083:28(70883-70906)Online publication date: 6-Feb-2024
  • (2023)Taming Entangled Accessibility Forum Threads for Efficient Screen ReadingProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584073(65-76)Online publication date: 27-Mar-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
August 2006
768 pages
ISBN:1595933697
DOI:10.1145/1148170
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 August 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. linguistic features
  2. single-pass clustering
  3. text stream
  4. thread detection

Qualifiers

  • Article

Conference

SIGIR06
Sponsor:
SIGIR06: The 29th Annual International SIGIR Conference
August 6 - 11, 2006
Washington, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)29
  • Downloads (Last 6 weeks)2
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Revisiting Conversation Discourse for Dialogue DisentanglementACM Transactions on Information Systems10.1145/369819143:1(1-34)Online publication date: 30-Nov-2024
  • (2024)Toward an end-to-end implicit addressee modeling for dialogue disentanglementMultimedia Tools and Applications10.1007/s11042-024-18428-083:28(70883-70906)Online publication date: 6-Feb-2024
  • (2023)Taming Entangled Accessibility Forum Threads for Efficient Screen ReadingProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584073(65-76)Online publication date: 27-Mar-2023
  • (2023)Conversation Disentanglement As-a-Service2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC)10.1109/ICPC58990.2023.00018(59-63)Online publication date: May-2023
  • (2023)What are Pros and Cons? Stance Detection and Summarization on Feature Request2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1109/ESEM56168.2023.10304865(1-12)Online publication date: 26-Oct-2023
  • (2023)Detecting depression and its severity based on social media digital cuesIndustrial Management & Data Systems10.1108/IMDS-12-2022-0754123:12(3038-3052)Online publication date: 29-Sep-2023
  • (2023)Unveiling Cybersecurity Threats from Online Chat Groups: A Triple Extraction ApproachKnowledge Science, Engineering and Management10.1007/978-3-031-40292-0_15(175-188)Online publication date: 9-Aug-2023
  • (2022)Towards Just-In-Time Feature Request Approval PredictionProceedings of the 13th Asia-Pacific Symposium on Internetware10.1145/3545258.3545265(262-271)Online publication date: 11-Jun-2022
  • (2022)POSLANProceedings of the 37th ACM/SIGAPP Symposium on Applied Computing10.1145/3477314.3507028(1756-1763)Online publication date: 25-Apr-2022
  • (2022)Leveraging Multi-aspect Semantics for Conversation Structure Modeling2022 7th IEEE International Conference on Data Science in Cyberspace (DSC)10.1109/DSC55868.2022.00073(483-490)Online publication date: Jul-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media