skip to main content
10.1145/2064448.2064473acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Tweet classification by data compression

Published: 24 October 2011 Publication History

Abstract

We propose a new method that uses data compression for classifying an unseen tweet as being related to an interesting topic or not. Our compression-based tweet classification method, called CTC, evaluates the compressibility of the tweet when given positive and negative examples. This enables our method to handle multilingual tweets in the same manner and to effectively utilize the word context of the tweet, which is extremely important information in the 140 character limit. Experiments with worldwide tweets assigned a single hashtag demonstrate that our method, which uses the Deflate algorithm (used in gzip) for empirical evaluations, achieved higher precision and recall rates than state-of-the-art online learning algorithms.

References

[1]
D. Benedetto, E. Caglioti, and V. Loreto. Language trees and zipping. Physical Review Letters, 88(4), 28 Jan. 2002.
[2]
A. Bratko, G. V. Cormack, B. Filipič, T. R. Lynam, and B. Zupan. Spam filtering using statistical data compression models. Journal of Machine Learning Research, 7:2673--2698, 2006.
[3]
M. Burrows and D. J. Wheeler. A block-sorting lossless data compression algorithm. Technical Report SRC-RR-124, Digiral Systems Research Center, 1994.
[4]
R. Cilibrasi and P. Vitányi. Clustering by compression. IEEE Transactions on Information Theory, 51(4):1523--1545, 2005.
[5]
J. G. Cleary and I. H. Witten. Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications, COM-32(4):396--402, 1984.
[6]
G. Cormack and R. N. S. Horspool. Data compression using dynamic markov modelling. The Computer Journal, 30(6):541--550, 1987.
[7]
K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. Online passive-agressive algorithm. Journal of Machine Learning Research, 7:551--585, 2006.
[8]
M. Dredze, K. Crammer, and F. Pereira. Confidence-weighted linear classification. In Proceedings of 25th International Conference on Machine Learning, pages 264--271, 2008.
[9]
D. Irani, S. Webb, C. Pu, and K. Li. Study of trend-stuffing on twitter through text classification. In Proceedings of 7th Annual Collaboration, Electronic messaging, Anti-Abuse and Spam Conference, 2010.
[10]
H. Kwak, C. Lee, H. Park, and S. Moon. What is twitter, a social network or a news media? In Proceedings of 19th International Conference on World Wide Web, pages 591--600, 2010.
[11]
M. Li, X. Chen, X. Li, B. Ma, and P. M. B. Vitányi. The similarity metric. IEEE Transactions on Information Theory, 50(12):3250--3264, 2004.
[12]
M. Li and P. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications. Springer, 2nd edition, 1997.
[13]
Y. Marton, N. Wu, and L. Hellerstein. On compression-based text classification. In Proceedings of the 27th European Conference on Information Retrieval, pages 300--314, 2005.
[14]
T. Sakaki, M. Okazaki, and Y. Matsuo. Earthquake shakes twitter users: Real-time event detection by social sensors. In Proceedings of 19th International Conference on World Wide Web, pages 851--860, 2010.
[15]
B. Sriram, D. Fuhry, and M. Demirbas. Short text classification in twitter to improve information filtering. In Proceedings of 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 841--842, 2010.
[16]
I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes: compressing and indexing documents and images. Morgan Kaufmann, 2nd edition, 1999.
[17]
J. Ziv and A. Lempel. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory, IT-23(3):337--343, 1977.

Cited By

View all
  • (2024)Twitter Data for Syndromic Surveillance: Insights and Methods2024 IEEE International Conference on Information Technology, Electronics and Intelligent Communication Systems (ICITEICS)10.1109/ICITEICS61368.2024.10625534(1-8)Online publication date: 28-Jun-2024
  • (2022)Editing Compression Dictionaries toward Refined Compression-Based Feature-SpaceInformation10.3390/info1306030113:6(301)Online publication date: 15-Jun-2022
  • (2019)Compression Analytics for Classification and Anomaly Detection Within Network CommunicationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2018.287817214:5(1366-1376)Online publication date: May-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
DETECT '11: Proceedings of the 2011 international workshop on DETecting and Exploiting Cultural diversiTy on the social web
October 2011
48 pages
ISBN:9781450309622
DOI:10.1145/2064448
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. data compression
  2. text classification
  3. twitter

Qualifiers

  • Research-article

Conference

CIKM '11
Sponsor:

Acceptance Rates

DETECT '11 Paper Acceptance Rate 6 of 7 submissions, 86%;
Overall Acceptance Rate 6 of 7 submissions, 86%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Twitter Data for Syndromic Surveillance: Insights and Methods2024 IEEE International Conference on Information Technology, Electronics and Intelligent Communication Systems (ICITEICS)10.1109/ICITEICS61368.2024.10625534(1-8)Online publication date: 28-Jun-2024
  • (2022)Editing Compression Dictionaries toward Refined Compression-Based Feature-SpaceInformation10.3390/info1306030113:6(301)Online publication date: 15-Jun-2022
  • (2019)Compression Analytics for Classification and Anomaly Detection Within Network CommunicationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2018.287817214:5(1366-1376)Online publication date: May-2019
  • (2019)Short-text learning in social media: a reviewThe Knowledge Engineering Review10.1017/S026988891900001834Online publication date: 6-Jun-2019
  • (2018)A Comparison of Supervised Machine Learning Approaches for Categorized TweetsInternational Conference on Intelligent Data Communication Technologies and Internet of Things (ICICI) 201810.1007/978-3-030-03146-6_47(422-430)Online publication date: 21-Dec-2018
  • (2017)Compression-Based Algorithms for Deception DetectionSocial Informatics10.1007/978-3-319-67217-5_16(257-276)Online publication date: 3-Sep-2017
  • (2016)A compression-based technique to classify metamorphic malware2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA)10.1109/AICCSA.2016.7945801(1-6)Online publication date: Nov-2016
  • (2015)Twitter Mining for Discovery, Prediction and CausalityInternational Journal of Intelligent Systems in Accounting and Finance Management10.1002/isaf.137622:3(227-247)Online publication date: 1-Jul-2015
  • (2014)Hierarchical multi-label classification of social text streamsProceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval10.1145/2600428.2609595(213-222)Online publication date: 3-Jul-2014
  • (2014)Mining Social Media Data for Understanding Students’ Learning ExperiencesIEEE Transactions on Learning Technologies10.1109/TLT.2013.22965207:3(246-259)Online publication date: Jul-2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media