Improved annotation of the blogosphere via autotagging and hierarchical clustering

ABSTRACT

Tags have recently become popular as a means of annotating and organizing Web pages and blog entries. Advocates of tagging argue that the use of tags produces a 'folksonomy', a system in which the meaning of a tag is determined by its use among the community as a whole. We analyze the effectiveness of tags for classifying blog entries by gathering the top 350 tags from Technorati and measuring the similarity of all articles that share a tag. We find that tags are useful for grouping articles into broad categories, but less effective in indicating the particular content of an article. We then show that automatically extracting words deemed to be highly relevant can produce a more focused categorization of articles. We also show that clustering algorithms can be used to reconstruct a topical hierarchy among tags, and suggest that these approaches may be used to address some of the weaknesses in current tagging systems.

REFERENCES

• 1: F. Baader, D. Calvanese, D. McGuinness, D. Nardi, and P. Patel-Schneider, editors. The Description Logic Handbook. Cambridge Press, 2003.
• 2: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 3: T. Berners-Lee and D. Connolly. Hypertext markup language specification -- 2.0. Technical Report RFC 1866, MIT/W3C, 1996.
• 4: W. B. Cavnar and J. M. Trenkle. N-gram-based text categorization. In Symposium On Document Analysis and Information Retrieval, pages 161--176, University of Nevada-Las Vegas, 1994.
• 5: Douglass R. Cutting , David R. Karger , Jan O. Pedersen , John W. Tukey, Scatter/Gather: a cluster-based approach to browsing large document collections, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.318-329, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133214]
• 6: Roy Thomas Fielding , Richard N. Taylor, Architectural styles and the design of network-based software architectures, 2000
• 7: A. Mathes. Folksonomies - cooperative classification and communication through shared metadata. Available at: http://www.adammathes.com/academic/computer-mediated-communication/folksonomies.html, 2004.
• 8: George A. Miller, WordNet: a lexical database for English, Communications of the ACM, v.38 n.11, p.39-41, Nov. 1995 [doi>10.1145/219717.219748]
• 9: E. Quintarelli. Folksonomies: power to the people. Paper presented at the ISKO Italy-UniMIB meeting. Available at http://www.iskoi.org/doc/folksonomies.htm, June 2005.
• 10: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 11: C. Shirky. Folksonomy. Blog entry at http://www.corante.com/many/archives/2004/08/25/folksonomy.php, August 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation

Keywords:
automated annotation, blogs, hierarchical clustering, tagging