Mining topic-specific concepts and definitions on the web

ABSTRACT

Traditionally, when one wants to learn about a particular topic, one reads a book or a survey paper. With the rapid expansion of the Web, learning in-depth knowledge about a topic from the Web is becoming increasingly important and popular. This is also due to the Web's convenience and its richness of information. In many cases, learning from the Web may even be essential because in our fast changing world, emerging topics appear constantly and rapidly. There is often not enough time for someone to write a book on such topics. To learn such emerging topics, one can resort to research papers. However, research papers are often hard to understand by non-researchers, and few research papers cover every aspect of the topic. In contrast, many Web pages often contain intuitive descriptions of the topic. To find such Web pages, one typically uses a search engine. However, current search techniques are not designed for in-depth learning. Top ranking pages from a search engine may not contain any description of the topic. Even if they do, the description is usually incomplete since it is unlikely that the owner of the page has good knowledge of every aspect of the topic. In this paper, we attempt a novel and challenging task, mining topic-specific knowledge on the Web. Our goal is to help people learn in-depth knowledge of a topic systematically on the Web. The proposed techniques first identify those sub-topics or salient concepts of the topic, and then find and organize those informative pages, containing definitions and descriptions of the topic and sub-topics, just like those in a book. Experimental results using 28 topics show that the proposed techniques are highly effective.

REFERENCES

• 1: Rakesh Agrawal , Ramakrishnan Srikant, Fast Algorithms for Mining Association Rules in Large Databases, Proceedings of the 20th International Conference on Very Large Data Bases, p.487-499, September 12-15, 1994
• 2: Corin R. Anderson , Eric Horvitz, Web montage: a dynamic personalized start page, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511537]
• 3: Naveen Ashish , Craig A. Knoblock, Wrapper generation for semi-structured Internet sources, ACM SIGMOD Record, v.26 n.4, p.8-15, Dec. 1997 [doi>10.1145/271074.271078]
• 4: AskJeeves, Inc., AskJeeves Question-Answering Search Engine, http://www.ask.com.
• 5: Roberto J. Bayardo, Jr., Efficiently mining long patterns from databases, Proceedings of the 1998 ACM SIGMOD international conference on Management of data, p.85-93, June 01-04, 1998, Seattle, Washington, United States
• 6: Bennett, N.A., He, Q., Powell, K., Schatz, B.R.: Extracting noun phrases for all of MEDLINE, In Proc. American Medical Informatics Assoc., 1999.
• 7: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, p.107-117, April 1998, Brisbane, Australia
• 8: S. Ceri , S. Comai , E. Damiani , P. Fraternali , L. Tanca, Complex queries in XML-GL, Proceedings of the 2000 ACM symposium on Applied computing, p.888-893, March 2000, Como, Italy [doi>10.1145/338407.338677]
• 9: Soumen Chakrabarti , Byron Dom , Prabhakar Raghavan , Sridhar Rajagopalan , David Gibson , Jon Kleinberg, Automatic resource compilation by analyzing hyperlink structure and associated text, Proceedings of the seventh international conference on World Wide Web 7, p.65-74, April 1998, Brisbane, Australia
• 10: William W. Cohen , Wei Fan, Learning page-independent heuristics for extracting data from Web pages, Proceeding of the eighth international conference on World Wide Web, p.1641-1652, May 1999, Toronto, Canada
• 11: Cooper, R.J. & Rüüger, S. M.: A simple question answering system, In Proc. of TREC 9, 2000.
• 12: Daille, B.: Study and implementation of combined techniques for automatic extraction of terminology, In The Balancing Act: Combining Symbolic and Statistical Approaches to Language. The MIT Press, 1996
• 13: Jeffrey Dean , Monika R. Henzinger, Finding related pages in the World Wide Web, Proceeding of the eighth international conference on World Wide Web, p.1467-1479, May 1999, Toronto, Canada
• 14: Ronen Feldman , Yair Liberzon , Binyamin Rosenfeld , Jonathan Schler , Jonathan Stoppi, A framework for specifying explicit bias for revision of approximate information extraction rules, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.189-197, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347125]
• 15: Eibe Frank , Gordon W. Paynter , Ian H. Witten , Carl Gutwin , Craig G. Nevill-Manning, Domain-Specific Keyphrase Extraction, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, p.668-673, July 31-August 06, 1999
• 16: Tao Guan , Kam-Fai Wong, KPS: a Web information mining algorithm, Proceeding of the eighth international conference on World Wide Web, p.1495-1507, May 1999, Toronto, Canada
• 17: Harabagiu, S., Moldovan, D., Pasca, M., Mihalcea, R., Surdeanu, M., Bunescu, R., Girju, R., Rus, V. & Morarescu, P. : FALCON: Boosting knowledge for answering engines, In Proc. of TREC 9, 2000.
• 18: Katz, B.: From sentence parsing to information access on the WWW, In AAAI Spring Symposium on Natural Language Processing for the WWW, 1997 http://www.ai.mit.edu/projects/infolab/ailab.html
• 19: Klavans, J. L. & Muresan, S.: DEFINDER: Rule-based methods for the extraction of medical terminology and their associated definitions from on-line text, In proc. of American Medical Informatics Assoc., 2000.
• 20: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, p.668-677, January 25-27, 1998, San Francisco, California, United States
• 21: Ravi Kumar , Prabhakar Raghavan , Sridhar Rajagopalan , Andrew Tomkins, Extracting Large-Scale Knowledge Bases from the Web, Proceedings of the 25th International Conference on Very Large Data Bases, p.639-650, September 07-10, 1999
• 22: Cody C. T. Kwok , Oren Etzioni , Daniel S. Weld, Scaling question answering to the Web, Proceedings of the 10th international conference on World Wide Web, p.150-161, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.371973]
• 23: Lawrence, S.: Context in Web Search, In IEEE Data Engineering Bulletin 23(3): 25--32, 2000.
• 24: Liu, B., Hsu, W., Ma, Y.: Integrating classification and association rule mining, In Proc. of KDD-98, 1998.
• 25: Yoelle S. Maarek , Israel Z. Ben Shaul, Automatically organizing bookmarks per contents, Proceedings of the fifth international World Wide Web conference on Computer networks and ISDN systems, p.1321-1333, May 1996, Paris, France
• 26: Mendelzon, A., Mihaila, G. & Milo, T.: Querying the World Wide Web, In Journal of Digital Libraries 1(1): 68--88, 1997.
• 27: Daniel Siaw Weng Ngu , Xindong Wu, SiteHelper: a localized agent that helps incremental exploration of the World Wide Web, Selected papers from the sixth international conference on World Wide Web, p.1249-1255, September 1997, Santa Clara, California, United States
• 28: Page, L., Brin, S., Motwani, R. & Winograd, T. : The PageRank citation ranking: Bringing order to the Web, In Stanford CS Technical Report, 1998.
• 29: Porter, M.F. : An algorithm for suffix stripping, Program 14(3):130--137,1980 http://www.tartarus.org/~martin/PorterStemmer/
• 30: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 31: Smadja, F.: Retrieving collocations from text: Xtract, In Using Large Corpora. London: MIT Press pp143--177, 1994
• 32: George M. Underwood , Paul P. Maglio , Rob Barrett, User-centered push for timely information delivery, Proceedings of the seventh international conference on World Wide Web 7, p.33-41, April 1998, Brisbane, Australia
• 33: Voutilainen, A. : NPtool: A detector of English noun phrase, In Proc. of Workshop on Very Large Corpora, 1993.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models; Selection process; Search process; Query formulation

General Terms:
Algorithms, Experimentation

Keywords:
definition mining, domain concept mining, information integration, knowledge compilation, web content mining