Automatic expansion of domain-specific lexicons by term categorization

ABSTRACT

We discuss an approach to the automatic expansion of domain-specific lexicons, that is, to the problem of extending, for each c_i in a predefined set C = {c₁,…,c_m} of semantic domains, an initial lexicon Lⁱ₀ into a larger lexicon Lⁱ₁. Our approach relies on term categorization, defined as the task of labeling previously unlabeled terms according to a predefined set of domains. We approach this as a supervised learning problem in which term classifiers are built using the initial lexicons as training data. Dually to classic text categorization tasks in which documents are represented as vectors in a space of terms, we represent terms as vectors in a space of documents. We present the results of a number of experiments in which we use a boosting-based learning device for training our term classifiers. We test the effectiveness of our method by using WordNetDomains, a well-known large set of domain-specific lexicons, as a benchmark. Our experiments are performed using the documents in the Reuters Corpus Volume 1 as implicit representations for our terms.

REFERENCES

• 1: Chinatsu Aone , Scott Bennett, Applying machine learning to anaphora resolution, Connectionist, Statistical, and Symbolic Approaches to Learning for Natural Language Processing, p.302-314, January 1996
• 2: Ault, T. and Yang, Y. 2001. kNN, Rocchio and metrics for information filtering at TREC-10. In Proceedings of 10th Text Retrieval Conference (TREC-10). E. M. Voorhees, Ed. National Institute of Standards and Technology, Gaithersburg, MD. 84--93.
• 3: Eric Brill , Philip Resnik, A rule-based approach to prepositional phrase attachment disambiguation, Proceedings of the 15th conference on Computational linguistics, August 05-09, 1994, Kyoto, Japan [doi>10.3115/991250.991346]
• 4: Chen, H., Schuffels, C., and Orwing, R. 1996. Internet categorization and search: A machine learning approach. J. Visual Comm. Image Represent. Special Issue on Digital Libraries, 7, 1, 88--102.
• 5: C. J. Crouch, An approach to the automatic construction of global thesauri, Information Processing and Management: an International Journal, v.26 n.5, p.629-640, 1990 [doi>10.1016/0306-4573(90)90106-C ]
• 6: Carolyn J. Crouch , Bokyung Yang, Experiments in automatic statistical thesaurus construction, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.77-88, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133180]
• 7: Dagan, I. 2000. Contextual word similarity. In Handbook of Natural Language Processing, R. Dale, H. Moisl, and H. Somers, Eds. Marcel Dekker Inc, New York, NY. Chapter 19, 459--476.
• 8: Dagan, I., Marcus, S., and Markovitch, S. 1995. Contextual word similarity and estimation from sparse data. Comput. Speech Lang. 9, 2, 123--152.
• 9: Fellbaum, C., Ed. 1998. WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA.
• 10: Gale, W., Church, K., and Yarowsky, D. 1993. A method for disambiguating word senses in a large corpus. Comput. Humanities 26, 5/6, 415--439.
• 11: Gregory Grefenstette, Explorations in Automatic Thesaurus Discovery, Kluwer Academic Publishers, Norwell, MA, 1994
• 12: Hirschman, L., Grishman, R., and Sager, N. 1988. Grammatically-based automatic word class formation. Inform. Process. Manage. 11, 1/2, 39--57.
• 13: Lynette Hirschman , Marc Light , Eric Breck , John D. Burger, Deep Read: a reading comprehension system, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.325-332, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034731]
• 14: Jing, Y. and Croft, W. B. 1994. An association thesaurus for information retrieval. In Proceedings of 4th International Conference Recherche d'Information Assistee par Ordinateur (RIAO'94). New York, NY. 146--160.
• 15: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 16: Lesk, M. E. 1969. Word-word association in document retrieval systems. Ameri. Document. 20, 1, 27--38.
• 17: David D. Lewis , Yiming Yang , Tony G. Rose , Fan Li, RCV1: A New Benchmark Collection for Text Categorization Research, The Journal of Machine Learning Research, 5, p.361-397, 12/1/2004
• 18: Dekang Lin, Automatic retrieval and clustering of similar words, Proceedings of the 17th international conference on Computational linguistics, p.768-774, August 10-14, 1998, Montreal, Quebec, Canada
• 19: Lund, K. and Burgess, C. 1996. Producing high-dimensional semantic spaces from lexical co-occurrence. Behav. Resear. Meth. Instrument. Comput. 28, 2, 203--208.
• 20: Magnini, B. and Cavaglià, G. 2000. Integrating subject field codes into WordNet. In Proceedings of 2nd International Conference on Language Resources and Evaluation (LREC'00). Athens, Greece. 1413--1418.
• 21: Bernardo Magnini , Carlo Strapparava , Giovanni Pezzulo , Alfio Gliozzo, The role of domain information in Word Sense Disambiguation, Natural Language Engineering, v.8 n.4, p.359-373, December 2002 [doi>10.1017/S1351324902003029]
• 22: Moldovan, D., Harabagiu, S., Paşca, M., Mihalcea, R., Goodrum, R., Gîrju, R., and Rus, V. 1999. LASSO: A tool for surfing the answer net. In Proceedings of 8th Text Retrieval Conference (TREC-8). Gaithersburg, MD. 175--183.
• 23: Nardiello, P., Sebastiani, F., and Sperduti, A. 2003. Discretizing continuous attributes in AdaBoost for text categorization. In Proceedings of 25th European Conference on Information Retrieval (ECIR'03), Pisa, Italy, Springer Verlag, 320--334.
• 24: Patrick Pantel , Dekang Lin, Automatically discovering word senses, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Demonstrations, p.21-22, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073427.1073438]
• 25: Yonggang Qiu , Hans-Peter Frei, Concept based query expansion, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.160-169, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160713]
• 26: Ellen Riloff , Jessica Shepherd, A corpus-based bootstrapping algorithm for Semi-Automated semantic lexicon construction, Natural Language Engineering, v.5 n.2, p.147-156, June 1999 [doi>10.1017/S1351324999002235]
• 27: Brian Roark , Eugene Charniak, Noun-phrase co-occurrence statistics for semiautomatic semantic lexicon construction, Proceedings of the 17th international conference on Computational linguistics, p.1110-1116, August 10-14, 1998, Montreal, Quebec, Canada
• 28: Rose, T., Stevenson, M., and Whitehead, M. 2002. The Reuters Corpus Volume 1---from yesterday's news to tomorrow's language resources. In Proceedings of 3rd International Conference on Language Resources and Evaluation (LREC'02). Las Palmas, Spain, 827--832.
• 29: Gerda Ruge, Experiment on linguistically-based term associations, Information Processing and Management: an International Journal, v.28 n.3, p.317-332, 1992 [doi>10.1016/0306-4573(92)90078-E ]
• 30: Sahlgren, M. 2004. Random indexing of words in narrow context windows for vector-based semantic analysis. In Acquisition and Representation of Word Meaning: Theoretical and Computational Perspectives, A. Lenci, S. Montemagni, and V. Pirrelli, Eds. Istituti Editoriali Poligrafici Internazionali, Pisa, Italy.
• 31: Salton, G. 1971. Experiments in automatic thesaurus construction for information retrieval. In Proceedings of the IFIP Congress. Vol. TA-2. Ljubljana, Yugoslavia, 43--49.
• 32: Robert E. Schapire , Yoram Singer, BoosTexter: A Boosting-based Systemfor Text Categorization, Machine Learning, v.39 n.2-3, p.135-168, May-June 2000
• 33: Robert E. Schapire , Yoram Singer , Amit Singhal, Boosting and Rocchio applied to text filtering, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.215-223, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290996]
• 34: Bruce R. Schatz , Eric H. Johnson , Pauline A. Cochrane , Hsinchun Chen, Interactive term suggestion for users of digital libraries: using subject thesauri and co-occurrence lists for information retrieval, Proceedings of the first ACM international conference on Digital libraries, p.126-133, March 20-23, 1996, Bethesda, Maryland, United States [doi>10.1145/226931.226956]
• 35: Schäuble, P. and Knaus, D. 1992. The various roles of information structures. In Proceedings of the 16th Annual Conference of the Gesellschaft für Klassifikation, Dortmund, Germany, O. Opitz, B. Lausen, and R. Klar, Eds. 282--290. Springer Verlag, Heidelberg, Germany, 1993.
• 36: Schmid, H. 1994. Probabilistic part-of-speech tagging using decision trees. In Proceedings of the International Conference on New Methods in Language Processing. Manchester, UK, 44--49.
• 37: H. Schütze, Dimensions of meaning, Proceedings of the 1992 ACM/IEEE conference on Supercomputing, p.787-796, November 16-20, 1992, Minneapolis, Minnesota, United States
• 38: Hinrich Schütze , Jan O. Pedersen, A cooccurrence-based thesaurus and two applications to information retrieval, Information Processing and Management: an International Journal, v.33 n.3, p.307-318, May 1997 [doi>10.1016/S0306-4573(96)00068-4 ]
• 39: Fabrizio Sebastiani, Machine learning in automated text categorization, ACM Computing Surveys (CSUR), v.34 n.1, p.1-47, March 2002 [doi>10.1145/505282.505283]
• 40: Fabrizio Sebastiani , Alessandro Sperduti , Nicola Valdambrini, An improved boosting algorithm and its application to text categorization, Proceedings of the ninth international conference on Information and knowledge management, p.78-85, November 06-11, 2000, McLean, Virginia, United States [doi>10.1145/354756.354804]
• 41: Páraic Sheridan , Jean Paul Ballerini, Experiments in multilingual information retrieval using the SPIDER system, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.58-65, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243213]
• 42: Paraic Sheridan , Martin Braschler , Peter Schäuble, Cross-Language Information Retrieval in a Multilingual Legal Domain, Proceedings of the First European Conference on Research and Advanced Technology for Digital Libraries, p.253-268, September 01-03, 1997
• 43: Frank Smadja, Retrieving collocations from text: Xtract, Computational Linguistics, v.19 n.1, March 1993
• 44: Soderland, S., Fisher, D., Aseltine, J., and Lehnert, W. 1995. CRYSTAL: Inducing a conceptual dictionary. In Proceedings of 14th International Joint Conference on Artificial Intelligence (IJCAI'95). Montreal, Canada, 1314--1319.
• 45: Spärck Jones, K. 1971. Automatic Keyword Classification for Information Retrieval. Butterworths, London, UK.
• 46: Michael Thelen , Ellen Riloff, A bootstrapping method for learning semantic lexicons using extraction pattern contexts, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.214-221, July 06, 2002 [doi>10.3115/1118693.1118721]
• 47: Tokunaga, T., Iwayama, M., and Tanaka, H. 1995. Automatic thesaurus construction based on grammatical relations. In Proceedings of 14th International Joint Conference on Artificial Intelligence (IJCAI'95). Montreal, Canada, 1308--1313.
• 48: Yiming Yang , Jan O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.412-420, July 08-12, 1997
• 49: David Yarowsky, Word-sense disambiguation using statistical models of Roget's categories trained on large corpora, Proceedings of the 14th conference on Computational linguistics, August 23-28, 1992, Nantes, France [doi>10.3115/992133.992140]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

Keywords:
Lexicons, machine learning, text classification