A concept-based model for enhancing text categorization

ABSTRACT

Most of text categorization techniques are based on word and/or phrase analysis of the text. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in their documents, but one term contributes moreto the meaning of its sentences than the other term. Thus, the underlying model should indicate terms that capture these mantics of text. In this case, the model can capture terms that present the concepts of the sentence, which leads todiscover the topic of the document. A new concept-based model that analyzes terms on the sentence and document levels rather than the traditional analysis of document only is introduced. The concept-based model can effectively discriminate between non-important terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. The proposed model consists of concept-based statistical analyzer, conceptual ontological graph representation,and concept extractor. The term which contributes to the sentence semantics is assigned two different weights by the concept-based statistical analyzer and the conceptual ontological graph representation. These two weights are combined into a new weight. The concepts that have maximum combined weights are selected by the concept extractor. A set of experiments using the proposed concept-basedmodel on different datasets in text categorization is conducted. The experiments demonstrate the comparison between traditional weighting and the concept-based weighting obtained by the combined approach of the concept-based statistical analyzer and the conceptual ontological graph. The evaluation of results is relied on two quality measures, the Macro-averaged F1 and the Error rate. These quality measures are improved when the newly developedconcept-based model is used to enhance the quality of thetext categorization.

REFERENCES

• 1: K. Aas and L. Eikvil. Text categorisation: A survey. technical report 941. Technical report, Norwegian Computing Center, June 1999.
• 2: Michael John Collins , Mitchell P. Marcus, Head-driven statistical models for natural language parsing, University of Pennsylvania, Philadelphia, PA, 1999
• 3: R. Feldman and I. Dagan. Knowledge discovery in textual databases (kdt). In Proceedings of First International Conference on Knowledge Discovery and Data Mining, pages 112--117, 1995.
• 4: C. Fillmore. The case for case. Chapter in: Universals in Linguistic Theory. Holt, Rinehart and Winston, Inc., New York, 1968.
• 5: W. Francis and H. Kucera. Manual of information to accompany a standard corpus of present-day edited american english, for use with digital computers, 1964.
• 6: Daniel Gildea , Daniel Jurafsky, Automatic labeling of semantic roles, Computational Linguistics, v.28 n.3, p.245-288, September 2002 [doi>10.1162/089120102760275983]
• 7: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 8: Daniel Jurafsky , James H. Martin, Speech and Language Processing (2nd Edition), Prentice-Hall, Inc., Upper Saddle River, NJ, 2006
• 9: P. Kingsbury and M. Palmer. Propbank: the next level of treebank. In Proceedings of Treebanks and Lexical Theories, 2003.
• 10: M. F. Porter. An algorithm for suffix stripping. Program, 14(3): 130--137, July 1980.
• 11: Sameer Pradhan , Kadri Hacioglu , Valerie Krugler , Wayne Ward , James H. Martin , Daniel Jurafsky, Support Vector Learning for Semantic Argument Classification, Machine Learning, v.60 n.1-3, p.11-39, September 2005 [doi>10.1007/s10994-005-0912-2]
• 12: Sameer Pradhan , Kadri Hacioglu , Wayne Ward , James H. Martin , Daniel Jurafsky, Semantic Role Parsing: Adding Semantic Structure to Unstructured Text, Proceedings of the Third IEEE International Conference on Data Mining, p.629, November 19-22, 2003
• 13: S. Pradhan, W. Ward, K. Hacioglu, J. Martin, and D. Jurafsky. Shallow semantic parsing using support vector machines. In Proceedings of the Human Language Technology/North American Association for Computational Linguistics (HLT/NAACL), 2004.
• 14: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 15: G. Salton , A. Wong , C. S. Yang, A vector space model for automatic indexing, Communications of the ACM, v.18 n.11, p.613-620, Nov. 1975 [doi>10.1145/361219.361220]
• 16: Shady Shehata , Fakhri Karray , Mohamed Kamel, Enhancing Text Clustering Using Concept-based Mining Model, Proceedings of the Sixth International Conference on Data Mining, p.1043-1048, December 18-22, 2006 [doi>10.1109/ICDM.2006.64]
• 17: Shady Shehata , Fakhri Karray , Mohamed Kamel, Enhancing Text Retrieval Performance using Conceptual Ontological Graph, Proceedings of the Sixth IEEE International Conference on Data Mining - Workshops, p.39-44, December 18-22, 2006 [doi>10.1109/ICDMW.2006.71]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Language parsing and understanding

General Terms:
Algorithms, Design, Experimentation, Theory

Keywords:
concept-based categorization, concept-based indexing, concepts