Algorithms for the verification of the semantic relation between a compound and a given lexeme

ABSTRACT

Text mining on a lexical basis is quite well developed for the English language. In compounding languages, however, lexicalized words are often a combination of two or more semantic units. New words can be built easily by concatenating existing ones, without putting any white spaces in between.

That poses a problem to existing search algorithms: Such compounds could be of high interest for a search request, but how can be examined whether a compound comprises a given lexeme? A string match can be considered as an indication, but does not prove semantic relation. The same problem is faced when using lexicon based approaches where signal words are defined as lexemes only and need to be identified in all forms of appearance, and hence also as component of a compound. This paper explores the characteristics of compounds and their constituent elements for German, and compares seven algorithms with regard to runtime and error rates. The results of this study are relevant to query analysis and term weighting approaches in information retrieval system design.

References

1. Alfonseca, E., Bilac, S., Pharies, S.: Decompounding query keywords from compounding languages, in: Proceedings of ACL-08, Columbus, 2008, pp. 253--256.
2. Alfonseca, E., Bilac, S., Pharies, S.: German Decompounding in a Difficult Corpus, Springer, Berlin, 2008, pp. 128--139.
3. Baccianella, S., Esuli, A., Sebastiani, F.: SENTIWORDNET 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. Proceedings of the 7th conference on International Language Resources and Evaluation LREC10, pp. 2200--2204, 2008.
4. Bozsahin, C.: The Combinatory Morphemic Lexicon, Middle East Technical University, Turkey, 2002.
5. Braschler, M., Ripplinger, B.: How Effective is Stemming and Decompounding for German Text Retrieval?, in: Information Retrieval 7, 2004, pp. 291--316.
6. Brown, R. D.: Corpus-Driven Splitting of Compound Words, in: Proceedings of the TMI 2002, Keihanna, Japan, 2011, pp. 12--21.
7. Canoo Engineering AG: Deutsche Wörterbücher und Grammatik. Available at: http://www.canoo.net/services/WordformationRules/ueberblick/ (Feb. 2012)
8. Carstensen, K.-U., Ebert, Ch., Ebert, C., Jekat, S., Langer, H. and Klabunde, R.: Computerlinguistik und Sprachtechnologie, Elsevier, München, 2009.
9. Geyken, A. and Hanneforth, T.: TAGH: A Complete Morphology for German Based on Weighted Finite State Automata. In Proceedings of the FSMNLP 2005, Springer, Berlin, 2006, 55--66.
10. Gupta, G. K.: Introduction to Data Mining with Case Studies, Prentice-Hall of India, New Delhi, 2006.
11. Hess, W.: Grundlagen der Phonetik, Rheinische Friedrich Wilhelms Universität Bonn, 2001.
12. Holz, F., Biemann, C.: Unsupervised and Knowledge-free Learning of Compound Splits and Periphrases, Springer, Berlin, 2008, pp. 117--127.
13. Ingason, A. K., Helgadóttir, S., Loftsson, H. and Rögnvaldsson, E.: A Mixed Method Lemmatization Algorithm Using a Hierarchy of Linguistic Identities (HOLI). In Proceedings of GoTAL 2008, LNAI vol. 5221. Berlin: Springer, 2008, pp. 205--216.
14. Jürgenson, I. B.: Neuronale Korrelate phonotaktischer Verarbeitung, Dissertation: Universitätsmedizin Berlin, 2009.
15. Kellner, G.: Wege der Kommunikationsoptimierung. Anwendung von NLP im Bereich der Künstlichen Intelligenz. VDM, Saarbrücken, 2010.
16. Kellner, G., Berendt, B.: Extracting Knowledge about Cognitive Style. The Use of Sensory Vocabulary in Forums: A Text Mining Approach, in Proceedings of the NLPKE 2011, IEEE Press, 2011.
17. Macherey, K., Dai, A. M., Talbot, D., Popat, A. C. and Och, F.: Language-independent compound splitting with morphological operations, in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011; pp. 1395--1404.
18. Porter, M. F.: An algorithm for suffix stripping, in: Program, Nr. 3, 1980, pp. 130--137.
19. Porter, M. F., Boulton, R., Miles, P. et al.: Snowball Project: German Stemming Algorithm. Available at: http://snowball.tartarus.org/algorithms/german/stemmer.html (Feb. 2012)
20. Protaziuk, G., Kryszkiewicz, M., Rybinski, H., Delteil, A.: Discovering Compound and Proper Nouns, Springer, Berlin, 2007, pp. 505--515.
21. Stymne, S.: German Compounds in Factored Statistical Machine Translation, in Proceedings of GoTAL, 6th International Conference on Natural Language Processing, Springer LNCS/LNAI Vol. 5221, 2008, pp. 464--475.
22. Williams, E.: On the Notions "Lexically Related" and "Head of a Word". Linguistic Inquiry 12/2, 1981, pp. 245--274.
23. Zemb, J. M.: Vergleichende Grammatik Französisch--Deutsch. Part 1: Comparaison de deux systèmes. Part 2: L'économie de la langue et le jeu de la parole. Duden, Mannheim, 1984.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Natural language processing