Named entity translation matching and learning: With application for mining unseen translations

ABSTRACT

This article introduces a named entity matching model that makes use of both semantic and phonetic evidence. The matching of semantic and phonetic information is captured by a unified framework via a bipartite graph model. By considering various technical challenges of the problem, including order insensitivity and partial matching, this approach is less rigid than existing approaches and highly robust. One major component is a phonetic matching model which exploits similarity at the phoneme level. Two learning algorithms for learning the similarity information of basic phonemic matching units based on training examples are investigated. By applying the proposed named entity matching model, a mining system is developed for discovering new named entity translations from daily Web news. The system is able to discover new name translations that cannot be found in the existing bilingual dictionary.

REFERENCES

• 1: Ravindra K. Ahuja , Thomas L. Magnanti , James B. Orlin, Network flows: theory, algorithms, and applications, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 2: Yaser Al-Onaizan , Kevin Knight, Translating named entities using monolingual and bilingual resources, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073150]
• 3: Allan, J., Lavrenko, V., and Nallapati, R. 2002. UMass at TDT 2002. In Topic Detection and Tracking Workshop.
• 4: Amit Bagga , Breck Baldwin, Entity-based cross-document coreferencing using the Vector Space Model, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
• 5: Daniel M. Bikel , Richard Schwartz , Ralph M. Weischedel, An Algorithm that Learns What‘s in a Name, Machine Learning, v.34 n.1-3, p.211-231, Feb. 1999
• 6: Black, A. W., Lenzo, K., and Pagel, V. 1998. Issues in building general letter to sound rules. In Proceedings of the 3rd European Speech Communication Association (ESCA). International Workshop on Speech Synthesis. 77--80.
• 7: Eric Brill, Transformation-based error-driven learning and natural language processing: a case study in part-of-speech tagging, Computational Linguistics, v.21 n.4, p.543-565, December 1995
• 8: Peter F. Brown , Vincent J. Della Pietra , Stephen A. Della Pietra , Robert L. Mercer, The mathematics of statistical machine translation: parameter estimation, Computational Linguistics, v.19 n.2, June 1993
• 9: Chen, H., Ding, Y., Tsai, S., and Bian, G. 1998. Description of the NTU system used for MET-2. In Proceedings of the 7th Message Understanding Conference.
• 10: Hsin-Hsi Chen , Wen-Cheng Lin , Changhua Yang , Wei-Hao Lin, Translating–transliterating named entities for multilingual information access, Journal of the American Society for Information Science and Technology, v.57 n.5, p.645-659, March 2006 [doi>10.1002/asi.v57:5]
• 11: Pu-Jen Cheng , Jei-Wen Teng , Ruei-Cheng Chen , Jenq-Haur Wang , Wen-Hsiang Lu , Lee-Feng Chien, Translating unknown queries with web corpora for cross-language information retrieval, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009020]
• 12: Pascale Fung, A Statistical View on Bilingual Lexicon Extraction: From Parallel Corpora to Non-parallel Corpora, Proceedings of the Third Conference of the Association for Machine Translation in the Americas on Machine Translation and the Information Soup, p.1-17, October 28-31, 1998
• 13: Pascale Fung , Lo Yuen Yee, An IR approach for translating new words from nonparallel, comparable texts, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
• 14: Jianfeng Gao , Ming Zhou , Jian-Yun Nie , Hongzhao He , Weijun Chen, Resolving query translation ambiguity using a decaying co-occurrence model and syntactic dependence relations, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564409]
• 15: Fei Huang, Improved Named Entity Translation and Bilingual Named Entity Extraction, Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, p.253, October 14-16, 2002
• 16: Fei Huang , Stephan Vogel , Alex Waibel, Automatic extraction of named entity translingual equivalence based on multi-feature cost minimization, Proceedings of the ACL 2003 workshop on Multilingual and mixed-language named entity recognition, p.9-16, July 12, 2003 [doi>10.3115/1119384.1119386]
• 17: Fei Huang , Stephan Vogel , Alex Waibel, Extracting named entity translingual equivalence with limited resources, ACM Transactions on Asian Language Information Processing (TALIP), v.2 n.2, p.124-129, June 2003 [doi>10.1145/974740.974745]
• 18: Huang, F., Vogel, S., and Waibel, A. 2004. Improving named entity translation combining phonetic and semantic similarities. In Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. 281--288.
• 19: Jyrki Kivinen , Manfred K. Warmuth, Exponentiated gradient versus gradient descent for linear predictors, Information and Computation, v.132 n.1, p.1-63, Jan. 10, 1997 [doi>10.1006/inco.1996.2612 ]
• 20: Kevin Knight , Jonathan Graehl, Machine transliteration, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.128-135, July 07-12, 1997, Madrid, Spain
• 21: Kuhn, H. 1955. The Hungarian method for the assignment problem. Naval Rese. Logist. Quart. 2, 83--97.
• 22: Wai Lam , Pik-Shan Cheung , Ruizhang Huang, Mining events and new name translations from online daily news, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996418]
• 23: Wai Lam , Ruizhang Huang , Pik-Shan Cheung, Learning phonetic similarity for matching named entity translations and mining new translations, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009043]
• 24: Li, X., Morie, P., and Roth, D. 2004. Robust reading: Identification and tracing of ambiguous names. In Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. 17--24.
• 25: Wei-Hao Lin , Hsin-Hsi Chen, Backward machine transliteration by learning phonetic similarity, proceeding of the 6th conference on Natural language learning, p.1-7, August 31, 2002 [doi>10.3115/1118853.1118870]
• 26: Wen-Hsiang Lu , Lee-Feng Chien , Hsi-Jian Lee, Translation of web queries using anchor text mining, ACM Transactions on Asian Language Information Processing (TALIP), v.1 n.2, p.159-172, June 2002 [doi>10.1145/568954.568958]
• 27: Meng, H., Lo, W.-K., Chen, B., and Tang, K. 2001. Generating phonetic cognates to handle named entities in English-Chinese cross-language spoken document retrieval. In IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU01). 311--314.
• 28: Jian-Yun Nie , Michel Simard , Pierre Isabelle , Richard Durand, Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the Web, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.74-81, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312656]
• 29: Cheng Niu , Wei Li , Rohini K. Srihari, Weakly supervised learning for cross-document person name disambiguation supported by information extraction, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.597-es, July 21-26, 2004, Barcelona, Spain [doi>10.3115/1218955.1219031]
• 30: Pagel, V., Lenzo, K., and Black, A. W. 1998. Letter to sound rules for accented lexicon compression. In Proceedings of International Conference on Spoken Language Processing. 252--255.
• 31: Reinhard Rapp, Automatic identification of word translations from unrelated English and German corpora, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.519-526, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034756]
• 32: Philip Resnik , Noah A. Smith, The Web as a parallel corpus, Computational Linguistics, v.29 n.3, p.349-380, September 2003 [doi>10.1162/089120103322711578]
• 33: Li Shao , Hwee Tou Ng, Mining new word translations from comparable corpora, Proceedings of the 20th international conference on Computational Linguistics, p.618-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220444]
• 34: Thompson, P. and Dozier, C. 1997. Name searching and information retrieval. In Proceedings of the 2nd Conference on Empirical Methods in Natural Language Processing. 134--140.
• 35: Voorhees, E. and Tice, D. 2000. The TREC-8 question answering track evaluation. In Proceedings of the 8th Text Retrieval Conference (TREC-8).
• 36: Widrow, B. and Hoff, M. 1960. Adaptive switching circuits. IRE WESCON Convention Record, 96--104.
• 37: Shiren Ye , Tat-Seng Chua , Liu Jimin, An agent-based approach to Chinese named entity recognition, Proceedings of the 19th international conference on Computational linguistics, p.1-7, August 24-September 01, 2002, Taipei, Taiwan [doi>10.3115/1072228.1072308]
• 38: Bing Zhao , Stephan Vogel, Adaptive Parallel Sentences Mining from Web Bilingual News Collection, Proceedings of the 2002 IEEE International Conference on Data Mining (ICDM'02), p.745, December 09-12, 2002

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

General Terms:
Algorithms, Design

Keywords:
Text mining, learning phonetic information, named entity translation