skip to main content
10.1145/1076034.1076085acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
Article

Linear discriminant model for information retrieval

Published: 15 August 2005 Publication History

Abstract

This paper presents a new discriminative model for information retrieval (IR), referred to as linear discriminant model (LDM), which provides a flexible framework to incorporate arbitrary features. LDM is different from most existing models in that it takes into account a variety of linguistic features that are derived from the component models of HMM that is widely used in language modeling approaches to IR. Therefore, LDM is a means of melding discriminative and generative models for IR. We present two algorithms of parameter learning for LDM. One is to optimize the average precision (AP) directly using an iterative procedure. The other is a perceptron-based algorithm that minimizes the number of discordant document-pairs in a rank list. The effectiveness of our approach has been evaluated on the task of ad hoc retrieval using six English and Chinese TREC test sets. Results show that (1) in most test sets, LDM significantly outperforms the state-of-the-art language modeling approaches and the classical probabilistic retrieval model; (2) it is more appropriate to train LDM using a measure of AP rather than likelihood if the IR system is graded on AP; and (3) linguistic features (e.g. phrases and dependences) are effective for IR if they are incorporated properly.

References

[1]
Cohen, W. R. Shapire and Y. Singer. 1999. Learning to order things. Journal of Artificial Intelligence Research, 10, pp. 243--270.
[2]
Collins, Michael. 2002. Discriminative training methods for Hidden Markov Models: theory and experiments with the perceptron algorithm. In: EMNLP. pp 1--8.
[3]
Crammer, K and Y. Singer. 2001. Pranking with ranking. In: NIPS.
[4]
Duda, Richard O, Hart, Peter E. and Stork, David G. 2001. Pattern classification. John Wiley & Sons, Inc.
[5]
Fletcher, R. 1987. Practical methods of optimization. John Wiley & Sons, Inc.
[6]
Freund, Yoav, Raj Iyer, Robert E. Schapire, and Yoram Singer. 1998. An efficient boosting algorithm for combining preferences. In ICML'98, pp. 170--178.
[7]
Gao, Jianfeng, Hao Yu, Peng Xu and Wei Yuan. 2005. Minimum sample risk methods for language modeling. To appear.
[8]
Gao, Jianfeng, Mu Li, Andi Wu and Changning Huang. 2004. A pragmatic approach to Chinese word segmentation. Tech-Report of Microsoft Research. MSR-TR-2004-123.
[9]
Gao, Jianfeng, Jian-Yun Nie, Guangyuan Wu and Guihong Cao. 2004. Dependence language model for information retrieval. In: SIGIR, pp. 170--177.
[10]
Gao, Jianfeng, Joshua Goodman and Jiangbo Miao. 2001. The use of clustering techniques for language model -- application to Asian language. Computational Linguistics and Chinese Language Processing. Vol. 6, No. 1, pp 27--60.
[11]
Harman, D. K. 1995. Overview of the fourth Text REtrieval Conference (TREC-4). In: TREC-4, pp 1--24.
[12]
Herbrich, R. T. Graepel and K. Obermayer. 2000. Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers, pp. 115--132. MIT Press, Cambridge, MA.
[13]
Joachims, T. 1999. Making large-scale SVM learning practical. In B. Scholkopt, C. Burges and A. Smola, editors, Advances in Kernel Methods -- Support Vector Learning. MIT Press, Cambridge, MA.
[14]
Joachims, T. 2002. Optimizing search engines using clickthrough data. In: SIGKDD, pp. 133--143.
[15]
Jones, K. S., S. Walker and S. Robertson. 1998. A probabilistic model of information retrieval: development and status. Technical Report TR-446, Cambridge University Computer Laboratory.
[16]
Juang, Biing-Hwang, Wu Chou and Chin-Hui Lee. 1997. Minimum classification error rate methods for speech recognition. IEEE Tran. Speech and Audio Processing. Vol. 5, No. 3. pp. 257--265.
[17]
Lafferty, John and Chengxiang Zhai. 2001. Document language models, query models, and risk minimization for information retrieval. In: SIGIR, pp. 111--119.
[18]
Miller, D. H., Leek, T. and Schwartz, R. 1999. A hidden Markov model information retrieval system. In: SIGIR'99, pp. 214--221.
[19]
Nallapati, R. 2004. Discriminative models for information retrieval. In: SIGIR, pp. 67--71.
[20]
Nallapati, R. and J. Allan. 2002. Capturing term dependencies using a language model based on sentence trees. In: CIKM, pp. 383--390.
[21]
Ng, A. N. and M. I. Jordan. 2002. On discriminative vs. generative classifiers: a comparison of logistic regression and naïve Bayes. In: NIPS, pp. 841--848.
[22]
Och, Franz. 2003. Minimum error rate training in statistical machine translation. In: ACL, pp. 160--167.
[23]
Ponte, J. and W. B. Croft. 1998. A language modeling approach to information retrieval, In: SIGIR'98, pp. 275--281.
[24]
Press, W. H., S. A. Teukolsky, W. T. Vetterling andB. P. Flannery. 1992. Numerical Recipes In C: The Art of Scientific Computing. New York: Cambridge Univ. Press.
[25]
Quirk, C., A. Merezes and C. Cherry. 2005. Dependency tree translation: syntactically informed phrasal SMT. To appear.
[26]
Robertson, S. E. and S. Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In: SIGIR, pp. 232--241.
[27]
Robertson, S. E. and Walker, S. 2000. Microsoft Cambridge at TREC-9: Filtering track. In: TREC-9, pp. 361--368.
[28]
Song, F. and Croft, B. 1999. A general language model for information retrieval. In: CIKM'99, pp. 316--321.
[29]
Vapnik, V. N. 1999. The nature of statistical learning theory. Springer-Verlag, New York.
[30]
Zhai, C., and J. Lafferty. 2002. Two-stage language models for information retrieval. In: SIGIR, pp. 49--56.

Cited By

View all
  • (2024)NeuralPMG: A Neural Polyphonic Music Generation System Based on Machine Learning AlgorithmsCognitive Computation10.1007/s12559-024-10280-616:5(2779-2802)Online publication date: 15-May-2024
  • (2020)Variations in Variational Autoencoders - A Comparative EvaluationIEEE Access10.1109/ACCESS.2020.30181518(153651-153670)Online publication date: 2020
  • (2020)Advanced Quality Control Models for Concrete AdmixturesJournal of Materials in Civil Engineering10.1061/(ASCE)MT.1943-5533.000302432:2(04019349)Online publication date: Feb-2020
  • Show More Cited By

Index Terms

  1. Linear discriminant model for information retrieval

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
    August 2005
    708 pages
    ISBN:1595930345
    DOI:10.1145/1076034
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 August 2005

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. discriminative training
    2. hidden Markov model
    3. language model
    4. optimization
    5. perceptron

    Qualifiers

    • Article

    Conference

    SIGIR05
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)6
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)NeuralPMG: A Neural Polyphonic Music Generation System Based on Machine Learning AlgorithmsCognitive Computation10.1007/s12559-024-10280-616:5(2779-2802)Online publication date: 15-May-2024
    • (2020)Variations in Variational Autoencoders - A Comparative EvaluationIEEE Access10.1109/ACCESS.2020.30181518(153651-153670)Online publication date: 2020
    • (2020)Advanced Quality Control Models for Concrete AdmixturesJournal of Materials in Civil Engineering10.1061/(ASCE)MT.1943-5533.000302432:2(04019349)Online publication date: Feb-2020
    • (2019)Learning to Rank in Entity Relationship GraphsINFORMS Journal on Computing10.1287/ijoc.2018.083731:4(671-688)Online publication date: 7-Jun-2019
    • (2019)Medical Image Denoising Using Multi-Resolution TransformsMeasurement10.1016/j.measurement.2019.01.001Online publication date: Jun-2019
    • (2019)A Graph-Based Approach for Semantic Medical SearchFrontier Computing10.1007/978-981-13-3648-5_10(89-98)Online publication date: 19-May-2019
    • (2016)Indexing by Latent Dirichlet Allocation and an Ensemble ModelJournal of the Association for Information Science and Technology10.1002/asi.2344467:7(1736-1750)Online publication date: 1-Jul-2016
    • (2015)Resources Sequencing Using Automatic Prerequisite--Outcome AnnotationACM Transactions on Intelligent Systems and Technology10.1145/25053496:1(1-30)Online publication date: 11-Mar-2015
    • (2015)Modeling query-document dependencies with topic language models for information retrievalInformation Sciences: an International Journal10.1016/j.ins.2015.03.056312:C(1-12)Online publication date: 10-Aug-2015
    • (2014)Learning to Rank for Information Retrieval and Natural Language Processing, Second EditionSynthesis Lectures on Human Language Technologies10.2200/S00607ED2V01Y201410HLT0267:3(1-121)Online publication date: 2-Oct-2014
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media