skip to main content
article

Semantics and feature discovery via confidence-based ensemble

Published: 01 May 2005 Publication History

Abstract

Providing accurate and scalable solutions to map low-level perceptual features to high-level semantics is essential for multimedia information organization and retrieval. In this paper, we propose a confidence-based dynamic ensemble (CDE) to overcome the shortcomings of the traditional static classifiers. In contrast to the traditional models, CDE can make dynamic adjustments to accommodate new semantics, to assist the discovery of useful low-level features, and to improve class-prediction accuracy. We depict two key components of CDE: a multi-level function that asserts class-prediction confidence, and the dynamic ensemble method based upon the confidence function. Through theoretical analysis and empirical study, we demonstrate that CDE is effective in annotating large-scale, real-world image datasets.

References

[1]
Benitez, A. B. and Chang, S.-F. 2002. Semantic knowledge construction from annotated image collection. In Proceedings of the IEEE International Conference on Multimedia. IEEE Computer Society Press, Los Alamitos, Calif.
[2]
Bouchaffra, D., Govindaraju, V., and Srihari, S. N. 1999. A methodology for mapping scores to probabilities. IEEE Trans. Patt. Anal. Mach. Intel. 21, 9, 923--927.
[3]
Breiman, L. 1996. Bagging predicators. Mach. Learn. 24, 2, 123--140.
[4]
Chang, E., Goh, K., Sychay, G., and Wu, G. 2003. Content-based soft annotation for multimodal image retrieval using bayes point machines. IEEE Trans. Circ. Syst. Video Tech. (Special Issue on Conceptual and Dynamical Aspects of Multimedia Content Description) 13, 1, 26--38.
[5]
Chang, S.-F., Chen, W., and Sundaram, H. 1998. Semantic visual templates: Linking visual features to semantics. In Proceedings of the IEEE International Conference on Image Processing, IEEE Computer Society Press, Los Alamitos, Calif. 531--535.
[6]
Chow, C. K. 1970. On optimum recognition error and reject tradeoff. IEEE Trans. Inf. Theory 16, 1, 41--46.
[7]
Collobert, R. and Bengio, S. 2001. SVMTorch: Support vector machines for large-scale regression problems. J. Mach. Learn. Res. 1, 143--160.
[8]
Fan, J., Gao, Y., and Luo, H. 2004. Multi-level annotation of natural scenes using dominant image components and semantic concepts. In Proceedings of the ACM International Conference on Multimedia. ACM, New York, 540 --547.
[9]
Goh, K. and Chang, E. 2004. One, two class SVMs for multi-class image annotation. UCSB Technical Report.
[10]
Goh, K., Chang, E., and Cheng, K. T. 2001. SVM binary classifier ensembles for image classification. In Proceedings of the ACM CIKM. ACM, New York, 395--402.
[11]
Hastie, T. and Tibshirani, R. 1998. Classification by pairwise coupling. Adv. Neural Inf. Proc. Syst. 10, 507--513.
[12]
He, X., Ma, W.-Y., King, O., Li, M., and Zhang, H. 2002. Learning and inferring a semantic space from user's relevance feedback for image retrieval. In Proceedings of the ACM Multimedia. ACM, New York, 343--347.
[13]
Ho, T. K., Hull, J., and Srihari, S. 1994. Decision combination in multiple classifier systems. IEEE Trans. Patt. Anal. Mach. Intel. 16, 1, 66--75.
[14]
Li, B., Goh, K., and Chang, E. 2003. Confidence-based dynamic ensemble for image annotation and semantics discovery. In Proceedings of the ACM International Conference on Multimedia. ACM, New York, 195--206.
[15]
Li, J. and Wang, J. Z. 2003. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Patt. Anal. Mach. Intell. 25, 9, 1075--1088.
[16]
Naphade, M. R., Lin, C.-Y., Smith, J., Tseng, B., and Basu, S. 2002. Learning to annotate video databases. SPIE Electronic Imaging 2002---Storage and Retrieval for Media Databases 4676, 264--275.
[17]
Platt, J. 1999. Probabilistic outputs for SVMs and comparisons to regularized likelihood methods. Adv. Large Margin Class. 61--74.
[18]
Platt, J., Cristianini, N., and Shawe-Taylor, J. 2000. Large margin DAGs for multiclass classification. Adv. Neural Inf. Proc. Syst. 12, 547--553.
[19]
Poddar, P. and Rao, P. 1993. Hierarchical ensemble of neural networks. In Proceedings of the International Conference on Neural Networks 1, 287--292.
[20]
Rodriguez, C., Muguerza, J., Navarro, M., Zarate, A., Martin, J., and Perez, J. 1998. A two-stage classifier for broken and blurred digits in forms. In Proceedings of the International Conference on Pattern Recognition 2, 1101--1105.
[21]
Schapire, R. F. and Singer, Y. 1998. Improved boosting algorithms using confidence-rated predictions. In Proceedings of the 11th Annual Conference on Computational Learning Theory. 80--91.
[22]
Shen, H. T., Ooi, B. C., and Tan, K. L. 2000. Giving meanings to www images. In Proceedings of ACM Multimedia, ACM, New York, 39--48.
[23]
Srihari, R., Zhang, Z., and Rao, A. 2000. Intelligent indexing and semantic retrieval of multimodal documents. Inf. Retriev. 2, 245--275.
[24]
Tong, S. and Chang, E. 2001. Support vector machine active learning for image retrieval. In Proceedings of the ACM International Conference on Multimedia, ACM, New York, 107--118.
[25]
Vapnik, V. 1982. Estimation of Dependences Based on Empirical Data. Springer-Verlag, New York.
[26]
Vapnik, V. 1998. Statistical Learning Theory. Wiley, New York.
[27]
Wang, J., Li, J., and Wiederhold, G. 2001. Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Trans. Patt. Anal. Mach. Intel. 23, 9, 947--963.
[28]
Wenyin, L., Dumais, S., Sun, Y., Zhang, H., Czerwinski, M., and Field, B. 2001. Semi-automatic image annotation. In Proceedings of Interact 2001: Conference on Human-Computer Interaction, 326--333.
[29]
Wu, G. and Chang, E. 2003. Adaptive feature-space conformal transformation for learning imbalanced data. In Proceedings of the International Conference on Machine Learning, 816--823.
[30]
Wu, H., Li, M., Zhang, H., and Ma, W.-Y. 2002. Improving image retrieval with semantic classification using relevance feedback. Vis. Datab. 327--339.

Cited By

View all
  • (2023)Smart Multimedia Information RetrievalAnalytics10.3390/analytics20100112:1(198-224)Online publication date: 20-Feb-2023
  • (2021)AI-Based Semantic Multimedia Indexing and Retrieval for Social Media on SmartphonesInformation10.3390/info1201004312:1(43)Online publication date: 19-Jan-2021
  • (2020)Towards AI-based Semantic Multimedia Indexing and Retrieval for Social Media on Smartphones2020 15th International Workshop on Semantic and Social Media Adaptation and Personalization (SMA10.1109/SMAP49528.2020.9248445(1-9)Online publication date: 29-Oct-2020
  • Show More Cited By

Index Terms

  1. Semantics and feature discovery via confidence-based ensemble

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 1, Issue 2
    May 2005
    84 pages
    ISSN:1551-6857
    EISSN:1551-6865
    DOI:10.1145/1062253
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 May 2005
    Published in TOMM Volume 1, Issue 2

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Classification confidence
    2. image annotation
    3. semantics discovery

    Qualifiers

    • Article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Smart Multimedia Information RetrievalAnalytics10.3390/analytics20100112:1(198-224)Online publication date: 20-Feb-2023
    • (2021)AI-Based Semantic Multimedia Indexing and Retrieval for Social Media on SmartphonesInformation10.3390/info1201004312:1(43)Online publication date: 19-Jan-2021
    • (2020)Towards AI-based Semantic Multimedia Indexing and Retrieval for Social Media on Smartphones2020 15th International Workshop on Semantic and Social Media Adaptation and Personalization (SMA10.1109/SMAP49528.2020.9248445(1-9)Online publication date: 29-Oct-2020
    • (2012)Cross-media topic analysis and information retrieval2012 9th International Conference on Fuzzy Systems and Knowledge Discovery10.1109/FSKD.2012.6233856(1211-1215)Online publication date: May-2012
    • (2012)Study of cross-media topic analysis based on visual topic model2012 24th Chinese Control and Decision Conference (CCDC)10.1109/CCDC.2012.6244553(3467-3470)Online publication date: May-2012
    • (2012)Study of cross-media topic analysis and application2012 24th Chinese Control and Decision Conference (CCDC)10.1109/CCDC.2012.6244269(1681-1684)Online publication date: May-2012
    • (2011)Multimedia news exploration and retrieval by integrating keywords, relations and visual featuresMultimedia Tools and Applications10.1007/s11042-010-0639-351:2(625-648)Online publication date: 1-Jan-2011
    • (2011)A confidence paradigm for classification systemsNaval Research Logistics (NRL)10.1002/nav.2042658:3(236-254)Online publication date: 11-Mar-2011
    • (2009)Kernel Discriminant Analysis Using Triangular Kernel for Semantic Scene ClassificationProceedings of the 2009 Seventh International Workshop on Content-Based Multimedia Indexing10.1109/CBMI.2009.47(1-6)Online publication date: 3-Jun-2009
    • (2008)Multimodal search for effective image retrieval2008 15th International Conference on Systems, Signals and Image Processing10.1109/IWSSIP.2008.4604410(233-236)Online publication date: Jun-2008
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media