skip to main content
10.1145/1645953.1646017acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Probabilistic models for topic learning from images and captions in online biomedical literatures

Published: 02 November 2009 Publication History

Abstract

Biomedical images and captions are one of the major sources of information in online biomedical publications. They often contain the most important results to be reported, and provide rich information about the main themes in published papers. In the data mining and information retrieval community, there has been much effort on using text mining and language modeling algorithms to extract knowledge from the text content of online biomedical publications; however, the problem of knowledge extraction from biomedical images and captions has not been fully studied yet. In this paper, a hierarchical probabilistic topic model with background distribution (HPB) is introduced to uncover the latent semantic topics from the co-occurrence patterns of caption words, visual words and biomedical concepts. With downloaded biomedical figures, restricted captions are extracted with regard to each individual image panel. During the indexing stage, the 'bag-of-words' representation of captions is supplemented by an ontology-based concept indexing to alleviate the synonym and polysemy problems. As the visual counterpart of text words, the visual words are extracted and indexed from corresponding image panels. The model is estimated via collapsed Gibbs sampling algorithm. We compare the performance of our model with the extension of the Correspondence LDA (Corr-LDA) model under the same biomedical image annotation scenario using cross-validation. Experimental results demonstrate that our model is able to accurately extract latent patterns from complicated biomedical image-caption pairs and facilitate knowledge organization and understanding in online biomedical literatures.

References

[1]
T. Hofmann. Probabilistic Latent Semantic Indexing. Proceedings of the Twenty-Second Annual International SIGIR Conference, 1999.
[2]
A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, Content-based image rerieval at the end of the early years, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349--1380, 2000.
[3]
T. L. Griffiths, M. Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America, 101:5228--5235, 2004.
[4]
J. Yang, Y. G. Jiang, A. G. Hauptmann, C. W. Ngo, Evaluating Bag-of-Visual-Words Representations in Scene Classification. ACM SIGMM Int'l Workshop on Multimedia Information Retrieval (MIR'07), Augsburg, Germany, Sep. 2007.
[5]
W. W. Cohen, R. Wang, and R. F. Murphy. Understanding captions in biological publications. ACM KDD, 2005.
[6]
Sivic, J., Zisserman, A.: Video Google: A Text Retrieval Approach to Object Matching in Videos. International Conference on Computer Vision. (2003) 1470--1477
[7]
D. Blei and M. Jordan, Modeling Annotated Data, Proc. ACM SIGIR Conf. Research and Development in Information Retrieval, 2003.
[8]
J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid, Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study. International Journal of Computer Vision, vol. 73, no. 2, June 2007, pp. 213--238
[9]
T. Kadir and M. Brady. Scale, Saliency and Image Description. International Journal of Computer Vision. 45 (2):83--105, November 2001
[10]
O. Yakhnenko, V. Honavar, Annotating images and image objects using a hierarchical Dirichlet process model, proceedings of the 9th International Workshop on Multimedia Data Mining, pp. 1--7, 2008.
[11]
K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local descriptors. In IEEE Conference on Computer Vision and Pattern Recognition, vol.2 pp. 257--264, 2003
[12]
Zhou, X., Zhang, X., and Hu, X., "MaxMatcher: Biological Concept Extraction Using Approximate Dictionary Lookup," the 9th biennial The Pacific Rim International Conference on Artificial Intelligence (PRICAI 2006), Aug 9--11, 2006, Guilin, Guangxi, China, Page 1145--1149
[13]
Lowe, D. Distinctive Image Features from Scale-Invariant Key Points. International Journal of Computer Vision, 60(2): 91---110, 2004.
[14]
Van Rijsbergen, C.J., Information Retrieval, Butterworths, 1975.
[15]
Humphreys B. and Lindberg D. -- The UMLS project: making the conceptual connection between users and the information they need -- Bulletin of the Medical Library Association 81(2): 170, 1993.
[16]
Yu-Gang Jiang, Chong-Wah Ngo, Jun Yang: Towards optimal bag-of-features for object categorization and semantic video retrieval. CIVR 2007: 494--501
[17]
L. Fei-Fei and P. Perona, A Bayesian hierarchical model for learning natural scene categories. In CVPR, volume 2, pages. 524--531, 2005
[18]
X. Wang and E. Grimson, Spatial Latent Dirichlet Allocation, in Proceedings of Neural Information Processing Systems Conference (NIPS) 2007
[19]
D. Blei, A. Ng. and M. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993--1022,2003.
[20]
Newman, D., Chemudugunta, C., Smyth, P., Steyvers, M.: Statistical entity-topic models. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, Pennsylvania, USA, pp. 680--686 (2006)

Cited By

View all
  • (2020)Integrating social annotations into topic models for personalized document retrievalSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-019-03998-124:3(1707-1716)Online publication date: 1-Feb-2020
  • (2015)Cross-Platform Multi-Modal Topic Modeling for Personalized Inter-Platform RecommendationIEEE Transactions on Multimedia10.1109/TMM.2015.246322617:10(1787-1801)Online publication date: 1-Oct-2015
  • (2013)On handling textual errors in latent document modelingProceedings of the 22nd ACM international conference on Information & Knowledge Management10.1145/2505515.2505555(2089-2098)Online publication date: 27-Oct-2013
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management
November 2009
2162 pages
ISBN:9781605585123
DOI:10.1145/1645953
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Gibbs sampling
  2. automatic image annotation
  3. bioinformatics
  4. probabilistic models
  5. topic learning
  6. visual words

Qualifiers

  • Research-article

Conference

CIKM '09
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Integrating social annotations into topic models for personalized document retrievalSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-019-03998-124:3(1707-1716)Online publication date: 1-Feb-2020
  • (2015)Cross-Platform Multi-Modal Topic Modeling for Personalized Inter-Platform RecommendationIEEE Transactions on Multimedia10.1109/TMM.2015.246322617:10(1787-1801)Online publication date: 1-Oct-2015
  • (2013)On handling textual errors in latent document modelingProceedings of the 22nd ACM international conference on Information & Knowledge Management10.1145/2505515.2505555(2089-2098)Online publication date: 27-Oct-2013
  • (2012)Robust segmentation of biomedical figures for image-based document retrievalProceedings of the 2012 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM.2012.6392706(1-6)Online publication date: 4-Oct-2012
  • (2011)Towards noise-resilient document modelingProceedings of the 20th ACM international conference on Information and knowledge management10.1145/2063576.2063962(2345-2348)Online publication date: 24-Oct-2011
  • (2011)Perspective hierarchical dirichlet process for user-tagged image modelingProceedings of the 20th ACM international conference on Information and knowledge management10.1145/2063576.2063770(1341-1346)Online publication date: 24-Oct-2011
  • (2010)A probabilistic topic-connection model for automatic image annotationProceedings of the 19th ACM international conference on Information and knowledge management10.1145/1871437.1871552(899-908)Online publication date: 26-Oct-2010
  • (2010)The topic-perspective model for social tagging systemsProceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining10.1145/1835804.1835891(683-692)Online publication date: 25-Jul-2010

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media