Towards the improvement of textual anatomy image classification using image local features

ABSTRACT

Image classification methods based on text utilize terms extracted from image annotations (image caption, image-related article, etc.) to achieve classification. For images involving different anatomical structures (chest, spine, etc.), however, the precision of pure textual classification often suffers from highly complex text contents (e.g. text terms extracted out of two MR abdomen images may be quite different from each other: terms from one image may concerns gastroenteritis while the other contains terms involving hysteromyoma). This paper tackles the anatomy image classification problem using a hybrid approach. First, a mutual information (MI) based filter is applied to select a set of terms with top MI scores for each anatomical class and help reduce the noise existing in the raw text. Second, local features extracted from the images are transformed as visual descriptors. Last, a hybrid scheme on the results from the textual and visual methods is applied to achieved further improvement of the classification results. Experiments show that this hybrid scheme improves the results over the sole textual or visual method on different anatomical class settings.

References

1. H. Alto, R. Rangayyantt, and J. Desautels. Content-based retrieval and analysis of mammographic masses. Journal Of Electronic Imaging, 14(2), 2005.
2. B. Andr, T. Vercauteren, A. Perchant, A. Buchner, M. Wallace, and N. Ayache. Introducing space and time in local feature-based endomicroscopic image retrieval. Medical Content-Based Retrieval for Clinical Decision Support, pages 18--30, 2010.
3. U. Avni, H. Greenspan, E. Konen, M. Sharon, and J. Goldberger. X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Transactions on Medical Imaging, 30(3), 2011.
4. A. Bosch, X. Munoz, A. Oliver, and J. Marti. Modeling and classifying breast tissue density in mammograms. In CVPR, volume 2, pages 1552--1558, 2006.
5. G. Csurka, C. R. Dance, L. Fan, J. Willamowski, and C. Bray. Visual categorization with bags of keypoints. In European Conference on Computer Vision (ECCV) Workshop on Statistical Learning in Computer Vision, 2004.
6. CS@UWM. Software and dataset. http://guangzhou.cs.uwm.edu/med, 2011.
7. T. Deselaers, A. Hegerath, D. Keysers, and H. Ney. Sparse patch histograms for object classification in cluttered images. In DAGM Symposium, pages 202--211, 2006.
8. J. Dy, C. Brodley, A. Kak, L. Broderick, and A. Aisen. Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans. Pattern Anal. Mach. Intell., 25(3):373--378, 2003.
9. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The weka data mining software: An update. SIGKDD Explorations, 11(1), 2009.
10. ImageCLEF. Image retrieval in clef: Medical retrieval task. http://www.imageclef.org/2010/medical, 2010.
11. J. Kalpathy-Cramer and W. Hersh. Automatic image modality based classification and annotation to improve medical image retrieval. In Proceedings of the 12th World Congress on Health (Medical) Informatics (MEDINFO), pages 1334--1338, 2007.
12. F.-F. Li and P. Perona. A bayesian hierarchical model for learning natural scene categories. In CVPR, volume 2, pages 524--531, 2005.
13. L. R. Long, S. Antani, D.-J. Lee, D. M. Krainak, and G. R. Thomas. Biomedical information from a national collection of spine x-rays: film to content-based retrieval. In Proceedings SPIE, pages 70--84, May 2003.
14. D. Lowe. Object recognition from local scale-invariant features. In ICCV, volume 2, 1999.
15. M. Lux and S. A. Chatzichristofis. Lire: lucene image retrieval: an extensible java cbir library. In Proceeding of the 16th ACM international conference on Multimedia, MM '08, pages 1085--1088, New York, NY, USA, 2008. ACM.
16. C. Manning, P. Raghavan, and H. Schuze. Introduction to Information Retrieval. Cambridge University Press, 2009.
17. C. D. Manning, P. Raghavan, and H. Schutze. Introduction to Information Retrieval. Cambridge University Press, July 2008.
18. H. Muller, J. Kalpathy-Cramer, I. Eggel, S. Bedrick, J. Reisetter, C. E. K. Jr., and W. Hersh. Overview of the CLEF 2010 medical image retrieval track. In Working notes of the Image-CLEF 2010 challenge, 2010.
19. H. Muller, C. Lovis, and A. Geissbuhler. The medgift project on medical image retrieval. Medical Imaging and Telemedicine, 2005.
20. C. Schmid and R. Mohr. Local grayvalue invariants for image retrieval. IEEE Transactions on Pattern Analysis & Machine Intelligence, 19(5), 1997.
21. J. Sivic and A. Zisserman. Video google: a text retrieval approach to object matching in videos. In ICCV, volume 2, pages 1470--1477, 2008.
22. T. Tatiana, O. Francesco, and C. Barbara. Discriminative cue integration for medical image annotation. Pattern Recognition Letters, 29(15):1996--2002, November 2008.
23. P. Tirilly, K. Lu, X. Mu, T. Zhao, and Y. Cao. On modality classification and its use in text-based image retrieval in medical databases. In Proceedings of the 9th International Workshop on Content-based Multimedia Indexing, 2011.
24. T. Tommasi, F. Orabona, and B. Caputo. An svm confidence-based approach to medical image annotation. In proceedings of the 9th CLEF workshop 2008, 2008.
25. B. van Ginneken, L. Hogeweg, and M. Prokop. Computer-aided diagnosis in chest radiography: Beyond nodules. European Journal of Radiology, 72(2):226--230, 2009.
26. I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Amsterdam, 2005.
27. Y. Y. Yao. Information-theoretic measures for knowledge discovery and data mining. Entropy Measures, Maximum Entropy Principle and Emerging Applications, pages 115--136, 2003.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision representations

        • Image representations

  • Symbolic and algebraic manipulation

    • Symbolic and algebraic algorithms

• Theory of computation

  • Design and analysis of algorithms