Selection and Combination of Unsupervised Learning Methods for Image Retrieval

ABSTRACT

The evolution of technologies to store and share images has made imperative the need for methods to index and retrieve multimedia information based on visual content. The CBIR (Content-Based Image Retrieval) systems are the main solution in this scenario. Originally, these systems were solely based on the use of low-level visual features, but evolved through the years in order to incorporate various supervised learning techniques. More recently, unsupervised learning methods have been showing promising results for improving the effectiveness of retrieval results. However, given the development of different methods, a challenging task consists in to exploit the advantages of diverse approaches. As different methods present distinct results even for the same dataset and set of features, a promising approach is to combine these methods. In this work, a framework is proposed aiming at selecting the best combination of methods in a given scenario, using different strategies based on effectiveness and correlation measures. Regarding the experimental evaluation, six distinct unsupervised learning methods and two different datasets were used. The results as a whole are promising and also reveal good perspectives for future works.

References

1. R. Datta, D. Joshi, J. Li, and J. Z. Wang, "Image retrieval: Ideas, influences, and trends of the new age," ACM Comput. Surv., vol. 40, no. 2, pp. 5:1--5:60, May 2008.
2. R. D. S. Torres and A. X. Falcão, "Content-based image retrieval: Theory and applications," Revista de Informática Teórica e Aplicada, vol. 13, pp. 161--185, 2006.
3. J. Huang, S. R. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih, "Image indexing using color correlograms," in CVPR, 1997, pp. 762--768.
4. H. Ling and D. W. Jacobs, "Shape classification using the inner-distance," IEEE TPAMI, vol. 29, no. 2, pp. 286--299, 2007.
5. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-up robust features (surf)," Computer vision and image understanding, vol. 110, no. 3, pp. 346--359, 2008.
6. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding," arXiv preprint arXiv:1408.5093, 2014.
7. S. C. Hoi, W. Liu, and S.-F. Chang, "Semi-supervised distance metric learning for collaborative image retrieval and clustering," ACM Transactions on Multimedia Computing and Communication Applications, vol. 6, no. 3, pp. 18:1--18:26, August 2010.
8. P. Kontschieder, M. Donoser, and H. Bischof, "Beyond pairwise shape similarity analysis," in ACCV, 2009, pp. 655--666.
9. D. C. G. Pedronette and R. da S. Torres, "Exploiting contextual information for image re-ranking and rank aggregation," International Journal of Multimedia Information Retrieval, vol. 1, no. 2, pp. 115--128, 2012.
10. L. P. Valem, D. C. G. Pedronette, R. d. S. Torres, E. Borin, and J. Almeida, "Effective, efficient, and scalable unsupervised distance learning in image retrieval tasks," ICMR, 2015.
11. F. F. Faria, A. Veloso, H. M. Almeida, E. Valle, R. d. S. Torres, M. A. Gonçalves, and W. Meira, Jr., "Learning to rank for content-based image retrieval," in Proceedings of the International Conference on Multimedia Information Retrieval, ser. MIR '10. New York, NY, USA: ACM, 2010, pp. 285--294.
12. H. J. Escalante, C. A. Hérnadez, L. E. Sucar, and M. Montes, "Late fusion of heterogeneous methods for multimedia image retrieval," in Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval, ser. MIR '08. New York, NY, USA: ACM, 2008, pp. 172--179.
13. J. A. Vargas Muñoz, R. da Silva Torres, and M. A. Gonçalves, "A soft computing approach for learning to aggregate rankings," in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, ser. CIKM '15. New York, NY, USA: ACM, 2015, pp. 83--92.
14. D. C. G. Pedronette and R. da S. Torres, "Exploiting clustering approaches for image re-ranking," Journal of Visual Languages and Computing, vol. 22, no. 6, pp. 453--466, 2011.
15. X. Yang and L. J. Latecki, "Affinity learning on a tensor product graph with applications to shape and image retrieval," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR'2011), 2011, pp. 2369--2376.
16. X. Yang, S. Koknar-Tezel, and L. J. Latecki, "Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval." in CVPR, 2009, pp. 357--364.
17. X. Yang, X. Bai, L. J. Latecki, and Z. Tu, "Improving shape retrieval by learning graph transduction," in ECCV, vol. 4, 2008, pp. 788--801.
18. G. Park, Y. Baek, and H.-K. Lee, "Re-ranking algorithm using post-retrieval clustering for content-based image retrieval," Information Processing and Management, vol. 41, no. 2, pp. 177--194, 2005.
19. W. Voravuthikunchai, B. Crémilleux, and F. Jurie, "Image re-ranking based on statistics of frequent patterns," in Proceedings of International Conference on Multimedia Retrieval, ser. ICMR '14. New York, NY, USA: ACM, 2014, pp. 129:129--129:136.
20. D. C. G. Pedronette and R. da S. Torres, "Image re-ranking and rank aggregation based on similarity of ranked lists," in Computer Analysis of Images and Patterns (CAIP'2011), vol. 6854, 2011, pp. 369--376.
21. L. P. Valem and D. C. G. Pedronette, "Unsupervised similarity learning through cartesian product of ranking references for image retrieval tasks," Conference on Graphics, Image and Patterns (SIBGRAPI), 2016.
22. D. C. G. Pedronette, O. A. Penatti, and R. da S. Torres, "Unsupervised manifold learning using reciprocal knn graphs in image re-ranking and rank aggregation tasks," Image and Vision Computing, vol. 32, no. 2, pp. 120--130, 2014.
23. D. C. G. Pedronette, J. Almeida, and R. da S. Torres, "A graph-based ranked-list model for unsupervised distance learning on shape retrieval," Pattern Recognition Letters, vol. 83, Part 3, pp. 357--367, 2016, efficient Shape Representation, Matching, Ranking, and its Applications.
24. D. C. G. Pedronette and R. da S. Torres, "Image re-ranking and rank aggregation based on similarity of ranked lists," Pattern Recognition, vol. 46, no. 8, pp. 2350--2360, 2013.
25. C. Y. Okada, D. C. G. a. Pedronette, and R. da S. Torres, "Unsupervised distance learning by rank correlation measures for image retrieval," in Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, ser. ICMR '15. New York, NY, USA: ACM, 2015, pp. 331--338.
26. L. P. Valem and D. C. G. Pedronette, "An unsupervised distance learning framework for multimedia retrieval," ser. ICMR '17, 2017.
27. E. A. Fox and J. A. Shaw, "Combination of multiple searches," in The Second Text REtrieval Conference (TREC-2), ser. NIST Special Publication, vol. 500-215. NIST, 1994, pp. 243--252.
28. R. Fagin, R. Kumar, M. Mahdian, D. Sivakumar, and E. Vee, "Comparing and aggregating rankings with ties," in 23th ACM SIGMOD Symposium on Principles of Database Systems (PODS'04), 2004, pp. 47--58.
29. Y.-T. Liu, T.-Y. Liu, T. Qin, Z.-M. Ma, and H. Li, "Supervised rank aggregation," in International Conference on World Wide Web (WWW'2007), 2007, pp. 481--490.
30. D. C. G. Pedronette and R. d. S. Torres, "Combining re-ranking and rank aggregation methods for image retrieval," Multimedia Tools and Applications, vol. 75, no. 15, pp. 9121--9144, 2016.
31. P. Brodatz, Textures: A Photographic Album for Artists and Designers. Dover, 1966.
32. B. Tao and B. W. Dickinson, "Texture recognition and image retrieval using gradient indexing," JVCIR, vol. 11, no. 3, pp. 327--342, 2000.
33. L. J. Latecki, R. Lakmper, and U. Eckhardt, "Shape descriptors for non-rigid shapes with a single closed contour," in CVPR, 2000, pp. 424--429.
34. H. Ling, X. Yang, and L. J. Latecki, "Balancing deformability and discriminability for shape matching," in ECCV, vol. 3, 2010, pp. 411--424.

Index Terms

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval