Geometric consistency checks for kNN based image classification relying on local features

ABSTRACT

Applications of image content recognition, as for instance landmark recognition, can be obtained by using techniques of kNN classifications based on the use of local image features, such as SIFT or SURF. Quality of image classification can be improved by defining geometric consistency check rules based on space transformations of the scene depicted in images. However, this prevents the use of state of the art access methods for similarity searching and sequential scan of the images in the training sets has to be executed in order to perform classification. In this paper we propose a technique that allows one to use access methods for similarity searching, such as those exploiting metric space properties, in order to perform kNN classification with geometric consistency checks. We will see that the proposed approach, in addition to offer an obvious efficiency improvement, surprisingly offers also an improvement of the effectiveness of the classification.

References

1. Google goggles. http://www.google.com/mobile/goggles/. last accessed on 30-March-2010.
2. Pisa landmarks dataset. http://www.fabriziofalchi.it/pisaDataset/. last accessed on 3-March-2011.
3. SIFT keypoint detector. http://people.cs.ubc.ca/~lowe/. last accessed on 3-March-2011.
4. SURF detector. http://www.vision.ee.ethz.ch/~surf/. last accessed on 3-March-2011.
5. G. Amato and F. Falchi. kNN based image classification relying on local feature similarity. In SISAP '10: Proceedings of the Third International Conference on SImilarity Search and APplications, pages 101--108, New York, NY, USA, 2010. ACM.
6. G. Amato and F. Falchi. Local feature based image similarity functions for kNN classification. In Proceedings of the 3rd International Conference on Agents and Artificial Intelligence (ICAART 2011), pages 157--166. SciTePress, 2011. Vol. 1.
7. G. Amato, F. Falchi, and P. Bolettieri. Recognizing landmarks using automated classification techniques: an evaluation of various visual features. In in Proceeding of The Second International Conference on Advances in Multimedia (MMEDIA 2010), pages 78--83. IEEE Computer Society, 2010.
8. H. Bay, T. Tuytelaars, and L. V. Gool. Surf: Speeded up robust features. In In ECCV, pages 404--417, 2006.
9. O. Boiman, E. Shechtman, and M. Irani. In defense of nearest-neighbor based image classification. In CVPR. IEEE Computer Society, 2008.
10. T. Chen, K. Wu, K.-H. Yap, Z. Li, and F. S. Tsai. A survey on mobile landmark recognition for information retrieval. In MDM '09, pages 625--630. IEEE Computer Society, 2009.
11. S. Dudani. The distance-weighted k-nearest-neighbour rule. IEEE Transactions on Systems, Man and Cybernetics, SMC-6(4):325--327, 1975.
12. T. Fagni, F. Falchi, and F. Sebastiani. Image classification via adaptive ensembles of descriptor-specific classifiers. Pattern Recognition and Image Analysis, 20:21--28, 2010.
13. M. A. Fischler and R. C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381--395, 1981.
14. R. I. Hartley. In defence of the 8-point algorithm. In Proceedings of the Fifth International Conference on Computer Vision, ICCV '95, pages 1064--, Washington, DC, USA, 1995. IEEE Computer Society.
15. T. Homola, V. Dohnal, and P. Zezula. Proximity-based order-respecting intersection for searching in image databases. In In Proceedings of the 8th International Workshop on Adaptive Multimedia Retrieval (AMR 2010), 2010.
16. T. Homola, V. Dohnal, and P. Zezula. Sub-image searching through intersection of local descriptors. In Proceedings of the Third International Conference on SImilarity Search and APplications, SISAP '10, pages 127--128, New York, NY, USA, 2010. ACM.
17. D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91--110, 2004.
18. P. Serdyukov, V. Murdock, and R. van Zwol. Placing flickr photos on a map. In SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 484--491, New York, NY, USA, 2009. ACM.
19. J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. In Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2, ICCV '03, pages 1470--, Washington, DC, USA, 2003. IEEE Computer Society.
20. P. Tirilly, V. Claveau, and P. Gros. Distances and weighting schemes for bag of visual words image retrieval. In Proceedings of the international conference on Multimedia information retrieval, MIR '10, pages 323--332, New York, NY, USA, 2010. ACM.
21. P. Zezula, G. Amato, V. Dohnal, and M. Batko. Similarity Search: The Metric Space Approach, volume 32 of Advances in Database Systems. Springer-Verlag, 2006.
22. Y. Zheng, M. Z. 0003, Y. Song, H. Adam, U. Buddemeier, A. Bissacco, F. Brucher, T.-S. Chua, and H. Neven. Tour the world: Building a web-scale landmark recognition engine. In CVPR, pages 1085--1092. IEEE, 2009.

Index Terms

• Information systems

  • Information retrieval

    • Document representation