Deep Supervised Quantization by Self-Organizing Map

ABSTRACT

Approximate Nearest Neighbour (ANN) search is an important research topic in multimedia and computer vision fields. In this paper, we propose a new deep supervised quantization method by Self-Organizing Map (SOM) to address this problem. Our method integrates the Convolutional Neural Networks (CNN) and Self-Organizing Map into a unified deep architecture. The overall training objective includes supervised quantization loss and classification loss. With the supervised quantization loss, we minimize the differences on the maps between similar image pairs, and maximize the differences on the maps between dissimilar image pairs. By optimization, the deep architecture can simultaneously extract deep features and quantize the features into the suitable nodes in the Self-Organizing Map. The experiments on several public standard datasets prove the superiority of our approach over the existing ANN search methods. Besides, as a byproduct, our deep architecture can be directly applied to classification task and visualization with little modification, and promising performances are demonstrated on these tasks in the experiments.

References

1. Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014).
2. Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: a real-world web image database from National University of Singapore ACM International Conference on Image and Video Retrieval. 48.
3. Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 12 (2013), 2916--2929.
4. Yen-Chang Hsu and Zsolt Kira. 2015. Neural network-based clustering using pairwise constraints. arXiv preprint arXiv:1511.06321 (2015).
5. Yen-Chang Hsu, Zhaoyang Lv, and Zsolt Kira. 2016. Deep Image Category Discovery using a Transferred Similarity Function. arXiv preprint arXiv:1612.01253 (2016).
6. Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, 1 (2011), 117--128.
7. Teuvo Kohonen. 1982. Self-organized formation of topologically correct feature maps. Biological Cybernetics Vol. 43, 1 (1982), 59--69.
8. Teuvo Kohonen and Timo Honkela. 2007. Kohonen network. Scholarpedia, Vol. 2, 1 (2007), 1568.
9. Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical report, University of Toronto (2009).
10. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks Advances in Neural Information Processing Systems. 1097--1105.
11. Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks IEEE Conference on Computer Vision and Pattern Recognition. 3270--3278.
12. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE Vol. 86, 11 (1998), 2278--2324.
13. Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2015. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015).
14. Renjie Liao, Alex Schwing, Richard Zemel, and Raquel Urtasun. 2016. Learning deep parsimonious representations. In Advances in Neural Information Processing Systems. 5076--5084.
15. Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. IEEE Conference on Computer Vision and Pattern Recognition. 2064--2072.
16. Zhen Liu, Houqiang Li, Wengang Zhou, Ruizhen Zhao, and Qi Tian. 2014. Contextual hashing for large-scale image search. IEEE Transactions on Image Processing Vol. 23, 4 (2014), 1606--1614.
17. Mohammad Norouzi and David J Fleet. 2013. Cartesian k-means IEEE Conference on Computer Vision and Pattern Recognition. 3017--3024.
18. Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. 2013. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013).
19. Andrea Vedaldi and Karel Lenc. 2015. Matconvnet: Convolutional neural networks for matlab ACM International Conference on Multimedia. ACM, 689--692.
20. Jianfeng Wang, Jingdong Wang, Nenghai Yu, and Shipeng Li. 2013. Order preserving hashing for approximate nearest neighbor search ACM International Conference on Multimedia. 133--142.
21. Min Wang, Wengang Zhou, Qi Tian, Zhengjun Zha, and Houqiang Li. 2016 b. Linear Distance Preserving Pseudo-Supervised and Unsupervised Hashing Proceedings of the 2016 ACM on Multimedia Conference. ACM, 1257--1266.
22. Xiaojuan Wang, Ting Zhang, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. 2016 a. Supervised quantization for similarity search. In IEEE Conference on Computer Vision and Pattern Recognition. 2018--2026.
23. Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral hashing Advances in Neural Information Processing Systems. 1753--1760.
24. Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised Hashing for Image Retrieval via Image Representation Learning. Association for the Advancement of Artificial Intelligence, Vol. Vol. 1. 2.
25. Lei Zhang, Yongdong Zhang, Jinhui Tang, Xiaoguang Gu, Jintao Li, and Qi Tian. 2013. Topology preserving hashing for similarity search. ACM International Conference on Multimedia. 123--132.
26. Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, and Lei Zhang. 2015. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing Vol. 24, 12 (2015), 4766--4779.
27. Ting Zhang, Chao Du, and Jingdong Wang. 2014. Composite Quantization for Approximate Nearest Neighbor Search International Conference on Machine Learning. 838--846.
28. Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval IEEE Conference on Computer Vision and Pattern Recognition. 1556--1564.
29. Wengang Zhou, Houqiang Li, Richang Hong, Yijuan Lu, and Qi Tian. 2015. BSIFT: Toward data-independent codebook for large scale image search. IEEE Transactions on Image Processing Vol. 24, 3 (2015), 967--979.
30. Wengang Zhou, Yijuan Lu, Houqiang Li, and Qi Tian. 2012. Scalar quantization for large scale image search. ACM International Conference on Multimedia. 169--178.
31. Wengang Zhou, Ming Yang, Houqiang Li, Xiaoyu Wang, Yuanqing Lin, and Qi Tian. 2014. Towards codebook-free: Scalable cascaded hashing for mobile image search. IEEE Transactions on Multimedia Vol. 16, 3 (2014), 601--611.
32. Wengang Zhou, Ming Yang, Xiaoyu Wang, Houqiang Li, Yuanqing Lin, and Qi Tian. 2016. Scalable feature matching by dual cascaded scalar quantization for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 38, 1 (2016), 159--171.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

• Information systems

  • Information retrieval

    • Retrieval models and ranking

      • Top-k retrieval in databases

    • Retrieval tasks and goals

      • Clustering and classification