Learning to photograph

ABSTRACT

In this paper, we propose an intelligent photography system, which automatically and professionally generates/recommends user-favorite photo(s) from a wide view or a continuous view sequence. This task is quite challenging given that the evaluation of photo quality is under-determined and usually subjective. Motivated by the recent prevalence of online media, we present a solution y mining the underlying knowledge and experience of the photographers from massively crawled professional photos (about 100,000 images, which are highly ranked by users) of those popular photo sharing websites, e.g. Flickr.com. Generally far contexts are critical in characterizing the composition rules for professional photos, and thus we present a method called omni-range context modeling to learn the patch/object spatial correlation distribution for the concurrent patch/object pair of arbitrary distance. The learned photo omni-range context priors then serve as rules to guide the composition of professional photos. When a wide view is fed into the system, these priors are utilized together with other cues (e.g., placements of faces at different poses, patch number, etc) to form a posterior probability formulation for professional sub-view finding. Moreover, this system can function as intelligent professionalview guider based on real-time view quality assessment and the embedded compass (for recording capture direction). Beyond the salient areas targeted by most existing view recommendation algorithms, the proposed system targets at professional photo composition. Qualitative experiments as well as comprehensive user studies well demonstrate the validity and efficiency of the proposed omnirange context learning method as well as the automatic view finding framework.

References

1. http://www.portraitprofessional.com/.
2. S. Avidan and A. Shamir. Seam carving for content-aware image resizing. ACM Transactions on Graphics, 26(3):1--9, 2007.
3. S. Banerjee and B. L. Evans. In-camera automation of photographic composition rules. IEEE Transactions on Image Processing, 16:1807--1820, 2007.
4. S. Birchfield and S. Rangarajan. Spatiograms versus histograms for region-based tracking. In CVPR, volume 2, pages 1158--1163, 2005.
5. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, pages 886--893, 2005.
6. C. Dance, J. Willamowski, L. Fan, C. Bray, and G. Csurka. Visual categorization with bags of keypoints. In ECCV International Workshop on Statistical Learning in Computer Vision, 2004.
7. A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1--38, 1977.
8. P. Felzenszwalb and D. Huttenlocher. Efficient graph-based image segmentation. International Journal of Computer Vision, 59(2):167--181, 2004.
9. W. Gilks, S. Richardson, and D. Spiegelhalter. Markov Chain Monte Carlo in Practice: Interdisciplinary Statistics. Chapman and Hall/CRC, 1996.
10. Y. Guo, F. Liu, J. Shi, Z.-H. Zhou, and M. Gleicher. Image retargeting using mesh parametrization. IEEE Transactions on Multimedia, 11(5):856--867, 2009.
11. R. Haralick, K. Shanmugam, and I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, (6):610--621, 1973.
12. X. Hou and L. Zhang. Saliency detection: A spectral residual approach. In CVPR, pages 1--8, 2007.
13. S. C. Johnson. Hierarchical clustering schemes. Psychometrika, 32(3):2410--254, 1967.
14. Y. Ke, X. Tang, and F. Jing. The design of high-level features for photo quality assessment. In CVPR, pages 419--426, 2006.
15. J. Li, W. Wu, T. Wang, and Y. Zhang. One step beyond histogram: Image representation using markov stationary features. In CVPR, pages 1--8.
16. Y. Luo and X. Tang. Photo and video quality evaluation: Focusing on the subject. In ECCV, pages 386--399, 2008.
17. T. Mei, X.-S. Hua, H.-Q. Zhou, and S. Li. Modeling and mining of userscapture intention for home videos. IEEE Transactions on Multimedia, 9(1):66--77, 2007.
18. B. Ni, S. Yan, and A. Kassim. Contextualizing histogram. In CVPR, pages 1682--1689, 2009.
19. Y. Rui and T. Huang. A novel relevance feedback technique in image retrieval. In ACM MM, pages 67--70, 1999.
20. K. Sande, T. Gevers, and C. Snoek. Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (in press), 2009.
21. V. Setlur, S. Takagi, R. Raskar, M. Gleicher, and B. Gooch. Automatic image retargeting. In ACM International Conference on Mobile and Ubiquitous Multimedia, pages 59--68, 2005.
22. H. Sheikh, A. Bovik, and G. Veciana. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing, 14:2117--2128, 2005.
23. M. Stricker and M. Orengo. Similarity of color images. In Storage and Retrieval of Image and Video Databases III, volume 2, pages 381--392, 1995.
24. X. Sun, H. Yao, R. Ji, and S. Liu. Photo assessment based on computational visual attention model. In ACM MM, pages 541--544, 2009.
25. P. Viola and M. Jones. Robust real-time object detection. International Journal of Computer Vision, (2):137--154, 2004.
26. Y. Zheng, M. Zhao, S.-Y. Neo, T.-S. Chua, and Q. Tian. Visual synset: towards a higher-level visual representation. In CVPR, 2008.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks