A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous Counterparts
Authors: 
Zhuo Su, Kun Zeng, Hanhui Li, Xiaonan LuoAuthors Info & Claims
Zhuo Su, Kun Zeng, Hanhui Li, Xiaonan LuoAuthors Info & ClaimsArticle No.: 22, Pages 1 - 21
Abstract
For a given image, it is a challenging task to generate its corresponding counterpart with visual inconspicuous modification. The complexity of this problem reasons from the high correlativity between the editing operations and vision perception. Essentially, a significant requirement that should be emphasized is how to make the object modifications hard to be found visually in the generative counterparts. In this article, we propose a novel dual-domain perceptual framework to generate visual inconspicuous counterparts, which applies the perceptual bidirectional similarity metric (PBSM) and appearance similarity metric (ASM) to create the dual-domain perception error minimization model. The candidate targets are yielded by the well-known PatchMatch model with the strokes-based interactions and selective object library. By the dual-perceptual evaluation index, all candidate targets are sorted to select out the best result. For demonstration, a series of objective and subjective measurements are used to evaluate the performance of our framework.
References
[1]
Connelly Barnes, Dan Goldman, Eli Shechtman, and Adam Finkelstein. 2011. The PatchMatch randomized matching algorithm for image manipulation. Communications of the ACM 54, 11, 103--110.
[2]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. In Proceedings of ACM SIGGRAPH 2009 Papers, Vol. 28. 1--11.
[3]
Subhabrata Bhattacharya, Rahul Sukthankar, and Mubarak Shah. 2011. A holistic approach to aesthetic enhancement of photographs. ACM Transactions on Multimedia Computing, Communications and Applications 7S, 1, 21:1--21:21.
[4]
Ali Borji, Dicky Sihite, and Laurent Itti. 2012. Salient object detection: A benchmark. In Proceedings of the 12th European Conference on Computer Vision. 414--429.
[5]
Tao Chen, Mingming Cheng, Ping Tan, Ariel Shamir, and Shimin Hu. 2009. Sketch2Photo: Internet image montage. In Proceedings of ACM SIGGRAPH Asia 2009, Vol. 28. 124:1--124:10.
[6]
Mingming Cheng, Fanglue Zhang, Niloy Mitra, and Xiaolei and Huang. 2010. RepFinder: Finding approximately repeated scene elements for image editing. ACM Transactions on Graphics 29, 4, 1--8.
[7]
Mingming Cheng, Guoxin Zhang, Niloy J. Mitra, Xiaolei Huang, and Shimin Hu. 2011. Global contrast based salient region detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 409--416.
[8]
Mingming Cheng, Guoxin Zhang, Niloy J. Mitra, Xiaolei Huang, and Shimin Hu. 2015. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3, 569--582.
[9]
T. Cho, M. Butman, S. Avidan, and W. Freeman. 2008. The Patch Transform and its applications to image editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
[10]
A. Criminisi, P. Perez, and K. Toyama. 2003. Object removal by exemplar-based inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2. 721--728.
[11]
Kostas Daniilidis, Petros Maragos, Nikos Paragios, Connelly Barnes, Eli Shechtman, Dan Goldman, and Adam Finkelstein. 2010. The generalized PatchMatch correspondence algorithm. In Proceedings of the European Conference on Computer Vision. 29--43.
[12]
Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei Efros. 2012. What makes Paris look like Paris? ACM Transactions on Graphics 31, 4, 1--9.
[13]
Mathias Eitz, Kristian Hildebrand, Tamy Boubekeur, and Marc Alexa. 2009. PhotoSketch: A sketch based image query and compositing system. In Proceedings of SIGGRAPH 2009: Talks. 1--4.
[14]
Zeev Farbman, Gil Hoffer, Yaron Lipman, Daniel Cohen-Or, and Dani Lischinski. 2009. Coordinates for instant image cloning. In Proceedings of ACM SIGGRAPH 2009 Papers, Vol. 28. 1--9.
[15]
Chen Goldberg, Tao Chen, Fanglue Zhang, Ariel Shamir, and Shimin Hu. 2012. Data-driven object manipulation in images. Computer Graphics Forum 31, 2, 265--274.
[16]
Jonathan Harel, Christof Koch, and Pietro Perona. 2007. Graph-based visual saliency. In Proceedings of the 20th Annual Conference on Neural Information Processing Systems. 545--552.
[17]
Shimin Hu, Fanglue Zhang, Miao Wang, Ralph Martin, and Jue Wang. 2013. PatchNet: A patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics 32, 6, 196:1--196:12.
[18]
Hui Huang, Kangxue Yin, Minglun Gong, Dani Lischinski, Daniel Cohen-Or, Uri Ascher, and Baoquan Chen. 2013. “Mind the gap”: Tele-registration for structure-driven image completion. ACM Transactions on Graphics 32, 6, 1--10.
[19]
L. Itti, C. Koch, and E. Niebur. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 11, 1254--1259.
[20]
Jiaya Jia, Jian Sun, Chikeung Tang, and Heungyeung Shum. 2006. Drag-and-drop pasting. In Proceedings of ACM SIGGRAPH 2006 Papers. 631--637.
[21]
Weisi Lin and C. Jay Kuo. 2011. Perceptual visual quality metrics: A survey. Journal of Visual Communication and Image Representation 22, 4, 297--312.
[22]
Tie Liu, Jian Sun, Nanning Zheng, Xiaoou Tang, and Heungyeung Shum. 2007. Learning to detect a salient object. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 33. 1--8.
[23]
Cewu Lu, Li Xu, and Jiaya Jia. 2014. Contrast preserving decolorization with perception-based quality metrics. International Journal of Computer Vision 110, 2, 222--239.
[24]
Anush Moorthy and Alan Bovik. 2011. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Transactions on Image Processing 20, 12, 3350--3364.
[25]
Patrick Perez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. In Proceedings of ACM SIGGRAPH 2003 Papers. 313--318.
[26]
Yael Pritch, Eitam Kav-Venaki, and Shmuel Peleg. 2009. Shift-Map image editing. In Proceedings of the 12th International Conference on Computer Vision. 151--158.
[27]
Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. “GrabCut”—interactive foreground extraction using iterated graph cuts. In Proceedings of ACM SIGGRAPH 2004 Papers. 309--314.
[28]
M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir. 2010. A comparative study of image retargeting. ACM Transactions on Graphics 29, 5, 160:1--160:10.
[29]
Bryan Russell, Antonio Torralba, Kevin Murphy, and William Freeman. 2008. LabelMe: A database and Web-based tool for image annotation. International Journal of Computer Vision 77, 1--3, 157--173.
[30]
Ariel Shamir and Olga Sorkine. 2009. Visual media retargeting. In Proceedings of ACM SIGGRAPH Asia 2009 Courses. 1--13.
[31]
X. Shen, C. Zhou, L. Xu, and J. Jia. 2015. Mutual-structure for joint filtering. In Proceedings of the 2015 IEEE International Conference on Computer Vision. 3406--3414.
[32]
D. Simakov, Y. Caspi, E. Shechtman, and M. Irani. 2008. Summarizing visual data using bidirectional similarity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
[33]
Mingli Song, Dacheng Tao, Chun Chen, Xuelong Li, and Chang Chen. 2010. Color to gray: Visual cue preservation. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 9, 1537--1552.
[34]
Z. Su, K. Zeng, L. Liu, B. Li, and X. Luo. 2014. Corruptive artifacts suppression for example-based color transfer. IEEE Transactions on Multimedia 16, 4, 988--999.
[35]
Jian Sun, Lu Yuan, Jiaya Jia, and Heungyeung Shum. 2005. Image completion with structure propagation. ACM Transactions on Graphics 24, 3, 861--868.
[36]
Shaoyan Sun, Wengang Zhou, Qi Tian, and Houqiang Li. 2015. Scalable object retrieval with compact image representation from generic object regions. ACM Transactions on Multimedia Computing, Communications and Applications 12, 2, 29:1--29:21.
[37]
Michael Tao, Micah Johnson, and Sylvain Paris. 2013. Error-tolerant image compositing. International Journal of Computer Vision 103, 2, 178--189.
[38]
Joseph Tighe and Svetlana Lazebnik. 2013. Superparsing: Scalable nonparametric image parsing with superpixels. International Journal of Computer Vision 101, 2, 329--349.
[39]
Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600--612.
[40]
Pohung Wu, Chienchi Chen, Jianjiun Ding, Chiyu Hsu, and Yingwun Huang. 2013. Salient region detection improved by principle component analysis and boundary information. IEEE Transactions on Image Processing 22, 9, 3614--3624.
[41]
Yulin Xie, Huchuan Lu, and Minghsuan Yang. 2013. Bayesian saliency via low and mid level cues. IEEE Transactions on Image Processing 22, 5, 1689--1698.
[42]
Li Xu, Qiong Yan, and Jiaya Jia. 2013. A sparse control model for image and video editing. ACM Transactions on Graphics 32, 6, 197:1--197:10.
[43]
Yang Yang, Linjun Yang, Gangshan Wu, and Shipeng Li. 2012. A bag-of-objects retrieval model for Web image search. In Proceedings of the 20th ACM International Conference on Multimedia. 49--58.
[44]
Kun Zeng, Mingtian Zhao, Caiming Xiong, and Songchun Zhu. 2009. From image parsing to painterly rendering. ACM Transactions on Graphics 29, 1, 1--11.
[45]
Fanglue Zhang, Mingming Cheng, Jiaya Jia, and Shimin Hu. 2012. ImageAdmixture: Putting together dissimilar objects from groups. IEEE Transactions on Visualization and Computer Graphics 18, 11, 1849--1857.
[46]
Fanglue Zhang, Miao Wang, and Shimin Hu. 2013. Aesthetic image enhancement by dependence-aware object recomposition. IEEE Transactions on Multimedia 15, 7, 1480--1490.
[47]
Mingtian Zhao and Songchun Zhu. 2013. Abstract painting with interactive control of perceptual entropy. ACM Transactions on Applied Perception 10, 1, 1--21.
[48]
Wang Zhou and Li Qiang. 2011. Information content weighting for perceptual image quality assessment. IEEE Transactions on Image Processing 20, 5, 1185--1198.
Index Terms
- A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous Counterparts
Recommendations
Movement bias in visual attention for perceptually-guided selective rendering of animations
SCCG '07: Proceedings of the 23rd Spring Conference on Computer GraphicsThe Human Visual System (HVS) is a key part of the rendering pipeline. The human eye is only capable of sensing image detail in a 2° foveal region, relying on rapid eye movements, or saccades, to jump between points of interest. These points of interest ...
Image quality assessment metrics combining structural similarity and image fidelity with visual attention
Image quality assessment has a great importance in several image processing applications. Recently, various objective image quality metrics have been proposed in order to predict human visual perception. In this paper, novel image quality metrics, S-...
Perceptual constancy and the dynamics of extracting perceptual visual invariants in virtual immersion
SPPRA '08: Proceedings of the Fifth IASTED International Conference on Signal Processing, Pattern Recognition and ApplicationsVisual perception relies on perceptual constancy to guide motor behavior. This constancy can be assimilated to topological invariance extracted from visual exploration of the surrounding. In this paper, gaze behavior data coming from an experiment ...
Comments
Information & Contributors
Information
Published In
May 2017
226 pages
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 April 2017
Accepted: 01 March 2017
Revised: 01 February 2017
Received: 01 March 2016
Published in TOMM Volume 13, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Funding Sources
- Natural Science Foundation of Guangdong Province
- Sun Yat-sen University
- National Natural Science Foundation of China
- Science and Technology Planning Project of Guangdong Province
- Fundamental Research Funds for the Central Universities
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 141Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in