skip to main content
article

Real-time video abstraction

Published: 01 July 2006 Publication History

Abstract

We present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an approximation to anisotropic diffusion, and artificially increase contrast in higher contrast regions with difference-of-Gaussian edges. The abstraction step is extensible and allows for artistic or data-driven control. Abstracted images can optionally be stylized using soft color quantization to create cartoon-like effects with good temporal coherence. Our framework design is highly parallel, allowing for a GPU-based, real-time implementation. We evaluate the effectiveness of our abstraction framework with a user-study and find that participants are faster at naming abstracted faces of known persons compared to photographs. Participants are also better at remembering abstracted images of arbitrary scenes in a memory task.

Supplementary Material

JPG File (p1221-winnemoller-high.jpg)
JPG File (p1221-winnemoller-low.jpg)
High Resolution (p1221-winnemoller-high.mov)
Low Resolution (p1221-winnemoller-low.mov)

References

[1]
Arad, N., and Gotsman, C. 1999. Enhancement by image-dependent warping. IEEE Trans. on Image Processing 8, 9, 1063--1074.
[2]
Barash, D., and Comaniciu, D. 2004. A common framework for non-linear diffusion, adaptive smoothing, bilateral filtering and mean shift. Image and Video Computing 22, 1, 73--81.
[3]
Boomgaard, R. V. D., and de Weijer, J. V. 2002. On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks. 16th Internat. Conf. on Pattern Recog. 3, 927--390.
[4]
Canny, J. F. 1986. A computational approach to edge detection. IEEE Trans. on Pattern Analysis and Machine Intelligence 8, 769--798.
[5]
Collomosse, J. P., Rowntree, D., and Hall, P. M. 2005. Stroke surfaces: Temporally coherent artistic animations from video. IEEE Trans. on Visualization and Computer Graphics 11, 5, 540--549.
[6]
DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. ACM Trans. Graph. 21, 3, 769--776.
[7]
Elder, J. H. 1999. Are edges incomplete? Internat. Journal of Computer Vision 34, 2-3, 97--122.
[8]
Fischer, J., Bartz, D., and Strasser, W. 2005. Stylized Augmented Reality for Improved Immersion. In Proc. of IEEE VR, 195--202.
[9]
Gooch, B., Reinhard, E., and Gooch, A. 2004. Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23, 1, 27--44.
[10]
Hertzmann, A. 2001. Paint by relaxation. In CGI '01:Computer Graphics Internat. 2001, 47--54.
[11]
Itti, L., and Koch, C. 2001. Computational modeling of visual attention. Nature Reviews Neuroscience 2, 3, 194--203.
[12]
Loviscach, J. 1999. Scharfzeichner: Klare bilddetails durch verformung. Computer Technik 22, 236ff.
[13]
Marr, D., and Hildreth, E. C. 1980. Theory of edge detection. Proc. Royal Soc. London, Bio. Sci. 207, 187--217.
[14]
Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.
[15]
Perona, P., and Malik, J. 1991. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 7.
[16]
Pham, T. Q., and Vliet, L. J. V. 2005. Separable bilateral filtering for fast video preprocessing. In IEEE Internat. Conf. on Multimedia & Expo, CD1-4.
[17]
Privitera, C. M., and Stark, L. W. 2000. Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 9, 970--982.
[18]
Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. 23, 3, 679--688.
[19]
Saito, T., and Takahashi, T. 1990. Comprehensible rendering of 3-D shapes. In Proc. of ACM SIGGRAPH 90, 197--206.
[20]
Santella, A., and DeCarlo, D. 2004. Visual interest and NPR: an evaluation and manifesto. In Proc. of NPAR '04, 71--78.
[21]
Stevenage, S. V. 1995. Can caricatures really produce distinctiveness effects? British Journal of Psychology 86, 127--146.
[22]
Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proceedings of ICCV '98, 839.
[23]
Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583.
[24]
Winkenbach, G., and Salesin, D. H. 1994. Computer-generated pen-and-ink illustration. In Proc. of ACM SIGGRAPH 94, 91--100.
[25]
Wyszecki, G., and Styles, W. 1982. Color Science: Concepts and Methods, Quantitative Data and Formulae. Wiley, New York, NY.

Cited By

View all
  • (2024)Traditional art design expression based on embedded system developmentPeerJ Computer Science10.7717/peerj-cs.205510(e2055)Online publication date: 28-Jun-2024
  • (2024)Feature Weighted Cycle Generative Adversarial Network with Facial Landmark Recognition and Perceptual Color Distance for Enhanced Face Animation GenerationElectronics10.3390/electronics1323476113:23(4761)Online publication date: 2-Dec-2024
  • (2024)Fast Coherent Video Style Transfer via Flow Errors ReductionApplied Sciences10.3390/app1406263014:6(2630)Online publication date: 21-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 25, Issue 3
July 2006
742 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1141911
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2006
Published in TOG Volume 25, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. image abstraction
  2. non-photorealistic rendering
  3. visual communication
  4. visual perception

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)95
  • Downloads (Last 6 weeks)9
Reflects downloads up to 28 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Traditional art design expression based on embedded system developmentPeerJ Computer Science10.7717/peerj-cs.205510(e2055)Online publication date: 28-Jun-2024
  • (2024)Feature Weighted Cycle Generative Adversarial Network with Facial Landmark Recognition and Perceptual Color Distance for Enhanced Face Animation GenerationElectronics10.3390/electronics1323476113:23(4761)Online publication date: 2-Dec-2024
  • (2024)Fast Coherent Video Style Transfer via Flow Errors ReductionApplied Sciences10.3390/app1406263014:6(2630)Online publication date: 21-Mar-2024
  • (2024)Scene Classification on Fine Arts with Style TransferProceedings of the 6th workshop on the analySis, Understanding and proMotion of heritAge Contents10.1145/3689094.3689468(18-27)Online publication date: 28-Oct-2024
  • (2024)Stylized Rendering as a Function of ExpectationACM Transactions on Graphics10.1145/365816143:4(1-19)Online publication date: 19-Jul-2024
  • (2024)Sketch Video SynthesisComputer Graphics Forum10.1111/cgf.1504443:2Online publication date: 30-Apr-2024
  • (2024)Parameterized $L_{0}$ Image Smoothing With Unsupervised LearningIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33590608:2(1938-1951)Online publication date: May-2024
  • (2024)Enhancing Anime Avatar Generation and Image Transfer with GANs2024 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI)10.1109/IATMSI60426.2024.10503489(1-6)Online publication date: 14-Mar-2024
  • (2024)Research of Image Arbitrary Style Transfer based on Contrastive Learning2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC)10.1109/IAEAC59436.2024.10504039(69-74)Online publication date: 15-Mar-2024
  • (2024)A review of deep learning-based image style transfer researchThe Imaging Science Journal10.1080/13682199.2024.2418216(1-23)Online publication date: 23-Oct-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media