ABSTRACT
This paper describes a new framework for processing images by example, called “image analogies.” The framework involves two stages: a design phase, in which a pair of images, with one image purported to be a “filtered” version of the other, is presented as “training data”; and an application phase, in which the learned filter is applied to some new target image in order to create an “analogous” filtered result. Image analogies are based on a simple multi-scale autoregression, inspired primarily by recent results in texture synthesis. By choosing different types of source image pairs as input, the framework supports a wide variety of “image filter” effects, including traditional image filters, such as blurring or embossing; improved texture synthesis, in which some textures are synthesized with higher quality than by previous approaches; super-resolution, in which a higher-resolution image is inferred from a low-resolution source; texture transfer, in which images are “texturized” with some arbitrary source texture; artistic filters, in which various drawing and painting styles are synthesized based on scanned real-world examples; and texture-by-numbers, in which realistic scenes, composed of a variety of textures, are created using a simple painting interface.
- 1.Sunil Arya, David M. Mount, Nathan S. Netanyahu, Ruth Silverman, and Angela Y. Wu. An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions. Journal of the ACM, 45(6):891-923, 1998. Source code available from http://www.cs.umd.edu/~mount/ANN. Google ScholarDigital Library
- 2.Michael Ashikhmin. Synthesizing Natural Textures. 2001 ACM Symposium on Interactive 3D Graphics, pages 217-226, March 2001. Google ScholarDigital Library
- 3.Michael F. Barnsley, Lyman P. Hurd, and Louisa F. Anson. Fractal Image Compression. A.K. Peters Ltd, 1993. Google ScholarDigital Library
- 4.Jeremy S. De Bonet. Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images. Proceedings of SIGGRAPH 97, pages 361-368, August 1997. Google ScholarDigital Library
- 5.Matthew Brand. Voice Puppetry. Proceedings of SIGGRAPH 99, pages 21-28, August 1999. Google ScholarDigital Library
- 6.Matthew Brand and Aaron Hertzmann. Style machines. Proceedings of SIG- GRAPH 2000, pages 183-192, July 2000. Google ScholarDigital Library
- 7.Christoph Bregler, Michele Covell, and Malcolm Slaney. Video Rewrite: Driving Visual Speech with Audio. Proceedings of SIGGRAPH 97, pages 353-360, August 1997. Google ScholarDigital Library
- 8.Ian Buck, Adam Finkelstein, Charles Jacobs, Allison Klein, David H. Salesin, Joshua Seims, Richard Szeliski, and Kentaro Toyama. Performance-driven handdrawn animation. NPAR 2000: First International Symposium on Non Photorealistic Animation and Rendering, pages 101-108, June 2000. Google ScholarDigital Library
- 9.Kenneth Castleman. Digital Image Processing. Prentice-Hall, 1996. Google ScholarDigital Library
- 10.Cassidy J. Curtis, Sean E. Anderson, Joshua E. Seims, Kurt W. Fleischer, and David H. Salesin. Computer-Generated Watercolor. Proceedings of SIGGRAPH 97, pages 421-430, August 1997. Google ScholarDigital Library
- 11.Alexei Efros and Thomas Leung. Texture Synthesis by Non-parametric Sampling. 7th IEEE International Conference on Computer Vision, 1999. Google ScholarDigital Library
- 12.Alexei A. Efros and William T. Freeman. Quilting for Texture Synthesis and Transfer. Proceedings of SIGGRAPH 2001, August 2001. Google ScholarDigital Library
- 13.Alex Eilhauer, Alice Pritikin, Dylan Weed, and Steven J. Gortler. Combining Textures and Pictures with Specialized Texture Synthesis, 2000. http://www.people.fas.harvard.edu/ ~pritikin/cs/graphics/.Google Scholar
- 14.Chris Eliasmith. Dictionary of Philosophy of Mind. http://artsci.wustl.edu/ ~philos/MindDict/.Google Scholar
- 15.T.G. Evans. A program for the solution of geometric analogy intelligence test questions. In M. Minsky, editor, Semantic Information Processing. MIT Press, 1968.Google Scholar
- 16.James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes. Computer Graphics, Principles and Practice, Second Edition. Addison-Wesley, 1990. Google ScholarDigital Library
- 17.W. T. Freeman, E. C. Pasztor, and O. T. Carmichael. Learning Low- Level Vision. Intl. J. Computer Vision, 40(1):25-47, 2000. See also http://www.merl.com/reports/TR2000-05/. Google ScholarDigital Library
- 18.William T. Freeman, Joshua B. Tenenbaum, and Egon Pasztor. An examplebased approach to style translation for line drawings. Technical Report TR99-11, MERL, February 1999.Google Scholar
- 19.D. Gentner. Structure mapping: A theoretical framework for analogy. Cognitive Science, 7(2):155-170, 1983.Google ScholarCross Ref
- 20.Allen Gersho and Robert M. Gray. Vector Quantization and Signal Compression. Kluwer Academic Publishers, 1992. Google ScholarDigital Library
- 21.Paul E. Haeberli. Paint By Numbers: Abstract Image Representations. In Computer Graphics (SIGGRAPH '90 Proceedings), volume 24, pages 207-214, August 1990. Google ScholarDigital Library
- 22.J. Hamel and T. Strothotte. Capturing and re-using rendition styles for nonphotorealistic rendering. Computer Graphics Forum, 18(3):173-182, September 1999.Google ScholarCross Ref
- 23.David J. Heeger and James R. Bergen. Pyramid-Based Texture Analysis/Synthesis. Proceedings of SIGGRAPH 95, pages 229-238, August 1995. Google ScholarDigital Library
- 24.Aaron Hertzmann. Painterly Rendering with Curved Brush Strokes of Multiple Sizes. In SIGGRAPH 98 Conference Proceedings, pages 453-460, July 1998. Google ScholarDigital Library
- 25.Aaron Hertzmann. Algorithms for Rendering in Artistic Styles. PhD thesis, New York University, May 2001.Google Scholar
- 26.Aaron Hertzmann and Denis Zorin. Illustrating smooth surfaces. Proceedings of SIGGRAPH 2000, pages 517-526, July 2000. Google ScholarDigital Library
- 27.Youichi Horry, Ken ichi Anjyo, and Kiyoshi Arai. Tour Into the Picture: Using a Spidery Mesh Interface to Make Animation from a Single Image. Proceedings of SIGGRAPH 97, pages 225-232, August 1997. Google ScholarDigital Library
- 28.William James. The Principles of Psychology. 1890.Google Scholar
- 29.Bela Julesz. Textons, the elements of texture perception, and their interactions. Nature, 290:91-97, 1981.Google ScholarCross Ref
- 30.Allison W. Klein, Wilmot W. Li, Michael M. Kazhdan, Wagner T. Correa, Adam Finkelstein, and Thomas A. Funkhouser. Non-photorealistic virtual environments. Proceedings of SIGGRAPH 2000, pages 527-534, July 2000. Google ScholarDigital Library
- 31.Arthur Koestler. The Act of Creation. Picador, London, 1964.Google Scholar
- 32.Michael A. Kowalski, Lee Markosian, J. D. Northrup, Lubomir Bourdev, Ronen Barzel, Loring S. Holden, and John Hughes. Art-Based Rendering of Fur, Grass, and Trees. Proceedings of SIGGRAPH 99, pages 433-438, August 1999. Google ScholarDigital Library
- 33.G. Lakoff and M. Johnson. Metaphors we live by. University of Chicago Press, Chicago, IL, 1980.Google Scholar
- 34.Thomas Leung and Jitendra Malik. Recognizing surfaces using threedimensional textons. 7th IEEE International Conference on Computer Vision, September 1999. Google ScholarDigital Library
- 35.Jitendra Malik, Serge Belongie, Jianbo Shi, and Thomas Leung. Textons, Contours, and Regions: Cue Integration in Image Segmentation. 7th IEEE International Conference on Computer Vision, September 1999. Google ScholarDigital Library
- 36.Barbara J. Meier. Painterly Rendering for Animation. In SIGGRAPH 96 Conference Proceedings, pages 477-484, August 1996. Google ScholarDigital Library
- 37.Pietro Perona and Jitendra Malik. Scale-Space and Edge Detection using Anisotropic Diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence, 12:629-639, December 1990. Google ScholarDigital Library
- 38.Ferdinand Petrie and John Shaw. The Big Book of Painting Nature in Watercolor. Watson-Guptill Publications, 1990.Google Scholar
- 39.Kris Popat and Rosalind W. Picard. Cluster-based probability model and its application to image and texture processing. IEEE Trans. on Image Processing, 6(2):268-284, February 1997. Google ScholarDigital Library
- 40.J. Portilla and E. P. Simoncelli. A Parametric Texture Model based on Joint Statistics of Complex Wavelet Coefficients. International Journal of Computer Vision, 40(1):49-71, December 2000. Google ScholarDigital Library
- 41.Michael P. Salisbury, Sean E. Anderson, Ronen Barzel, and David H. Salesin. Interactive Pen-And-Ink Illustration. In Proceedings of SIGGRAPH '94 (Orlando, Florida, July 24-29, 1994), pages 101-108, July 1994. Google ScholarDigital Library
- 42.Michael P. Salisbury, Michael T. Wong, John F. Hughes, and David H. Salesin. Orientable Textures for Image-Based Pen-and-Ink Illustration. In SIGGRAPH 97 Conference Proceedings, pages 401-406, August 1997. Google ScholarDigital Library
- 43.Arno Schodl, Richard Szeliski, David H. Salesin, and Irfan Essa. Video Textures. Proceedings of SIGGRAPH 2000, pages 489-498, July 2000. Google ScholarDigital Library
- 44.K. Schunn and K. Dunbar. Priming, Analogy and Awareness in complex reasoning. Memory and Cognition, 24:271-284, 1996.Google ScholarCross Ref
- 45.Eero P. Simoncelli and William T. Freeman. The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation. Proc. 2nd Int'l Conf on Image Processing, October 1995. Google ScholarDigital Library
- 46.Oleg Veryovka and John W. Buchanan. Comprehensive Halftoning of 3D Scenes. Computer Graphics Forum, 18(3):13-22, September 1999.Google ScholarCross Ref
- 47.Oleg Veryovka and John W. Buchanan. Halftoning With Image-Based Dither Screens. Graphics Interface '99, pages 167-174, June 1999. Google ScholarDigital Library
- 48.B. Wandell. Foundations of Vision. Sinauer Associates Inc., 1995.Google Scholar
- 49.Li-Yi Wei and Marc Levoy. Fast Texture Synthesis Using Tree-Structured Vector Quantization. Proceedings of SIGGRAPH 2000, pages 479-488, July 2000. Google ScholarDigital Library
- 50.Georges Winkenbach and David H. Salesin. Computer-Generated Pen-And-Ink Illustration. In Proceedings of SIGGRAPH '94 (Orlando, Florida, July 24-29, 1994), pages 91-100, July 1994. Google ScholarDigital Library
- 51.Georges Winkenbach and David H. Salesin. Rendering Parametric Surfaces in Pen and Ink. In SIGGRAPH 96 Conference Proceedings, pages 469-476, August 1996. Google ScholarDigital Library
- 52.P.H. Winston. Learning and Reasoning by Analogy. Communications of the ACM, (23) 12, December 1980. Google ScholarDigital Library
- 53.Daniel N. Wood, Adam Finkelstein, John F. Hughes, Craig E. Thayer, and David H. Salesin. Multiperspective panoramas for cel animation. Proceedings of SIGGRAPH 97, pages 243-250, August 1997. Google ScholarDigital Library
- 54.Song Chun Zhu, Ying Nian Wu, and David Mumford. Filters, Random fields, And Maximum Entropy: Towards a Unified Theory for Texture Modeling. International Journal of Computer Vision, 12(2):1-20, March/April 1998. Google ScholarDigital Library
Index Terms
- Image analogies
Recommendations
Image Analogies
Seminal Graphics Papers: Pushing the Boundaries, Volume 2This paper describes a new framework for processing images by example, called "image analogies."based on scanned real-world examples; and texture-by-numbers, in which realistic scenes, composed of a variety of textures, are created using a simple ...
Directional texture transfer
NPAR '10: Proceedings of the 8th International Symposium on Non-Photorealistic Animation and RenderingA texture transfer algorithm modifies the target image replacing the high frequency information with the example source image. Previous texture transfer techniques normally use such factors as color distance and standard deviation for selecting the best ...
Image composition with blurring effect from depth of field
ICEC'07: Proceedings of the 6th international conference on Entertainment ComputingThis paper describes a new framework for image composition according to the blurring effect bred by changing depth of field from the target images. The framework involves two stages: a learning phase, in which the target image, with one part of the ...
Comments