research-article

Example-based video color grading

Authors:

Nicolas Bonneel,

Kalyan Sunkavalli,

Hanspeter PfisterAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 4

Article No.: 39, Pages 1 - 12

https://doi.org/10.1145/2461912.2461939

Published: 21 July 2013 Publication History

Abstract

In most professional cinema productions, the color palette of the movie is painstakingly adjusted by a team of skilled colorists -- through a process referred to as color grading -- to achieve a certain visual look. The time and expertise required to grade a video makes it difficult for amateurs to manipulate the colors of their own video clips. In this work, we present a method that allows a user to transfer the color palette of a model video clip to their own video sequence. We estimate a per-frame color transform that maps the color distributions in the input video sequence to that of the model video clip. Applying this transformation naively leads to artifacts such as bleeding and flickering. Instead, we propose a novel differential-geometry-based scheme that interpolates these transformations in a manner that minimizes their curvature, similarly to curvature flows. In addition, we automatically determine a set of keyframes that best represent this interpolated transformation curve, and can be used subsequently, to manually refine the color grade. We show how our method can successfully transfer color palettes between videos for a range of visual styles and a number of input video clips.

Supplementary Material

ZIP File (a39-bonneel.zip)

Supplemental material.

Download
352.64 MB

MP4 File (tp135.mp4)

Download
31.43 MB

References

[1]

Amari, S., and Nagaoka, H. 2000. Methods of Information Geometry, vol. 191 of Translations of Mathematical monographs. Oxford University Press.

[2]

An, X., and Pellacini, F. 2010. User-controllable color transfer. Computer Graphics Forum 29, 2, 263--271.

[3]

Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2006) 25, 3, 637--645.

Digital Library

[4]

Bai, X., Wang, J., Simons, D., and Sapiro, G. 2009. Video SnapCut: robust video object cutout using localized classifiers. In ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2009), 70:1--70:11.

Digital Library

[5]

Bonneel, N., van de Panne, M., Paris, S., and Heidrich, W. 2011. Displacement interpolation using lagrangian mass transport. In ACM Trans. on Graphics (Proc. of ACM SIGGRAPH Asia 2011), 158:1--158:12.

Digital Library

[6]

Chang, Y., Saito, S., and Nakajima, M. 2007. Example-based color transformation of image and video using basic color categories. Image Processing, IEEE Trans. on 16, 2, 329--336.

Digital Library

[7]

Devroye, L. 1986. Non-Uniform Random Variate Generation. Springer-Verlag. Section 2.2. Inversion by numerical solution of F(X) = U.

[8]

do Carmo, M. P. 1992. Riemannian Geometry. Springer, Jan.

[9]

Farbman, Z., and Lischinski, D. 2011. Tonal stabilization of video. ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2011) 30, 4, 89:1--89:9.

Digital Library

[10]

Ferradans, S., Xia, G.-S., Peyré, G., and Aujol, J.-F. 2012. Optimal transport mixing of gaussian texture models. Tech. rep., Preprint Hal-00662720.

[11]

Galassi, M., Davis, J., Gough, B., Jungman, G., Booth, M., and Rossi, F., 2011. GNU scientific library - reference manual, version 1.15, sec. 21.7 weighted samples.

[12]

Gastal, E. S. L., and Oliveira, M. M. 2011. Domain transform for edge-aware image and video processing. In ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2011), 69:1--69:12.

Digital Library

[13]

HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2011) 30, 4, 70:1--70:9.

Digital Library

[14]

Ilmanen, T. 1995. Lectures on mean curvature flow and related equations. In Lecture Notes, ICTP, Trieste.

[15]

Johnson, M. K., Dlale, K., Avidan, S., Pfister, H., Freeman, W. T., and Matusik, W. 2011. CG2Real: improving the realism of computer generated images using a large collection of photographs. IEEE Trans. on Visualization and Computer Graphics 17, 9 (Sept.), 1273--1285.

Digital Library

[16]

Kagarlitsky, S., Moses, Y., and Hel-Or, Y. 2009. Piecewise-consistent color mappings of images acquired under various conditions. In Computer Vision, 2009 IEEE 12th International Conference on, 2311--2318.

[17]

Kiser, C., Reinhard, E., Tocci, M., and Tocci, N. 2012. Real time automated tone mapping system for HDR video. In Proc. of the IEEE Int. Conference on Image Processing, IEEE.

[18]

Lang, M., Wang, O., Aydin, T., Smolic, A., and Gross, M. 2012. Practical temporal consistency for image-based graphics applications. ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2012) 31, 4 (July), 34:1--34:8.

Digital Library

[19]

Levin, A., Lischinski, D., and Weiss, Y. 2008. A closed-form solution to natural image matting. IEEE Trans. on Pattern Analysis and Machine Intelligence 30, 2 (Feb.), 228--242.

Digital Library

[20]

Murray, N., Skaff, S., Marchesotti, L., and Perronnin, F. 2011. Towards automatic concept transfer. In Proc. of the Eurographics Symposium on Non-Photorealistic Animation and Rendering, ACM, NPAR '11, 167--176.

Digital Library

[21]

Oldenborg, M. 2006. A comparison between techniques for color grading in games. PhD thesis, University of Skövde, School of Humanities and Informatics.

[22]

Oskam, T., Hornung, A., Sumner, R. W., and Gross, M. 2012. Fast and stable color balancing for images and augmented reality. In 2nd Int. Conf. on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), 49--56.

Digital Library

[23]

Paris, S. 2008. Edge-preserving smoothing and mean-shift segmentation of video streams. In Proc. of the 10th European Conference on Computer Vision, ECCV '08, 460--473.

Digital Library

[24]

Park, H.-S., and Jun, C.-H. 2009. A simple and fast algorithm for K-medoids clustering. Expert Systems with Applications 36, 2, Part 2 (Mar.), 3336--3341.

Digital Library

[25]

Pitié, F., and Kokaram, A. C. 2007. The linear mongekantorovitch linear colour mapping for example-based colour transfer. In 4th European Conference on Visual Media Production, 2007. IETCVMP, 1--9.

[26]

Pitié, F., Kokaram, A. C., and Dahyot, R. 2005. N-dimensional probablility density function transfer and its application to colour transfer. In Proceedings of the Tenth IEEE Int. Conf. on Computer Vision - Vol. 2, ICCV '05, 1434--1439.

Digital Library

[27]

Pouli, T., and Reinhard, E. 2011. Progressive color transfer for images of arbitrary dynamic range. Computers & Graphics 35, 1 (Feb.), 67--80.

Digital Library

[28]

Rabin, J., Delon, J., and Gousseau, Y. 2010. Regularization of transportation maps for color and contrast transfer. In 2010 17th IEEE Int. Conf. on Image Processing (ICIP), 1933--1936.

[29]

Reinhard, E., and Pouli, T. 2011. Colour spaces for colour transfer. In Proceedings of the Third international conference on Computational color imaging, Springer-Verlag, Berlin, Heidelberg, CCIW'11, 1--15.

Digital Library

[30]

Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5 (Sept.), 34--41.

Digital Library

[31]

Selan, J. 2012. Cinematic color: From your monitor to the big screen. Visual Effects Soc. Tech. Committee White Paper (Oct.).

[32]

Tai, Y.-W., Jia, J., and Tang, C.-K. 2005. Local color transfer via probabilistic segmentation by expectation-maximization. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1, 747--754 vol. 1.

Digital Library

[33]

Takatsu, A. 2011. Wasserstein geometry of gaussian measures. Osaka Journal of Mathematics 48, 4 (Dec.), 1005--1026. Mathematical Reviews number (MathSciNet): MR2648273.

[34]

Villani, C. 2003. Topics in Optimal Transportation. Graduate Studies in Mathematics Series. Amer Mathematical Society.

[35]

Wang, B., Yu, Y., and Xu, Y.-Q. 2011. Example-based image color and tone style enhancement. In ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2011), 64:1--64:12.

Digital Library

[36]

Xue, S., Agarwala, A., Dorsey, J., and Rushmeier, H. E. 2012. Understanding and improving the realism of image composites. ACM Trans. on Graphics (Proc. of ACM SIGGRAPH 2012) 31, 4, 84:1--84:10.

Digital Library

[37]

Yatziv, L., and Sapiro, G. 2006. Fast image and video colorization using chrominance blending. IEEE Trans. on Image Processing 15, 5 (May), 1120--1129.

Digital Library

Cited By

Xu YXia MHu KZhou SWeng L(2025)Style Transfer Review: Traditional Machine Learning to Deep LearningInformation10.3390/info1602015716:2(157)Online publication date: 19-Feb-2025
https://doi.org/10.3390/info16020157
Zaim IStylidis DAndriotis KThickett A(2025)Projecting destinations via organic tourist videos: the role of appearance, production and contentTourism Recreation Research10.1080/02508281.2024.2417335(1-18)Online publication date: 23-Jan-2025
https://doi.org/10.1080/02508281.2024.2417335
Wang ZZhou JWang SWang N(2024)A Comprehensive Method for Example-Based Color Transfer with Holistic–Local Balancing and Unit-Wise Riemannian Information Gradient AccelerationEntropy10.3390/e2611091826:11(918)Online publication date: 29-Oct-2024
https://doi.org/10.3390/e26110918
Show More Cited By

Index Terms

Example-based video color grading
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation

Recommendations

A method for color classification of fruits based on machine vision

A dominant color histogram matching method for fruits classification was presented in this paper. In classification of fruits based on machine vision, image was acquired with a color CCD camera that outputted color information in three channels, red, ...
3D color homography model for photo-realistic color transfer re-coding

Color transfer is an image editing process that naturally transfers the color theme of a source image to a target image. In this paper, we propose a 3D color homography model which approximates photo-realistic color transfer algorithm as a combination ...
The foundation of a color-chip evaluation system of jadeite-jade green with color difference control of medical device

Based on the uniform color space International Commission on Illumination (CIE) 1976 L*a*b*, the color green of 728 pieces of jadeite-jade was quantitatively characterized by the spectrophotometer Color i5. The color-chip system of jadeite-jade green ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 4

July 2013

1215 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2461912

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2013

Published in TOG Volume 32, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
1,336
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu YXia MHu KZhou SWeng L(2025)Style Transfer Review: Traditional Machine Learning to Deep LearningInformation10.3390/info1602015716:2(157)Online publication date: 19-Feb-2025
https://doi.org/10.3390/info16020157
Zaim IStylidis DAndriotis KThickett A(2025)Projecting destinations via organic tourist videos: the role of appearance, production and contentTourism Recreation Research10.1080/02508281.2024.2417335(1-18)Online publication date: 23-Jan-2025
https://doi.org/10.1080/02508281.2024.2417335
Wang ZZhou JWang SWang N(2024)A Comprehensive Method for Example-Based Color Transfer with Holistic–Local Balancing and Unit-Wise Riemannian Information Gradient AccelerationEntropy10.3390/e2611091826:11(918)Online publication date: 29-Oct-2024
https://doi.org/10.3390/e26110918
Chen XFang LYe LZhang Q(2024)Deep Video Harmonization by Improving Spatial-temporal ConsistencyMachine Intelligence Research10.1007/s11633-023-1447-321:1(46-54)Online publication date: 15-Jan-2024
https://doi.org/10.1007/s11633-023-1447-3
Florea LFlorea CVertan C(2024)Are Computers Able to Understand Art?Digital Transformation10.1007/978-3-031-55952-5_9(159-188)Online publication date: 27-Apr-2024
https://doi.org/10.1007/978-3-031-55952-5_9
Beaudouin-Lafon ME JXia H(2023)Color Field: Developing Professional Vision by Visualizing the Effects of Color FiltersProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606828(1-16)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606828
Bonneel NDigne J(2023)A survey of Optimal Transport for Computer Graphics and Computer VisionComputer Graphics Forum10.1111/cgf.1477842:2(439-460)Online publication date: 23-May-2023
https://doi.org/10.1111/cgf.14778
Stival LPedrini H(2023)Survey on Video Colorization: Concepts, Methods and ApplicationsJournal of Signal Processing Systems10.1007/s11265-023-01872-w95:6(679-702)Online publication date: 16-May-2023
https://dl.acm.org/doi/10.1007/s11265-023-01872-w
Liu SLiu S(2023)Color TransferImage and Video Color Editing10.1007/978-3-031-26030-8_2(9-20)Online publication date: 21-Mar-2023
https://doi.org/10.1007/978-3-031-26030-8_2
Tseng EZhang YJebe LZhang XXia ZFan YHeide FChen J(2022)Neural Photo-FinishingACM Transactions on Graphics10.1145/3550454.355552641:6(1-15)Online publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1145/3550454.3555526
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents