Article

Gaze-based interaction for semi-automatic photo cropping

Authors:

Anthony Santella,

Maneesh Agrawala,

Michael CohenAuthors Info & Claims

CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 771 - 780

https://doi.org/10.1145/1124772.1124886

Published: 22 April 2006 Publication History

Abstract

We present an interactive method for cropping photographs given minimal information about important content location, provided by eye tracking. Cropping is formulated in a general optimization framework that facilitates adding new composition rules, and adapting the system to particular applications. Our system uses fixation data</ to identify important image content and compute the best crop for any given aspect ratio or size, enabling applications such as automatic snapshot recomposition, adaptive documents, and thumbnailing. We validate our approach with studies in which users compare our crops to ones produced by hand and by a completely automatic approach. Experiments show that viewers prefer our gaze-based crops to uncropped images and fully automatic crops.

References

[1]

Arnheim, R. Art and Visual Perception. University of California Press, 1974.

[2]

Arnheim, R. The Power of the Center. University of California Press, 1988.

[3]

Banerjee, S. Composition Guided Image Aquisition. PhD thesis, University of Texas at Austin, 2004.

[4]

Byers, Z., Dixon, M., Smart, W. D., and Grimm, C. M. Say cheese!: Experiences with a robot photographer. Proceedings of the Fifteenth Innovative Applications of Artificial Intelligence Conference (IAAI-03), Acapulco, Mexico.

[5]

Chen, L., Xie, X., Fan, X., Ma, W., Shang, H., and Zhou, H. A visual attention mode for adapting images on small displays. MSR-TR-2002-125 Microsoft Research, Redmond, WA (2002).

[6]

Christoudias, C., Georgescu, B., and Meer, P. Synergism in low level vision. Proceedings ICPR 2002.

Digital Library

[7]

Crow, F. Summed-area tables for texture mapping. Siggraph '84. 207--2 12.

Digital Library

[8]

David, H. A. The method of paired comparisons. Charles Griffin and Company, London, 1969.

[9]

DeCarlo, D., and Santella, A. Stylization and abstraction of photographs. Proceedings of ACM SIGGRAPH 2002. 769--776.

Digital Library

[10]

Duchowski, A. Acuity-matching resolution degradation through wavelet coefficient scaling. IEEE Trans. on Image Processing 9, 8 (2000), 1437--1440.

Digital Library

[11]

Foley, J., van Dam, A., Feiner, S., and Hughes, J. Computer Graphics: Principles and Practice, 2nd edition. Addison Wesley, 1997.

Digital Library

[12]

Gooch, B., Reinhard, E., Moulding, C., and Shirley, P. Artistic composition for image creation. Proceedings of the 12th Eurographics workshop on Rendering Technique. 83--88.

Digital Library

[13]

Graham, D. Composing Pictures. Van Nostrand Reinhold, 1970.

[14]

Grill, T., and Scanlon, M. Photographic Composition Guidelines for Total image control through effective design. AMPHOTO, 1988.

[15]

Itti, L., and Koch, C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research 40 (2000), 1489--1506.

[16]

Itti, L., Koch, C., and Niebur, E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (1998), 1254--1259.

Digital Library

[17]

Jacob, R. J. Eye-movement-based human-computer interaction techniques: Toward non-command interfaces. 151--190.

[18]

Jacobs, C., Li, W., Schrier, E., Bargeron, D., and Salesin, D. Adaptive grid-based document layout. ACM Trans. Graph. 22, 3 (2003), 838--847.

Digital Library

[19]

Kendall, M. G. On the method of paired comparisons. Biometrika 31 (1940), 324 --345.

[20]

Li, S. Z., Zhu, L., Zhang, Z. Q., Blake, A., Zhang, H. J., and Shum, H. Statistical learning of multi-view face detection. Proceedings 7th European Conference on Computer Vision(ECCV 2002) 4 (2002), 67 --81.

Digital Library

[21]

Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. Lazy snapping. ACM Trans. Graph. 23, 3 (2004), 303--308.

Digital Library

[22]

Locher, P. J. The contribution of eye-movement research to an understanding of the nature of pictorial balance perception: a review of the literature. Empirical Studies of the Arts 14, 2 (1996), 146--163.

[23]

Locher, P. J., Stappers, P. J., and Overbeeke, K. The role of balance as an organizing design principle underlying adults' compositional strategies for creating visual displays. Acta Psychologica 99 (1998), 141--161.

[24]

Locher, P. J., Stappers, P. J., and Overbeeke, K. An empirical evaluation of the visual rightness theory of pictorial composition. Acta Psychologica 103 (1999), 261--280.

[25]

Lok, S., Feiner, S., and Ngai, G. Evaluation of visual balance for automated layout. Proceedings of the 9th international conference on Intelligent user interface. 101--106.

Digital Library

[26]

McManus, I., Edmondson, D., and Rodgers, J. Balance in pictures. British Journal of Psychology 76 (1985), 73--94.

[27]

Peterson, B. Learning to see Creatively: How to compose great photographs. AMPHOTO, 1988.

[28]

Setlur, V., Takagi, S., Raskar, R., Gleicher, M., and Gooch, B. Automatic image retargeting. ACM SIGGRAPH Technical Sketch.

Digital Library

[29]

Suh, B., Ling, H., Bederson, B. B., and Jacobs, D. W. Automatic thumbnail cropping and it's effectivness. ACM Conference on User Interface and Software Technolgy (UIST 2003) (2003), 95--104.

Digital Library

[30]

Vertegaal, R. The gaze groupware system: Mediating joint attention in mutiparty communication and collaboration. Proceedings CHI '99. 294--301.

Digital Library

[31]

Vertegaal, R. Designing attentive interfaces. Proceedings of the Eye Tracking Research and Applications (ETRA) Symposium 2002. 23--30.

Digital Library

Cited By

Özdel SRong YAlbaba BKuo YWang XKasneci E(2024)Gaze-Guided Graph Neural Network for Action Anticipation Conditioned on IntentionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653340(1-9)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653340
Ma CZhao LChen YGuo LZhang THu XShen DJiang XLiu T(2024)Rectify ViT Shortcut Learning by Visual SaliencyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.331053135:12(18013-18025)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3310531
Pan ZCui JWang KWu YCao Z(2024)Pseudo Label Fusion With Uncertainty Estimation for Semi-Supervised Cropping Box RegressionIEEE Transactions on Multimedia10.1109/TMM.2024.337712526(8157-8171)Online publication date: 13-Mar-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3377125
Show More Cited By

Index Terms

Gaze-based interaction for semi-automatic photo cropping
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Sensation-based photo cropping
MM '09: Proceedings of the 17th ACM international conference on Multimedia

This paper proposes a novel method for automatically cropping a photo using a quality classifier that assesses whether the cropped region is agreeable to users. We statistically build this quality classifier using large photo collections available on ...
Gaze gestures or dwell-based interaction?
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

The two cardinal problems recognized with gaze-based interaction techniques are: how to avoid unintentional commands, and how to overcome the limited accuracy of eye tracking. Gaze gestures are a relatively new technique for giving commands, which has ...
Eye gaze in virtual environments: evaluating the need and initial work on implementation
Distributed Simulation, Virtual Environments and Real-time Applications

For efficient collaboration between participants, eye gaze is seen as being critical for interaction. Video conferencing either does not attempt to support eye gaze (e.g. AcessGrid) or only approximates it in round table conditions (e.g. life size ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2006

1353 pages

ISBN:1595933727

DOI:10.1145/1124772

Editors:
Rebecca Grinter
Georgia Institute of Technology, USA
,
Thomas Rodden
University of Nottingham, UK
,
Paul Aoki
Intel, USA
,
Ed Cutrell
Microsoft, USA
,
Robin Jeffries
Google, USA
,
Gary Olson
University of Michigan, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI06

Sponsor:

CHI06: CHI 2006 Conference on Human Factors in Computing Systems

April 22 - 27, 2006

Québec, Montréal, Canada

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

284
Total Citations
View Citations
1,708
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)9

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Özdel SRong YAlbaba BKuo YWang XKasneci E(2024)Gaze-Guided Graph Neural Network for Action Anticipation Conditioned on IntentionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653340(1-9)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653340
Ma CZhao LChen YGuo LZhang THu XShen DJiang XLiu T(2024)Rectify ViT Shortcut Learning by Visual SaliencyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.331053135:12(18013-18025)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3310531
Pan ZCui JWang KWu YCao Z(2024)Pseudo Label Fusion With Uncertainty Estimation for Semi-Supervised Cropping Box RegressionIEEE Transactions on Multimedia10.1109/TMM.2024.337712526(8157-8171)Online publication date: 13-Mar-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3377125
Alsmirat MKharsa RAlzoubi R(2024)Supervised Deep Learning for Ideal Identification of Image Retargeting TechniquesIEEE Access10.1109/ACCESS.2024.3510675(1-1)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3510675
Givkashi MNaderi MKarimi NShirani SSamavi S(2024)Supervised deep learning for content-aware image retargeting with Fourier ConvolutionsMultimedia Tools and Applications10.1007/s11042-024-18876-883:36(83611-83627)Online publication date: 19-Mar-2024
https://doi.org/10.1007/s11042-024-18876-8
Le TLee TLin SDong W(2024)Deep learning-based importance map for content-aware media retargetingMultimedia Tools and Applications10.1007/s11042-024-18389-483:30(74301-74322)Online publication date: 15-Feb-2024
https://doi.org/10.1007/s11042-024-18389-4
Ayubi JChehel Amirani MValizadeh M(2024)A new content-aware image resizing based on Rényi entropy and deep learningNeural Computing and Applications10.1007/s00521-024-09517-036:15(8885-8899)Online publication date: 30-Mar-2024
https://doi.org/10.1007/s00521-024-09517-0
Lee BRijhwani DLang Svan Oorde‐Grainger SHaak ABleise CLylyk PRuijters DSinha A(2024)Tunable and real‐time automatic interventional x‐ray collimation from semi‐supervised deep feature extractionMedical Physics10.1002/mp.17522Online publication date: 6-Dec-2024
https://doi.org/10.1002/mp.17522
Avidan SShamir A(2023)Seam Carving for Content-Aware Image ResizingSeminal Graphics Papers: Pushing the Boundaries, Volume 210.1145/3596711.3596776(609-617)Online publication date: 2-Aug-2023
https://doi.org/10.1145/3596711.3596776
Nishiyasu TShimoda WSato Y(2023)Image Cropping under Design ConstraintsProceedings of the 5th ACM International Conference on Multimedia in Asia10.1145/3595916.3626412(1-7)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3595916.3626412
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten