skip to main content
10.1145/1866029.1866066acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Cosaliency: where people look when comparing images

Published: 03 October 2010 Publication History

Abstract

Image triage is a common task in digital photography. Determining which photos are worth processing for sharing with friends and family and which should be deleted to make room for new ones can be a challenge, especially on a device with a small screen like a mobile phone or camera. In this work we explore the importance of local structure changes?e.g. human pose, appearance changes, object orientation, etc.?to the photographic triage task. We perform a user study in which subjects are asked to mark regions of image pairs most useful in making triage decisions. From this data, we train a model for image saliency in the context of other images that we call cosaliency. This allows us to create collection-aware crops that can augment the information provided by existing thumbnailing techniques for the image triage task.

References

[1]
}}Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proc. SIGGRAPH). 2009.
[2]
}}Michael J. Black and P. Anandan. The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision Image Understanding, 1996.
[3]
}}Oren Boiman and Michal Irani. Detecting irregularities in images and in video. International Journal of Computer Vision, 2007.
[4]
}}Lubomir Bourdev and Jonathan Brandt. Robust object detection via soft cascade. In Proc. CVPR, 2005.
[5]
}}Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 2008.
[6]
}}Eric Hayman and Jan-Olof Eklundh. Statistical background subtraction for a mobile observer. In Proc. ICCV, 2003.
[7]
}}Laurent Itti, Christof Koch, and Ernst Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Pattern Recognition, 1998.
[8]
}}Nebojsa Jojic, Brendan J. Frey, and Anitha Kannan. Epitomic analysis of appearance and shape. In Proc. ICCV, 2003.
[9]
}}Tilke Judd, Krista Ehinger, Frédo Durand, and Antonio Torralba. Learning to predict where humans look. In Proc. ICCV, 2009.
[10]
}}Lalatendu Satpathy, Bridget Lewis, Saara Kamppari, Benjamin Elgart, Ajay Prasad, Yong Woo Rhee, Brad Myers and Sue Fussel. Digital photo lifecycle. http://www.webvizdesigners.com/projects_pluto.php.
[11]
}}Feng Liu and Michael Gleicher. Automatic image retargeting with fisheye-view warping. In Proc. UIST, 2005.
[12]
}}Ligang Liu, Renjie Chen, Lior Wolf, and Daniel Cohen-Or. Optimizing photo composition. Computer Graphics Forum (Proc. Eurographics), 2010.
[13]
}}David G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004.
[14]
}}Amazon Mechanical Turk. https://www.mturk.com/.
[15]
}}Aude Oliva and Antonio Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 2001.
[16]
}}16. D. A. Poggel, H. Strasburger, and M. MacKeben. Cueing attention by relative motion in the periphery of the visual field. Perception, 2007.
[17]
}}Richard J. Radke, Srinivas Andra, Omar Al-Kofahi, and Badrinath Roysam. Image change detection algorithms: A systematic survey. IEEE Transactions on Image Processing, 2005.
[18]
}}Ramin Samadani, Timothy A. Mauer, David M. Berfanger, and James H. Clark. Image thumbnails that represent blur and noise. IEEE Transactions on Image Processing, 2010.
[19]
}}19. Denis Simakov, Yaron Caspi, Eli Shechtman, and Michal Irani. Summarizing visual data using bidirectional similarity. In Proc. CVPR, 2008.
[20]
}}Chris Stauffer and W. E. L. Grimson. Adaptive background mixture models for real-time tracking. In Proc. CVPR, 1999.
[21]
}}Steven Drucker, Curtis Wong, Asta Roseway, Steve Glenner, and Steve De Mar. Photo-triage: Rapidly annotating your digital photographs. Technical report, MSR, 2003.
[22]
}}Bongwon Suh, Haibin Ling, Benjamin B. Bederson, and David W. Jacobs. Automatic thumbnail cropping and its effectiveness. In Proc. UIST, 2003.
[23]
}}Paul Viola and Michael Jones. Robust real-time object detection. International Journal of Computer Vision, 2001.
[24]
}}24. Holger Winnemöeller, Sven C. Olsen, and BrucE Gooch. Real-time video abstraction. ACM Transactions on Graphics (Proc. SIGGRAPH), 2006.
[25]
}}Hulya Yalcin, Martial Hebert, Robert Collins, and Michael J. Black. A flow-based approach to vehicle detection and background mosaicking in airborne video. In Proc. CVPR, 2005.

Cited By

View all
  • (2024)Interaction Techniques for Comparing VideoProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670948(1-13)Online publication date: 3-Jun-2024
  • (2024)Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch ClassificationComputer Vision – ACCV 202410.1007/978-981-96-0972-7_20(342-358)Online publication date: 10-Dec-2024
  • (2023)GCoNet+: A Stronger Group Collaborative Co-Salient Object DetectorIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.326457145:9(10929-10946)Online publication date: 1-Sep-2023
  • Show More Cited By

Index Terms

  1. Cosaliency: where people look when comparing images

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technology
    October 2010
    476 pages
    ISBN:9781450302715
    DOI:10.1145/1866029
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 October 2010

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. automated thumbnailing
    2. collection-aware cropping
    3. cosaliency
    4. saliency

    Qualifiers

    • Research-article

    Conference

    UIST '10

    Acceptance Rates

    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)24
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Interaction Techniques for Comparing VideoProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670948(1-13)Online publication date: 3-Jun-2024
    • (2024)Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch ClassificationComputer Vision – ACCV 202410.1007/978-981-96-0972-7_20(342-358)Online publication date: 10-Dec-2024
    • (2023)GCoNet+: A Stronger Group Collaborative Co-Salient Object DetectorIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.326457145:9(10929-10946)Online publication date: 1-Sep-2023
    • (2023)Co-Salient Object Detection with Co-Representation PurificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3234586(1-12)Online publication date: 2023
    • (2023)Co-Saliency Detection Guided by Group Weakly Supervised LearningIEEE Transactions on Multimedia10.1109/TMM.2022.316780525(1810-1818)Online publication date: 2023
    • (2023)Adaptive Group-Wise Consistency Network for Co-Saliency DetectionIEEE Transactions on Multimedia10.1109/TMM.2021.313824625(764-776)Online publication date: 2023
    • (2023)Co-Salient Object Detection with Uncertainty-Aware Group Exchange-Masking2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01881(19639-19648)Online publication date: Jun-2023
    • (2022)Co-saliency detection with intra-group two-stage group semantics propagation and inter-group contrastive learningKnowledge-Based Systems10.1016/j.knosys.2022.109356252(109356)Online publication date: Sep-2022
    • (2022)Multi-frame co-saliency spatio-temporal regularization correlation filters for object trackingJournal of Visual Communication and Image Representation10.1016/j.jvcir.2021.10332981:COnline publication date: 9-Apr-2022
    • (2022)Global attention network for collaborative saliency detectionInternational Journal of Machine Learning and Cybernetics10.1007/s13042-022-01531-914:2(407-417)Online publication date: 4-Apr-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media