research-article

CueFlik: interactive concept learning in image search

Authors:

Simon WinderAuthors Info & Claims

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 29 - 38

https://doi.org/10.1145/1357054.1357061

Published: 06 April 2008 Publication History

Abstract

Web image search is difficult in part because a handful of keywords are generally insufficient for characterizing the visual properties of an image. Popular engines have begun to provide tags based on simple characteristics of images (such as tags for black and white images or images that contain a face), but such approaches are limited by the fact that it is unclear what tags end users want to be able to use in examining Web image search results. This paper presents CueFlik, a Web image search application that allows end users to quickly create their own rules for re ranking images based on their visual characteristics. End users can then re rank any future Web image search results according to their rule. In an experiment we present in this paper, end users quickly create effective rules for such concepts as "product photos", "portraits of people", and "clipart". When asked to conceive of and create their own rules, participants create such rules as "sports action shot" with images from queries for "basketball" and "football". CueFlik represents both a promising new approach to Web image search and an important study in end user interactive machine learning.

References

[1]

Bederson, B.B. (2001). PhotoMesa: A Zoomable Image Browser Using Quantum Treemaps and Bubblemaps. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2001), 71--80.

Digital Library

[2]

Cai, D., He, X., Li, Z., Ma, W.-Y. and Wen, J.-R. (2004). Hierarchical Clustering of WWW Image Search Results Using Visual, Textual, and Link Information. Proceedings of the ACM Conference on Multimedia (Multimedia 2004), 952--959.

Digital Library

[3]

Fails, J.A. and Olsen, D.R. (2003). Interactive Machine Learning. Proceedings of the International Conference on Intelligent User Interfaces (IUI 2003), 39--45.

Digital Library

[4]

Fass, A.M., Bier, E.A. and Adar, E. (2000). PicturePiper: Using a Re-Configurable Pipeline to Find Images on the Web. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2000), 51--62.

Digital Library

[5]

Gajos, K. and Weld, D.S. (2004). SUPPLE: Automatically Generating User Interfaces. Proceedings of the International Conference on Intelligent User Interfaces (IUI 2004), 93--100.

Digital Library

[6]

Gajos, K. and Weld, D.S. (2005). Preference Elicitation for Interface Optimization. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2005), 173--182.

Digital Library

[7]

Globerson, A. and Roweis, S. (2005). Metric Learning by Collapsing Classes. Proceedings of the Conference on Neural Information Processing Systems (NIPS 2005), 451--458.

[8]

Gorkani, M.M. and Picard, R.W. (1994). Texture Orientation for Sorting Photos 'At a Glance'. Proceedings of the International Conference on Pattern Recognition (ICPR 1994), 459--464.

[9]

Hartmann, B., Abdulla, L., Mittal, M. and Klemmer, S.R. (2007). Authoring Sensor-Based Interactions by Demonstration with Direct Manipulation and Pattern Recognition. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2007), 145--154.

Digital Library

[10]

Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning. Springer.

[11]

Kristjannson, T., Culotta, A., Viola, P. and McCallum, A. (2004). Interactive Information Extraction with Constrained Conditional Random Fields. Proceedings of the National Conference on Artificial Intelligence (AAAI 2004), 412--418.

Digital Library

[12]

Li, F.-F. and Perona, P. (2005). A Bayesian Hierarchical Model for Learning Natural Scene Categories. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005), 524--531.

Digital Library

[13]

Niblack, C.W., Barber, R., Equitz, W., Flickner, M.D., Glasman, E.H., Petkovic, D., Yanker, P., Faloutsos, C. and Taubin, G. (1993). QBIC Project: Querying Images by Content, Using Color, Texture, and Shape. Proceedings of the Conference on Storage and Retrieval for Image and Video Databases 173--187.

[14]

Nistér, D. and Stewénius, H. (2006). Scalable Recognition with a Vocabulary Tree. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2006), 2161--2168.

Digital Library

[15]

Nocedal, J. and Wright, S.J. (2006). Numerical Optimization. Science Press.

[16]

Platt, J.C., Czerwinski, M. and Field, B.A. (2003). PhotoTOC: Automatic Clustering for Browsing Personal Photographs. Proceedings of the IEEE Pacific Rim Conference on Multimedia 6--10.

[17]

Schettini, R., Ciocca, G., Valsasna, A., Brambilla, C. and De Ponti, M. (2002). A Hierarchical Classification Strategy for Digital Documents. Pattern Recognition 35(8). 1759--1769.

[18]

Shilman, M., Tan, D.S. and Simard, P. (2006). CueTIP: A Mixed-Initiative Interface for Correcting Handwriting Errors. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2006), 323--332.

Digital Library

[19]

Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A. and Jain, R. (2000). Content-Based Image Retrieval at the End of the Early Years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12). 1349--1380.

Digital Library

[20]

Smith, J.R. and Chang, S.-F. (1997). VisualSEEK: A Fully Automated Content-Based Image Query System. Proceedings of the ACM Conference on Multimedia (Multimedia 1997), 87--98.

Digital Library

[21]

Vailaya, A., Figueiredo, M., Jain, A. and Zhang, H.J. (1999). Content-Based Hierarchical Classification of Vacation Images. Porceedings of the IEEE International Conference on Multimedia Computing and Systems (ICMCS 1999), 518--523.

Digital Library

[22]

Wang, S., Jing, F., He, J., Du, Q. and Zhang, L. (2007). IGroup: Presenting Web Image Search Results in Semantic Clusters. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2007), 587--596.

Digital Library

[23]

Winder, S.A. and Brown, M. (2007). Learning Local Image Descriptors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007), 1--8.

[24]

Yee, K.-P., Swearingen, K., Li, K. and Hearst, M. (2003). Faceted Metadata for Image Search and Browsing. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2003), 401--408.

Digital Library

Cited By

Kawabe WSugano Y(2024)Image-to-Text Translation for Interactive Image Recognition: A Comparative User Study with Non-expert UsersJournal of Information Processing10.2197/ipsjjip.32.35832(358-368)Online publication date: 2024
https://doi.org/10.2197/ipsjjip.32.358
Lee YLam MVasconcelos HBernstein MFinn C(2024)Clarify: Improving Model Robustness With Natural Language CorrectionsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676362(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676362
Das Antar AMolaei SChen YLee MBanovic N(2024)VIME: Visual Interactive Model Explorer for Identifying Capabilities and Limitations of Machine Learning Models for Sequential Decision-MakingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676323(1-21)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676323
Show More Cited By

Index Terms

CueFlik: interactive concept learning in image search
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Leveraging User Interaction Signals for Web Image Search
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

User interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, ...
How Well do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

Comparing to general Web search engines, image search engines present search results differently, with two-dimensional visual image panel for users to scroll and browse quickly. These differences in result presentation can significantly impact the way ...
Constructing an Interaction Behavior Model for Web Image Search
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

User interaction behavior is a valuable source of implicit relevance feedback. In Web image search a different type of search result presentation is used than in general Web search, which leads to different interaction mechanisms and user behavior. For ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2008

1870 pages

ISBN:9781605580111

DOI:10.1145/1357054

General Chairs:
Mary Czerwinski
Microsoft Research, USA
,
Arnie Lund
Microsoft, USA
,
Program Chair:
Desney Tan
Microsoft Research, USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 April 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '08

Sponsor:

CHI '08: CHI Conference on Human Factors in Computing Systems

April 5 - 10, 2008

Florence, Italy

Acceptance Rates

CHI '08 Paper Acceptance Rate 157 of 714 submissions, 22%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

136
Total Citations
View Citations
1,745
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)7

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kawabe WSugano Y(2024)Image-to-Text Translation for Interactive Image Recognition: A Comparative User Study with Non-expert UsersJournal of Information Processing10.2197/ipsjjip.32.35832(358-368)Online publication date: 2024
https://doi.org/10.2197/ipsjjip.32.358
Lee YLam MVasconcelos HBernstein MFinn C(2024)Clarify: Improving Model Robustness With Natural Language CorrectionsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676362(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676362
Das Antar AMolaei SChen YLee MBanovic N(2024)VIME: Visual Interactive Model Explorer for Identifying Capabilities and Limitations of Machine Learning Models for Sequential Decision-MakingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676323(1-21)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676323
Ye YZhu QXiao SZhang KZeng W(2024)The Contemporary Art of Image Search: Iterative User Intent Expansion via Vision-Language ModelProceedings of the ACM on Human-Computer Interaction10.1145/36410198:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641019
Mohammadzadeh BFrançoise JGouiffès MCaramiaux B(2024)Studying Collaborative Interactive Machine Teaching in Image ClassificationProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645204(195-208)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645204
Son KChoi DKim TKim J(2024)Demystifying Tacit Knowledge in Graphic Design: Characteristics, Instances, Approaches, and GuidelinesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642886(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642886
Son KChoi DKim TKim YKim J(2024)GenQuery: Supporting Expressive Visual Search with Generative ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642847(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642847
Chang CHe YDu XYang XXie H(2024)Dynamic Labeling: A Control System for Labeling Styles in Image Annotation TasksHuman Interface and the Management of Information10.1007/978-3-031-60107-1_8(99-118)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-60107-1_8
Huang TYu CShi WWang BYang DZhu YLi ZShi Y(2023)Interaction Proxy ManagerProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109297:3(1-39)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610929
Zhang YTennekes MDe Jong TCurier LCoecke BChen M(2023)Simulation-based Optimization of User Interfaces for Quality-assuring Machine Learning Model PredictionsACM Transactions on Interactive Intelligent Systems10.1145/359455214:1(1-32)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1145/3594552
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten