skip to main content
10.1145/2671188.2749312acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
research-article

Insight in Image Collections by Multimedia Pivot Tables

Published: 22 June 2015 Publication History

Abstract

We propose a multimedia analytics solution for getting insight in image collections by extending the powerful method of pivot tables, found in the ubiquitous spreadsheets, to multimedia. Our proposed solution is designed by considering the characteristics of multimedia data as well as insight and provides integral access to visual content through concept detection results, tags, geolocation, and other metadata. We present a set of scenarios of using the pivot tables for a collection of images, tags, and metadata from Flickr. User experiments have been instrumental in realizing the final design presented in this paper. The accompanying video shows the solution in action.

References

[1]
R. Burtner, S. Bohn, and D. Payne. Interactive visual comparison of multimedia data through type-specific views. In SPIE, Visualization and data analysis, 2013.
[2]
E. Chi, J. Riedl, P. Barry, and J. Konstan. Principles for information visualization spreadsheets. Computer Graphics and Applications, July/August 1998.
[3]
N. Chinchor, J. Thomas, P. C. Wong, M. Christel, and W. Ribarsky. Multimedia analysis + visual analytics = multimedia analytics. Computer Graphics and Applications, IEEE, 30(5):52--60, 2010.
[4]
M. Christel. Automated Metadata in Multimedia Information Systems: Creation, Refinement, Use in Surrogates, and Evaluation. Morgan and Claypool Publishers, 2009.
[5]
O. de Rooij, M. Worring, and J. J. van Wijk. Mediatable: Interactive categorization of multimedia collections. IEEE Computer Graphics and Applications, 30(5):42--51, 2010.
[6]
M. Dörk, D. Gruen, C. Williamson, and S. Carpendale. A visual backchannel for large-scale events. IEEE Trans. on Visualization and Computer Graphics, 16(6):1129--1138, 2010.
[7]
M. Dörk, H. Riche, R. Gonzalo, and S. Dumais. Pivotpaths: strolling through faceted information spaces. IEEE Transactions on Visualization and Computer Graphics, 2012.
[8]
D. Fisher, S. Drucker, R. Fernandez, and S. Ruble. Visualizations everywhere: A multiplatform infrastructure for linked visualizations. IEEE Transactions on Visualization and Computer Graphics, 16(6), 2010.
[9]
A. Girgensohn, F. Shipman, T. Turner, and L. Wilcox. Flexible access to photo libraries via time, place, tags, and visual features. In Proceedings of JCDL, 2010.
[10]
S. Gratzl, A. Lex, N. Gehlenborg, H. Pfister, and M. Streit. Lineup: Visual analysis of multi-attribute rankings. IEEE Trans. Vis. Comput. Graph., 19(12):2277--2286, 2013.
[11]
J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, and M. Venkatrao. Data cube: a relational aggregation operator generalizing group-by, cross-tab, and sub-totals. In Data Mining and Knowledge Discovery, 1997.
[12]
B. Höferlin, R. Netzel, M. Höferlin, D. Weiskopf, and G. Heidemann. Inter-active learning of ad-hoc classifiers for video visual analytics. In IEEE Conference on Visual Analytics Science and Technology (VAST), 2012, pages 23--32, 2012.
[13]
B. Jelen and M. Alexander. Pivot Table Data Chrunching. Que Publishing, 2005.
[14]
P. Joia, F. Paulovich, J. C. D. Coimbra, and L. Nonato. Local affine multidimensional projection. IEEE Transactions on Visualization and Computer Graphics, 17(12), 2011.
[15]
S. Kandel, E. Abelson, H. Garcia-Molina, A. Paepcke, and M. Theobald. Photospread: A spreadsheet for managing photos. In Proceedings of SIGCHI, 2008.
[16]
D. Keim, G. Andrienko, J.-D. Fekete, C. Gorg, J. Kohlhammer, and G. Melancon. Visual analytics : Definition, process, and challenges. In Information Visualization, LNCS 4960, 2008.
[17]
H. Luo, J. Fan, S. Satoh, J. Yang, and W. Ribarsky. Integrating multi-modal content analysis and hyperbolic visualization for large-scale news video retrieval and exploration. Signal Processing: Image Communication, 23(7), 2008.
[18]
S. MacNeil and N. Elmqvist. Visualization mosaics for multivariate visual exploration. Computer Graphics Forum, 2013.
[19]
G. P. Nguyen and M. Worring. Interactive access to large image collections using similarity-based visualization. Journal of Visual Languages and Computing, 19(2):203--224, 2008.
[20]
C. North. Toward measuring visualization insight. IEEE Comput. Graph. Appl., 26(3):6--9, 2006.
[21]
D.-S. Ryu, W.-K. Chung, and H.-G. Cho. Photoland: a new image layout system using spatio-temporal information in digital photos. In Proceedings of the 2010 ACM Symposium on Applied Computing, SAC '10, pages 1884--1891, 2010.
[22]
K. Schoeffmann, D. Ahlström, and L. Böszörmenyi. 3D storyboards for interactive visual search. In ICME, 2012.
[23]
C. G. M. Snoek and et.al. Mediamill at TRECVID 2013: Searching concepts, objects, instances and events in video. In Proceedings of TRECVID Workshop, Gaithersburg, USA, 2013.
[24]
C. Stolte, D. Tang, and P. Hanrahan. Polaris: a system for query, analysis, and visualization of multidimensional relations databases. IEEE Transactions on Visualization and Computer Graphics, 8(1), 2002.
[25]
G. Tómasson, H. Sigurthórsson, B. Jónsson, and L. Amsaleg. Photocube: effective and efficient multi-dimensional browsing of personal photo collections. In Proceedings of the ICMR, 2011.
[26]
C. Wang, J. Reese, H. Zhang, J. Tao, Y. Gu, J. Ma, and R. Nemiroff. Similarity-based visualization of large image collections. Information Visualization, 6, 2013.
[27]
C. Ware. Visual Thinking for Design. Morgan Kaufmann, 2008.
[28]
M. Worring and D. C. Koelma. Multimedia pivot tables. In Proceedings of Visual Analytics Science and Technology (VAST), 2013.
[29]
M. Worring, P. Sajda, S. Santini, D. A. Shamma, A. F. Smeaton, and Q. Yang. Where is the user in multimedia retrieval? IEEE Multimedia, 19:6--10, 2012.
[30]
J. Yang, J. Fan, D. Hubball, Y. Gao, H. Luo, W. Ribarsky, and W. M. Semantic image browser: Bridging information visualization with automated intelligent image analysis. In IEEE Symposium on Visual Analytics Science and Technology, 2006.
[31]
J. Zahalka and M. Worring. Towards interactive, intelligent, and integrated multimedia analytics. In IEEE Conference on Visual Analytics Science and Technology, 2014.
[32]
E. Zavesky, S.-F. Chang, and C.-C. Yang. Visual islands: Intuitive browsing of visual search results. In Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, pages 617--626, 2008.

Cited By

View all
  • (2021)Texture Browser: Feature‐based Texture ExplorationComputer Graphics Forum10.1111/cgf.1429240:3(99-109)Online publication date: 29-Jun-2021
  • (2019)A Semantic-Based Method for Visualizing Large Image CollectionsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.283548525:7(2362-2377)Online publication date: 1-Jul-2019
  • (2018)Integration of Exploration and Search: A Case Study of the M $$^3$$ ModelMultiMedia Modeling10.1007/978-3-030-05710-7_13(156-168)Online publication date: 8-Dec-2018
  • Show More Cited By

Index Terms

  1. Insight in Image Collections by Multimedia Pivot Tables

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval
    June 2015
    700 pages
    ISBN:9781450332743
    DOI:10.1145/2671188
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 22 June 2015

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. exploration
    2. information visualization
    3. visual analytics

    Qualifiers

    • Research-article

    Conference

    ICMR '15
    Sponsor:

    Acceptance Rates

    ICMR '15 Paper Acceptance Rate 48 of 127 submissions, 38%;
    Overall Acceptance Rate 254 of 830 submissions, 31%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)2
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2021)Texture Browser: Feature‐based Texture ExplorationComputer Graphics Forum10.1111/cgf.1429240:3(99-109)Online publication date: 29-Jun-2021
    • (2019)A Semantic-Based Method for Visualizing Large Image CollectionsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.283548525:7(2362-2377)Online publication date: 1-Jul-2019
    • (2018)Integration of Exploration and Search: A Case Study of the M $$^3$$ ModelMultiMedia Modeling10.1007/978-3-030-05710-7_13(156-168)Online publication date: 8-Dec-2018
    • (2016)Multimedia Pivot Tables for Multimedia Analytics on Image CollectionsIEEE Transactions on Multimedia10.1109/TMM.2016.261438018:11(2217-2227)Online publication date: 1-Nov-2016
    • (2016)Ten Research Questions for Scalable Multimedia AnalyticsMultiMedia Modeling10.1007/978-3-319-27674-8_26(290-302)Online publication date: 1-Jan-2016

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media