Mnemonic rendering: an image-based approach for exposing hidden changes in dynamic displays

ABSTRACT

Managing large amounts of dynamic visual information involves understanding changes happening out of the user's sight. In this paper, we show how current software does not adequately support users in this task, and motivate the need for a more general approach. We propose an image-based storage, visualization, and implicit interaction paradigm called mnemonic rendering that provides better support for handling visual changes. Once implemented on a system, mnemonic rendering techniques can benefit all applications. We explore its rich design space and discuss its expected benefits as well as limitations based on feedback from users of a small-screen and a wall-size prototype.

References

1. Banbury, B. and Tremblay, S., eds. (2004). A cognitive approach to situation awareness: theory and application. Ashgate.
2. Baudisch, P., Cutrell, E., and Robertson, G. (2003). High-density cursor: A visualization technique that helps users keep track of fast-moving mouse cursors. Interact. p. 236--243.
3. Baudisch, P. and Gutwin, C. (2004). Multiblending: displaying overlapping windows simultaneously without the drawbacks of alpha blending. ACM CHI Conference. p. 367--374.
4. Bier, E., Stone, M., Pier, K., Buxton, W., and DeRose, T. (1993). Toolglass and Magic Lenses: The see-through interface. ACM SIGGRAPH. p. 73--80.
5. Chang, B., Mackinlay, J., Zellweger, P., and Igarashi, T. (1998). A negotiation architecture for fluid documents. ACM UIST Symposium on User Interface Software and Technology. p. 123--132.
6. Chang, B. and Ungar, D. (1993). Animation: From cartoons to the user interface. ACM UIST Symposium on User Interface Software and Technology. p. 45--55.
7. Chapuis, O. and Roussel, N. (2005). Metisse is not a 3D desktop! ACM Symposium on User Interface Software and Technology. p. 13--22.
8. Cockburn, A., Greenberg, S., Jones, S., McKenzie, B., and Moyle, M. (2003). Improving web page revisitation: Analysis, design and evaluation. Special Issue on Web Navigation Skills, IT & Society, 3(1). p. 159--183.
9. Edwards, W. K., Igarashi, T., LaMarca, A., and Mynatt, E. D. (2000). A temporal model for multi-level undo and redo. ACM UIST Symposium on User Interface Software and Technology. p. 31--40.
10. Endsley, M., Bolté, B., and Jones, D. (2003). Designing for situation awareness: CRC.
11. Freeman, E. and Fertig, S. (1995). Lifestreams: Organizing your electronic life. AAAI Fall Symposium: AI Applications in Knowledge Navigation and Retrieval.
12. Grudin, J. (2001). Partitioning digital worlds: focal and peripheral awareness in multiple monitor use. ACM CHI Conference. p. 458--465.
13. Hunt, A. R. and Andkingstone, A. (2003). Covert and overt voluntary attention: linked or independent? Cognitive Brain Research, 18. p. 102--105.
14. Hutchings, D. R., Smith, G., Meyers, B., Czerwinski, M., and Robertson, G. (2004). Display space usage and window management operation comparisons between single monitor and multiple monitor users. ACM AVI Conference on Advanced Visual Interfaces. p. 32--39.
15. Hutchings, D. R. and Stasko, J. (2004). Revisiting display space management: understanding current practice to inform next-generation design. Graphics Interface Conference. p. 127--134.
16. Itti, L., Rees, G., and Tsotsos, J., eds. (2005). Neurobiology of attention. Elsevier.
17. Jacob, R. J. K. (1993). Eye-movement-based human-computer interaction techniques: Toward non-command interfaces. In Advances in Human-Computer Interaction, D. Hix, Editor. Ablex. chapter 6. p. 151--190.
18. Jaynes, C., Webb, S., Steele, R. M., Brown, M., and Seales, W. B. (2001). Dynamic shadow removal from front projection displays. IEEE Visualization. p. 175--182.
19. Kawasaki, Y. and Igarashi, T. (2004). Regional undo for spreadsheets. ACM UIST Symposium on User Interface Software and Technology. Demo abstract.
20. Kurlander, D. and Feiner, S. (1992). A history-based macro by example system. ACM UIST Symposium on User Interface Software and Technology. p. 99--106.
21. Mertz, C., Chatty, S., and Vinot, J. (2000). Pushing the limits of ATC user interface design beyond S&M interaction: the DigiStrips experience. 3rd USA/Europe Air Traffic Management R&D Seminar.
22. Moscovich, T. and Hughes, J. F. (2004). Navigating documents with the virtual scroll ring. ACM UIST Symposium on User Interface Software and Technology. p. 57--60.
23. Rekimoto, J. (1999). Time-machine computing: a time-centric approach for the information environment. ACM UIST Symposium on User Interface Software and Technology. p. 45--54.
24. Renaud, K. (2000). Expediting rapid recovery from interruptions by providing a visualization of application activity. OZCHI. p. 348--355.
25. Rensink, R. A. (2002). Change detection. Annual Review of Psychology, 53. p. 245--277.
26. Ringel, M. (2003). When one isn't enough: An analysis of virtual desktop usage strategies and their implications for design. ACM CHI Conference (Ext. Abstracts). p. 762--763.
27. Salembier, P. and Marques, F. (1999). Region-based representations of image and video: Segmentation tools for multimedia services. IEEE Transactions on Circuits and Systems for Video Technology, 9(8). p. 1147--1169.
28. Skopik, A. and Gutwin, C. (2005). Improving revisitation in fisheye views with visit wear. ACM CHI Conference. p. 771--780.
29. Smith, G. M. and schraefel, m. c. (2004). The radial scroll tool: scrolling support for stylus- or touch-based document navigation. ACM UIST Symposium on User Interface Software and Technology. p. 53--56.
30. Tam, J. and Greenberg, S. (In press - accepted 2005). A framework for asynchronous change awareness in collaborative documents and workspaces. International Journal of Human Computer Studies.
31. Tomitsch, M. (2003). Trends and evolution of window interfaces. Thesis, University of Technology: Vienna.
32. Turk, M. (2004). Computer vision in the interface. Communications of the ACM, 47(1). p. 60--67.
33. Varakin, D. A. and Levin, D. T. (2004). Unseen and unaware: Implications of recent research on failures of visual awareness for human-computer interface design. Human-Computer Interaction, 19. p. 389--422.
34. Venolia, G. D., Dabbish, L., Cadiz, J. J., and Gupta, A. (2001). Supporting email Workflow. Tech Report MSR-TR-2001-88. Microsoft Research.
35. Vertegaal, R., Mamuji, A., Sohn, C., and Cheng, D. Media eyepliances: using eye tracking for remote control focus selection of appliances. ACM CHI Conference (Extended Abstracts). p. 1861--1864.
36. Wang, X. and Jin, J. (2001). A quantitative analysis for decomposing visual signal of the gaze displacement. Pan-Sydney area workshop on visual informtaion processing conference on visual information processing 2001. p. 153--159.
37. Wexelblat, A. and Maes, P. (1999). Footprints: History-rich tools for information foraging. ACM CHI Conference. p. 279--277.
38. Whittaker, S. and Sidner, C. (1996). Email overload: exploring personal information management of email. ACM CHI Conference. p. 276--283.
39. Woods, D. D. and Watts, J. C. (1997). How not to have to navigate through too many displays. In Handbook of Human-Computer Interaction. Elsevier Science.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Graphical user interfaces