Smart bookmarks: automatic retroactive macro recording on the web

ABSTRACT

We present a new web automation system that allows users to create a smart bookmark, consisting of a starting URL plus a script of commands that returns to a particular web page or state of a web application. A smart bookmark can be requested for any page, and the necessary commands are automatically extracted from the user's interaction history. Unlike other web macro recorders, which require the user to start recording before navigating to the desired page, smart bookmarks are generated retroactively, after the user has already reached a page, and the starting point of the macro is found automatically. Smart bookmarks have a rich graphical visualization that combines textual commands, web page screenshots, and animations to explain what the bookmark does. A bookmark's script consists of keyword commands, interpreted without strict reliance on syntax, allowing bookmarks to be easily edited and shared.

REFERENCES

• 1: Anupam, V., Freire, J., Kumar, B., and Lieuwen, D. "Automating web navigation with the WebVCR." Proc. WWW 2000, pp. 503--517.
• 2: Michael Bolin , Matthew Webber , Philip Rha , Tom Wilson , Robert C. Miller, Automation and customization of rendered web pages, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095062]
• 3: Amelie Marian, Detecting Changes in XML Documents, Proceedings of the 18th International Conference on Data Engineering, p.41, February 26-March 01, 2002
• 4: Mira Dontcheva , Steven M. Drucker , Geraldine Wade , David Salesin , Michael F. Cohen, Summarizing personal web browsing sessions, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166273]
• 5: Jun Fujima , Aran Lunzer , Kasper Hornbæk , Yuzuru Tanaka, Clip, connect, clone: combining application elements to build custom interfaces for information access, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029664]
• 6: David F. Huynh , Robert C. Miller , David R. Karger, Enabling web browsers to augment web sites' filtering and sorting functionalities, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166274]
• 7: Shaun Kaasten , Saul Greenberg, Integrating back, history and bookmarks in web browsers, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634291]
• 8: David Kurlander , Steven Feiner, A history-based macro by example system, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.99-106, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142633]
• 9: Greg Little , Robert C. Miller, Translating keyword commands into executable code, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166275]
• 10: Greg Little , Tessa A. Lau , Allen Cypher , James Lin , Eben M. Haber , Eser Kandogan, Koala: capture, share, automate, personalize business processes on the web, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240767]
• 11: Francesmary Modugno , Albert T. Corbett , Brad A. Myers, Graphical representation of programs in a demonstrational visual shell—an empirical evaluation, ACM Transactions on Computer-Human Interaction (TOCHI), v.4 n.3, p.276-308, Sept. 1997 [doi>10.1145/264645.264659]
• 12: Thomas A. Phelps , Robert Wilensky, Robust Hyperlinks Cost Just Five Words Each, University of California at Berkeley, Berkeley, CA, 2000
• 13: Safonov, A., Konstan, J., and Carlis, J. "Beyond Hard-to-Reach Pages: Interactive, Parametric Web Macros." Proc. Human Factors and the Web, 2001.
• 14: Safonov, A., Konstan, J., and Carlis, J. "End-user Web Automation: Challenges, Experiences, Recommendations." Proc. WebNet 2001, 1077--1085.
• 15: Atsushi Sugiura , Yoshiyuki Koseki, Internet scrapbook: automating Web browsing tasks by demonstration, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.9-18, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288395]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design

General Terms:
Algorithms, Design, Human Factors

Keywords:
browsers, macro recording, web automation