ScreenCrayons: annotating anything

ABSTRACT

ScreenCrayons is a system for collecting annotations on any type of document or visual information from any application. The basis for the system is a screen capture upon which the user can highlight the relevant portions of the image. The user can define any number of topics for organizing notes. Each topic is associated with a highlighting "crayon." In addition the user can supply annotations in digital ink or text. Algorithms are described that summarize captured images based on the highlight strokes so as to provide overviews of many annotations as well as being able to "zoom in" on particular information about a given note and the context of that note.

REFERENCES

• 1: Annette Adler , Anuj Gujar , Beverly L. Harrison , Kenton O'Hara , Abigail Sellen, A diary study of work-related reading: design implications for digital reading devices, Proceedings of the SIGCHI conference on Human factors in computing systems, p.241-248, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274679]
• 2: Alias, Sketchbook Pro, http://www.alias.com/eng/ products-services/sketchbook_pro/index.shtml
• 3: Baird, H.S., Background structure in document images. International Journal of Pattern Recognition and Artificial Intelligence, 8(5):1013--1030, 1994.
• 4: David Bargeron , Tomer Moscovich, Reflowing digital ink annotations, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642678]
• 5: Thomas M. Breuel, Two Geometric Algorithms for Layout Analysis, Proceedings of the 5th International Workshop on Document Analysis Systems V, p.188-199, August 19-21, 2002
• 6: A. J. Bernheim Brush , David Bargeron , Anoop Gupta , J. J. Cadiz, Robust annotation positioning in digital documents, Proceedings of the SIGCHI conference on Human factors in computing systems, p.285-292, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365117]
• 7: Jay Budzik , Kristian J. Hammond, User interactions with everyday applications as context for just-in-time information access, Proceedings of the 5th international conference on Intelligent user interfaces, p.44-51, January 09-12, 2000, New Orleans, Louisiana, United States [doi>10.1145/325737.325776]
• 8: Jeff Conklin , Michael L. Begeman, gIBIS: a hypertext tool for exploratory policy discussion, Proceedings of the 1988 ACM conference on Computer-supported cooperative work, p.140-152, September 26-28, 1988, Portland, Oregon, United States [doi>10.1145/62266.62278]
• 9: E-Quill, http://www.rocketdownload.com/details/HTML/5337.htm
• 10: Jerry Fails , Dan Olsen, A design tool for camera-based interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642690]
• 11: Ellen Francik, Rapid, integrated design of a multimedia communication system, Proceedings of a workshop on Human-computer interface design : success stories, emerging methods, and real-world context: success stories, emerging methods, and real-world context, p.36-69, June 1995, Boulder, Colorado, United States
• 12: Gene Golovchinsky , Laurent Denoue, Moving markup: repositioning freeform annotations, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.571989]
• 13: Marcel Götze , Stefan Schlechtweg , Thomas Strothotte, The intelligent pen: toward a uniform treatment of electronic documents, Proceedings of the 2nd international symposium on Smart graphics, p.129-135, June 11-13, 2002, Hawthorne, New York [doi>10.1145/569005.569024]
• 14: Frank,G. Halasz, Reflections on NoteCards: seven issues for the next generation of hypermedia systems, Communications of the ACM, v.31 n.7, p.836-852, July 1988 [doi>10.1145/48511.48514]
• 15: Catherine C. Marshall , Frank G. Halasz , Russell A. Rogers , William C. Janssen, Jr., Aquanet: a hypertext tool to hold your knowledge in place, Proceedings of the third annual ACM conference on Hypertext, p.261-275, December 15-18, 1991, San Antonio, Texas, United States [doi>10.1145/122974.123000]
• 16: Catherine C. Marshall , Morgan N. Price , Gene Golovchinsky , Bill N. Schilit, Designing e-books for legal research, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.41-48, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379445]
• 17: Nematode Songbook, http://mgd.nacse.org/hyperSQL/squiggles/songs.html.
• 18: Dan R. Olsen, Jr. , Daniel Boyarski , Thom Verratti , Matthew Phelps , Jack L. Moffett , Edson L. Lo, Generalized pointing: enabling multiagent interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.526-533, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274715]
• 19: Bill N. Schilit , Gene Golovchinsky , Morgan N. Price, Beyond paper: supporting active reading with free form digital ink annotations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.249-256, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274680]
• 20: Lynn D. Wilcox , Bill N. Schilit , Nitin Sawhney, Dynomite: a dynamically organized ink and audio notebook, Proceedings of the SIGCHI conference on Human factors in computing systems, p.186-193, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258700]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Human Factors

Keywords:
annotation, digital ink, image summarization, screen capture