A perceptually-supported sketch editor

ABSTRACT

The human visual system makes a great deal more of images than the elemental marks on a surface. In the course of viewing, creating, or editing a picture, we actively construct a host of visual structures and relationships as components of sensible interpretations. This paper shows how some of these computational processes can be incorporated into perceptually-supported image editing tools, enabling machines to better engage users at the level of their own percepts. We focus on the domain of freehand sketch editors, such as an electronic whiteboard application for a pen-based computer. By using computer vision techniques to perform covert recognition of visual structure as it emerges during the course of a drawing/editing session, a perceptually supported image editor gives users access to visual objects as they are perceived by the human visual system. We present a flexible image interpretation architecture based on token grouping in a multiscale blackboard data structure. This organization supports multiple perceptual interpretations of line drawing data, domain-specific knowledge bases for interpretable visual structures, and gesture-based selection of visual objects. A system implementing these ideas, called PerSketch, begins to explore a new space of WYPIWYG (What You Perceive Is What You Get) image editing tools.

REFERENCES

• 1: S. H. Joseph , T. P. Pridmore, Knowledge-Directed Interpretation of Mechanical Engineering Drawings, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.9, p.928-940, September 1992 [doi>10.1109/34.161351 ]
• 2: Lee, S. Recognizing Hand-Drawn Electrical Circuit Symbols with Attributed Graph Matching. In H. S. Baird, H. Bunke, and K. Yamamoto (eds.), Structured Document Image Analysis. Springer-Verlag, New York, 1992.
• 3: Mart, D. Early Processing of Visual Information. Phil. Trans. R. Soc. Lond., B 275 (1976), 483-519.
• 4: R. Mohan , R. Nevatia, Using Perceptual Organization to Extract 3D Structures, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.11 n.11, p.1121-1139, November 1989 [doi>10.1109/34.42852 ]
• 5: Montalvo, F. Diagram Understanding: The Symbolic Descriptions Behind the Scenes. In T. ichikawa, E. Jungert, and R. Korfhage (eds.), Visual Languages and Applications. Plenum Press, New York, 1990.
• 6: Moran, T. Deformalizing Computer and Communication Systems. Position Paper for the InterCHI93 Research Symposium. 1993.
• 7: Okazaki, S., and Tsuji, Y. An Adaptive Recognition Method for Line Drawings Using Construction Rules. NEC Research and Development Journal, 92 (1989).
• 8: Elin Rønby Pedersen , Kim McCall , Thomas P. Moran , Frank G. Halasz, Tivoli: an electronic whiteboard for informal workgroup meetings, Proceedings of the SIGCHI conference on Human factors in computing systems, p.391-398, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169309]
• 9: S. Sarkar , K. L. Boyer, Integration, Inference, and Management of Spatial Information Using Bayesian Networks: Perceptual Organization, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.15 n.3, p.256-274, March 1993 [doi>10.1109/34.204907 ]
• 10: Sato, T., and Tojo, A. Recognition and Understanding of Hand-Drawn Diagrams. Proc. 6th International Conference on Pattern Recognition. IEEE Computer Society Press, New Jersey, 1982.
• 11: E. Saund, Symbolic Construction of a 2-D Scale-Space Image, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.8, p.817-830, August 1990 [doi>10.1109/34.57672 ]
• 12: Eric Saund, Identifying salient circular arcs on curves, CVGIP: Image Understanding, v.58 n.3, p.327-337, Nov. 1993 [doi>10.1006/ciun.1993.1045 ]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.4 Graphics Utilities
          Subjects: Graphics editors

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision

General Terms:
Algorithms, Design

Keywords:
PerSketch, computer vision, drawing tools, gestures, graphics editing, image editing, interactive graphics, machine vision, pen computing, perceptual grouping, perceptual organization, scale space blackboard WYSIWYG, sketch tools, token grouping