Margaret M. Burnett
Abstract
In the past, attempts to extend the spreadsheet paradigm to support graphical objects, such as colored circles or user-defined graphical types, have led to approaches featuring either a direct way of creating objects graphically or strong compatibility with the spreadsheet paradigm, but not both. This inability to conveniently go beyond numbers and strings without straying outside the spreadsheet paradigm has been a limiting factor in the applicability of spreadsheet languages. In this article we present graphical definitions, an approach that removes this limitation, allowing both simple and complex graphical objects to be programmed directly using direct manipulation and gestures, in a manner that fits seamlessly within the spreadsheet paradigm. We also describe an empirical study, in which subjects programmed such objects faster and with fewer errors using this approach than when using a traditional approach to formula specification. Because the approach is expressive enough to be used with both built-in and user-defined types, it allows the directness of demonstrational and spreadsheet techniques to be used in programming a wider range of applications than has been possible before.
- ATWOOD, J., BURNETT, M., WALPOLE, R., WILCOX, E., AND YANG, S. 1996. Steering programs via time travel. In Proceedings of the 1996 IEEE Symposium on Visual Languages (Boulder, Colo., Sept. 3-6). IEEE Computer Society Press, Los Alamitos, Calif., 4-11. Google Scholar
- BAROTH, E. AND HARTSOUGH, C. 1995. Visual programming in the real world. In Visual Object-Oriented Programming: Concepts and Environments, M. Burnett, A. Goldberg, and T. Lewis, Eds. Prentice-Hall, Englewood Cliffs, N.J. Google Scholar
- BURNETT, M. AND AMBLER, A. 1992. A declarative approach to event-handling in visual programming languages. In Proceedings of the 1992 IEEE Workshop on Visual Languages (Seattle, Wash., Sept.). IEEE Computer Society Press, Los Alamitos, Calif., 34-40.Google Scholar
- BURNETT, M. AND AMBLER, A. 1994. Interactive visual data abstraction in a declarative visual programming language. J. Vis. Lang. Comput. 5, I (Mar.). 29-60.Google Scholar
- CARLSON, P., BURNETT, M., AND CADIZ, J. J. 1996. A seamless integration of algorithm animation into a visual programming language. In Proceedings of Advanced Visual Interfaces '96 (Gubbio, Italy, May 27-29). ACM, New York, 194-202. Google Scholar
- COOK, C., BURNETT, M., AND BOOM, D. 1997. A bug's eye view of immediate visual feedback in direct-manipulation programming systems. In Empirical Studies of Programmers: Proceedings of the 7th Workshop (Alexandria, Va., Oct. 24-26). ACM, New York, 20-41. Google Scholar
- CYPHER, A., ED. 1993. Watch What I Do: Programming by Demonstration. MIT Press, Cambridge, Mass. Google Scholar
- DUPUIS, C. AND BURNETT, M. 1997. An animated Turing machine simulator in Forms/3. Tech. Rep. TR 97-60-08, Dept. of Computer Science, Oregon State Univ., Corvallis, Ore. Google Scholar
- GOTTFRIED, H. AND BURNETT, M. 1997. Programming complex objects in spreadsheets: An empirical study comparing textual formula entry with direct manipulation and gestures. In Empirical Studies of Programmers: Proceedings of the 7th Workshop (Alexandria, Va., Oct. 24-26). ACM, New York, 42-68. Google Scholar
- GREEN, T. AND PETRE, M. 1996. Usability analysis of visual programming environments: A "cognitive dimensions" framework. J. Vis. Lang. Comput. 7, 2 (June). 131-174.Google Scholar
- GREEN, T., PETRE, M., AND BELLAMY, R. 1991. Comprehensibility of visual and textual programs: A test of superlativism against the "match-mismatch" conjecture. In Empirical Studies of Programmers: Proceedings of the 4th Workshop (New Brunswick, N.J., Dec. 7-9). Ablex, Norwood, N.J., 121-146.Google Scholar
- GROSS, M. AND DO, E. 1996. Ambiguous intentions: A paper-like interface for creative design. In Proceedings of the ACM Symposium on User Interface Software and Technology (Seattle, Wash., Nov. 6-9). ACM, New York, 183-192. Google Scholar
- HUDSON, S. 1994. User interface specification using an enhanced spreadsheet model. ACM Trans. Graph. 13, 3 (July). 209-239. Google Scholar
- HUGHES, C. AND MOSHELL, J. 1990. Action Graphics: A spreadsheet-based language for animated simulation. In Visual Languages and Applications, T. Ichikawa, E. Jungert, and R. Korfhage, Eds. Plenum Publishing, New York, 203-235.Google Scholar
- HUTCHINS, E., HOLLAN, J., AND NORMAN, D. 1986. Direct manipulation interfaces. In User Centered System Design: New Perspectives on Human-Computer Interaction, D. Norman and S. Draper, Eds. Lawrence Erlbaum, Hillsdale, N.J., 87-124.Google Scholar
- KAY, A. 1984. Computer software. Sci. Am. 250, 5 (Sept.). 53-59.Google Scholar
- KURLANDER, D. 1993. Chimera: Example-based graphical editing. In Watch What I Do: Programming by Demonstration, A. Cypher, Ed. MIT Press, Cambridge, Mass. Google Scholar
- LANDAY, J. AND MYERS, B. 1993. Extending an existing user interface toolkit to support gesture recognition. In Adjunct Proceedings INTERCHI '93 (Amsterdam, The Netherlands, Apr. 24-29). ACM, New York, 91-92. Google Scholar
- LAWRENCE, A., BADRE, A., AND STASKO, J. 1994. Empirically evaluating the use of animations to teach algorithms. In Proceedings of the 1994 IEEE Symposium on Visual Languages (St. Louis, Mo., Oct. 4-7). IEEE Computer Society Press, Los Alamitos, Calif., 48-54.Google Scholar
- LEOPOLD, J. AND AMBLER, A. 1997. Keyboardless visual programming using voice, handwriting, and gesture. In Proceedings of the 1997 IEEE Symposium on Visual Languages (Capri, Italy, Sept. 23-26). IEEE Computer Society Press, Los Alamitos, Calif., 28-35. Google Scholar
- LEWIS, C. 1990. NoPumpG: Creating interactive graphics with spreadsheet machinery. In Visual Programming Environments: Paradigms and Systems, E. Glinert, Ed. IEEE Computer Society Press, Los Alamitos, Calif., 526-546.Google Scholar
- LIEBERMAN, H. 1993. Mondrian: A teachable graphical editor. In Watch What I Do: Programming by Demonstration, A. Cypher, Ed. MIT Press, Cambridge, Mass. Google Scholar
- MCCARTNEY, T., GOLDMAN, K., AND STAFF, D. 1995. EUPHORIA: End-user construction of direct manipulation user interfaces for distributed applications. Softw. Concepts Tools 16, 4, 147-159.Google Scholar
- MIYASHITA, K., MATSUOKA, S., TAKAHASHI, S., YONEZAWA, A., AND KAMADA, T. 1992. Declarative programming of graphical interfaces by visual examples. In Proceedings of the ACM Symposium on User Interface Software and Technology (Monterey, Calif., Nov. 15-18). ACM, New York, 107-116. Google Scholar
- MIYASHITA, K., MATSUOKA, S., TAKAHASHI, S., AND YONEZAWA, A. 1994. Iterative generation of graphical user interfaces by multiple visual examples. In Proceedings of the ACM Symposium on User Interface Software and Technology (Marina del Rey, Calif., Nov. 2-4). ACM, New York, 85-94. Google Scholar
- MODUGNO, F., CORBETT, A., AND MYERS, B. 1996. Evaluating program representation in a demonstrational visual shell. In Empirical Studies of Programmers: Proceedings of the 6th Workshop (Alexandria, Va., Jan.). Ablex, Norwood, N.J., 131-146. Google Scholar
- MOHER, T., MAK, D., BLUMENTHAL, B., AND LEVENTHAL, L. 1993. Comparing the comprehensibility of textual and graphical programs: The case of Petri nets. In Empirical Studies of Programmers: Proceedings of the 5th Workshop (Palo Alto, Calif.). Ablex, Norwood, N.J., 137-161.Google Scholar
- MYERS, B. 1991. Graphical techniques in a spreadsheet for specifying user interfaces. In Proceedings of CHI '91: Conference on Human Factors in Computing Systems (New Orleans, La., Apr. 28-May 2). ACM, New York, 243-249. Google Scholar
- MYERS, B., GUISE, D., DANNENBERG, R., WANDER ZANDEN, B., KOSmE, D., PERVIN, E., MIcKIS~, A., AND MARCHAL, P. 1990. Garnet: Comprehensive support for graphical, highly interactive user interfaces. Computer 23, 11 (Nov.). 71-85. Google Scholar
- NARDI, B. 1993. A Small Matter of Programming: Perspectives on End User Computing. MIT Press, Cambridge, Mass. Google Scholar
- PANDEY, R. AND BURNETT, M. 1993. Is it easier to write matrix manipulation programs visually or textually? An empirical study. In Proceedings of the 1993 IEEE Symposium on Visual Languages (Bergen, Norway, Aug. 24-27). IEEE Computer Society Press, Los Alamitos, Calif., 344-351.Google Scholar
- PETRE, M. 1995. Why looking isn't always seeing: Readership skills and graphical programming. Commun. ACM 38, 6 (June), 33-44. Google Scholar
- REPENNING, A. AND AMBACH, J. 1996. Tactile programming: A unified manipulation paradigm supporting program comprehension, composition and sharing. In Proceedings of the 1996 IEEE Symposium on Visual Languages (Boulder, Colo., Sept. 3-6). IEEE Computer Society Press, Los Alamitos, Calif., 102-109. Google Scholar
- SHNEIDERMAN, B. 1983. Direct manipulation: A step beyond programming languages. Computer 16, 8 (Aug.), 57-69.Google Scholar
- SMEDLEY, T., COX, P., AND BYRNE, S. 1996. Expanding the utility of spreadsheets through the integration of visual programming and user interface objects. In Proceedings of Advanced Visual Interfaces '96 (Gubbio, Italy, May 27-29). ACM, New York, 148-155. Google Scholar
- SMITH, D., CYPHER, A., AND SPOHRER, J. 1994. KidSim: Programming agents without a programming language. Commun. ACM 37, 7 (July), 54-67. Google Scholar
- STASKO, J., BADRE, A., AND LEWIS, C. 1993. Do algorithm animations assist learning? An empirical study and analysis. In Proceedings of INTERCHI '93 (Amsterdam, The Netherlands, Apr. 24-29). ACM, New York, 61-66. Google Scholar
- WANDER ZANDEN, B. AND MYERS, B. 1995. Demonstrational and constraint-based technologies for pictorially specifying application objects and behaviors. ACM Trans. Comput. Hum. Interact. 2, 4 (Dec.), 308-356. Google Scholar
- VAN ZEE, P., BURNETT, M., AND CHESIRE, M. 1996. Retire Superman: Handling exceptions seamlessly in declarative visual programming languages. In Proceedings of the 1996 IEEE Symposium on Visual Languages (Boulder, Colo., Sept. 3-6). IEEE Computer Society Press, Los Alamitos, Calif., 222-230. Google Scholar
- WHITLEY, K. 1997. Visual programming languages and the empirical evidence for and against. J. Vis. Lang. Comput. 8, I (Feb.), 109-142.Google Scholar
- WILCOX, E., ATWOOD, J., BURNETT, M., CADIZ, J., AND COOK, C. 1997. Does continuous visual feedback aid debugging in direct-manipulation programming systems? In Proceedings of CHI '97: Conference on Human Factors in Computing Systems (Atlanta, Ca., Mar. 22-27). ACM, New York, 258-265. Google Scholar
- WILDE, N. AND LEWIS, C. 1990. Spreadsheet-based interactive graphics: From prototype to tool. In Proceedings of the CHI '90 Conference on Human Factors in Computing Systems (Seattle, Wash., Apr. 1-5). ACM, New York, 153-159. Google Scholar
- WILDE, N. 1993. A WYSIWYC (what you see is what you compute) spreadsheet. In Proceedings of the 1993 IEEE Symposium on Visual Languages (Bergen, Norway, Aug. 24-27). IEEE Computer Society Press, Los Alamitos, Calif., 72-76.Google Scholar
- YANG, S. AND BURNETT, M. 1994. From concrete forms to generalized abstractions through perspective-oriented analysis of logical relationships. In Proceedings of the 1994 IEEE Symposium on Visual Languages (St. Louis, Mo., Oct. 4-7). IEEE Computer Society Press, Los Alamitos, Calif., 6-14.Google Scholar
- YANG, S., BURNETT, M., DEKOVEN, E., AND ZLOOF, M. 1997. Representation design benchmarks: A design-time aid for VPL navigable static representations. J. Vis. Lang. Comput. 8, 5/6 (Oct./Dec.), 563-599.Google Scholar
Index Terms
- Graphical definitions: expanding spreadsheet languages through direct manipulation and gestures
Recommendations
Graphical definitions: making spreadsheets visual through direct manipulation and gestures
VL '97: Proceedings of the 1997 IEEE Symposium on Visual Languages (VL '97)Until now, attempts to extend the one way constraint evaluation model of the spreadsheet paradigm to support complex objects, such as colored circles or user defined types, have led to approaches featuring either a direct way of creating objects ...
Informal prototyping of continuous graphical interactions by demonstration
UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technologyInformal prototyping tools have shown great potential in facilitating the early stage design of user interfaces. How-ever, continuous interactions, an important constituent of highly interactive interfaces, have not been well supported by previous ...
Reactile: Programming Swarm User Interfaces through Direct Physical Manipulation
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsWe explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of ...
Reviews
James Curtis Miller
The authors introduce their research into adding complex graphical definitions and manipulations (allowing operations to be invoked using gestures) to a spreadsheet paradigm. They first explain what they mean by a spreadsheet language and briefly introduce the history of the problem, including a fairly brief but extensive summary of related work. They then introduce their work by means of examples—one fairly simple and one more complex. They explain how a user produces a graphical definition and defines operations by means of gestures, which in this case are implemented using a mouse. The paper contains sections that cover both syntactic and semantic issues. I found the section on semantic issues interesting. The authors then demonstrate that their ideas satisfy the earlier definition of a spreadsheet language. This interesting paper appears to introduce a new and potentially useful method of defining and using graphical objects within a spreadsheet. I found the figures a bit difficult to make out, but the paper is clear and easy to follow. The summary of related work and the references are excellent. The paper is accessible and should interest almost anyone who uses graphics or spreadsheets.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments