Article

Augmentation-based learning: combining observations and user edits for programming-by-demonstration

Authors:

Daniel Oblinger,

Vittorio Castelli,

Lawrence BergmanAuthors Info & Claims

IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces

Pages 202 - 209

https://doi.org/10.1145/1111449.1111494

Published: 29 January 2006 Publication History

Get Access

Abstract

In this paper we introduce a new approach to Programming-by-Demonstration in which the user is allowed to explicitly edit the procedure model produced by the learning algorithm while demonstrating the task. We describe a new algorithm, Augmentation-Based Learning, that supports this approach by considering both demonstrations and edits as constraints on the hypothesis space, and resolving con icts in favor of edits.

References

[1]

Y. Bengio and P. Frasconi. Input-Output HMM's for sequence processing. IEEE Trans. Neural Networks, 7(5):1231--1249, Sept. 1996.

Digital Library

Google Scholar

[2]

P. Chou. Optimal partitioning for classification and regression trees. IEEE Trans. Pattern Analysis and Machine Intelligence, 13(4):340--354, Apr. 1991.

Digital Library

Google Scholar

[3]

A. Cypher. Eager: Programming repetitive tasks by demonstration. In A. Cypher, editor, Watch What I Do: Programming by Demonstration, pages 205--217. MIT Press, Cambridge, MA, 1993.

Digital Library

Google Scholar

[4]

A. Cypher, editor. Watch what I do: Programming by demonstration. MIT Press, Cambridge, MA, 1993.

Digital Library

Google Scholar

[5]

J. Lafferty, A. McCallum, and F. Pereira. Conditional random field: Probabilistic models for segmenting and labeling sequence data. In Proc. Int. Conf. on Machine Learning, pages 282--289, 2001.

Digital Library

Google Scholar

[6]

T. Lau, L. Bergman, V. Castelli, and D. Oblinger. Sheepdog: Learning procedures for technical support. In Proc. 2004 Int. Conf. on Intelligent User Interfaces, pages 106--116, 2004.

Digital Library

Google Scholar

[7]

T. Lau, P. Domingos, and D. Weld. Version space algebra and its application to programming by demonstration. In Proc. Seventeenth Int. Conf. on Machine Learning, pages 527--534, June 2000. IBMT.J.

Digital Library

Google Scholar

[8]

T. Lau, P. Domingos, and D. Weld. Learning programs from traces using version space algebra. In Proc. 2nd Int. Conf. on Knowledge Capture, 2003.

Digital Library

Google Scholar

[9]

H. Lieberman, editor. Your Wish is My Command: Giving Users the Power to Instruct their Software. Morgan Kaufmann, 2001.

Digital Library

Google Scholar

[10]

R. G. McDaniel and B. A. Myers. Building applications using only demonstration. In Proc. 1998 Int. Conf. on Intelligent User Interfaces, pages 282--289, 1998.

Digital Library

Google Scholar

[11]

N. Medvivovic, A. Egyed, and D. Rosenblum. Round-trip software engineering using uml: From architecture to design and back,. In Proc. 2nd Workshop Object-Oriented Reengineering (WOOR 99), pages 1--8, Monterey, CA, USA, 1999.

Google Scholar

[12]

D. Oblinger, V. Castelli, T. Lau, and L. Bergman. Similarity-based alignment and generalization. In Proc. Sixteenth Europ. Conf. on Machine Learning, page To appear, October 2005.

Digital Library

Google Scholar

[13]

M. Prabaker, L. Bergman, and V. Castelli. An evaluation of using programming by demonstration and guided walkthrough techniques for authoring and following documentation. In submitted to CHI, 2006.

Digital Library

Google Scholar

Cited By

View all

Gervasio MHaines WMorley DLee TOverholtzer CSaadati SSpaulding APu PPazzani MAndré ERiecken D(2011)How to serve soupProceedings of the 16th international conference on Intelligent user interfaces10.1145/1943403.1943458(331-334)Online publication date: 13-Feb-2011
https://dl.acm.org/doi/10.1145/1943403.1943458
Fritz CGil YPu PPazzani MAndré ERiecken D(2011)A formal framework for combining natural instruction and demonstration for end-user programmingProceedings of the 16th international conference on Intelligent user interfaces10.1145/1943403.1943440(237-246)Online publication date: 13-Feb-2011
https://dl.acm.org/doi/10.1145/1943403.1943440
Castelli VBergman LLau TOblinger D(2010)Sheepdog, parallel collaborative programming-by-demonstrationKnowledge-Based Systems10.1016/j.knosys.2009.06.00823:2(94-109)Online publication date: 1-Mar-2010
https://dl.acm.org/doi/10.1016/j.knosys.2009.06.008
Show More Cited By

Index Terms

Augmentation-based learning: combining observations and user edits for programming-by-demonstration
1. Computing methodologies
  1. Machine learning

Recommendations

Augmentation-Based Learning combining observations and user edits for Programming-by-Demonstration

In this paper, we introduce a new approach to Programming-by-Demonstration in which the author is allowed to explicitly edit the procedure model produced by the learning algorithm while demonstrating the task. We describe Augmentation-Based Learning, a ...
Distributed augmentation-based learning: a learning algorithm for distributed collaborative programming-by-demonstration
IUI '07: Proceedings of the 12th international conference on Intelligent user interfaces

The learning algorithms used in Programming-by-Demonstration (PBD) are either on-line and incremental or off-line and batch. Neither category is entirely suitable for capturing know-how from demonstrations in a distributed, collaborative environment, ...
Sheepdog, parallel collaborative programming-by-demonstration

We introduce parallel collaborative programming-by-demonstration (PBD) as a principled approach to capturing knowledge on how to perform computer-based procedures by independently recording multiple experts executing these tasks and combining the ...

Comments

Information & Contributors

Information

Published In

IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces

January 2006

392 pages

ISBN:1595932879

DOI:10.1145/1111449

General Chairs:
Ernest Edmonds
University of Technology Sydney, Australia
,
Doug Riecken
IBM Watson Research, USA
,
Program Chairs:
Cécile L. Paris
CSIRO ICT Centre, Australia
,
Candace L. Sidner
MERL, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 January 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IUI06

Sponsor:

IUI06: 11th International Conference on Intelligent User Interfaces

January 29 - February 1, 2006

Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
464
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Gervasio MHaines WMorley DLee TOverholtzer CSaadati SSpaulding APu PPazzani MAndré ERiecken D(2011)How to serve soupProceedings of the 16th international conference on Intelligent user interfaces10.1145/1943403.1943458(331-334)Online publication date: 13-Feb-2011
https://dl.acm.org/doi/10.1145/1943403.1943458
Fritz CGil YPu PPazzani MAndré ERiecken D(2011)A formal framework for combining natural instruction and demonstration for end-user programmingProceedings of the 16th international conference on Intelligent user interfaces10.1145/1943403.1943440(237-246)Online publication date: 13-Feb-2011
https://dl.acm.org/doi/10.1145/1943403.1943440
Castelli VBergman LLau TOblinger D(2010)Sheepdog, parallel collaborative programming-by-demonstrationKnowledge-Based Systems10.1016/j.knosys.2009.06.00823:2(94-109)Online publication date: 1-Mar-2010
https://dl.acm.org/doi/10.1016/j.knosys.2009.06.008
Stumpf SRajaram VLi LWong WBurnett MDietterich TSullivan EHerlocker J(2009)Interacting meaningfully with machine learning systemsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2009.03.00467:8(639-662)Online publication date: 1-Aug-2009
https://dl.acm.org/doi/10.1016/j.ijhcs.2009.03.004
Blythe JRuss TStaab S(2008)Case-based reasoning for procedure learning by instructionProceedings of the 13th international conference on Intelligent user interfaces10.1145/1378773.1378815(301-304)Online publication date: 13-Jan-2008
https://dl.acm.org/doi/10.1145/1378773.1378815
Chen JWeld DStaab S(2008)Recovering from errors during programming by demonstrationProceedings of the 13th international conference on Intelligent user interfaces10.1145/1378773.1378794(159-168)Online publication date: 13-Jan-2008
https://dl.acm.org/doi/10.1145/1378773.1378794
Stumpf SSullivan EFitzhenry EOberst IWong WBurnett MStaab S(2008)Integrating rich user feedback into intelligent user interfacesProceedings of the 13th international conference on Intelligent user interfaces10.1145/1378773.1378781(50-59)Online publication date: 13-Jan-2008
https://dl.acm.org/doi/10.1145/1378773.1378781
Rybski PStolarz JYoon KVeloso M(2008)Using dialog and human observations to dictate tasks to a learning robot assistantIntelligent Service Robotics10.1007/s11370-008-0016-51:2(159-167)Online publication date: 13-Feb-2008
https://doi.org/10.1007/s11370-008-0016-5
Castelli VBergman LChin DZhou MLau TPuerta A(2007)Distributed augmentation-based learningProceedings of the 12th international conference on Intelligent user interfaces10.1145/1216295.1216327(160-169)Online publication date: 28-Jan-2007
https://dl.acm.org/doi/10.1145/1216295.1216327
Stumpf SRajaram VLi LBurnett MDietterich TSullivan EDrummond RHerlocker JChin DZhou MLau TPuerta A(2007)Toward harnessing user feedback for machine learningProceedings of the 12th international conference on Intelligent user interfaces10.1145/1216295.1216316(82-91)Online publication date: 28-Jan-2007
https://dl.acm.org/doi/10.1145/1216295.1216316
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations