skip to main content
10.1145/1228716.1228726acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
Article

Efficient model learning for dialog management

Published: 10 March 2007 Publication History

Abstract

Intelligent planning algorithms such as the Partially Observable Markov Decision Process (POMDP) have succeeded in dialog management applications [10, 11, 12] because they are robust to the inherent uncertainty of human interaction. Like all dialog planning systems, however, POMDPs require an accurate model of the user (e.g., what the user might say or want). POMDPs are generally specified using a large probabilistic model with many parameters. These parameters are difficult to specify from domain knowledge, and gathering enough data to estimate the parameters accurately a priori is expensive.In this paper, we take a Bayesian approach to learning the user model simultaneously with dialog manager policy. At the heart of our approach is an efficient incremental update algorithm that allows the dialog manager to replan just long enough to improve the current dialog policy given data from recent interactions. The update process has a relatively small computational cost, preventing long delays in the interaction. We are able to demonstrate a robust dialog manager that learns from interaction data, out-performing a hand-coded model in simulation and in a robotic wheelchair application.

References

[1]
R. Dearden, N. Friedman, and D. Andre. Model based bayesian exploration. pages 150--159, 1999.
[2]
G. J. Gordon. Stable function approximation in dynamic programming. In Proceedings of the Twelfth International Conference on Machine Learning, San Francisco, CA, 1995. Morgan Kaufmann.
[3]
R. Jaulmes, J. Pineau, and D. Precup. Learning in non-stationary partially observable markov decision processes. Workshop on Non-Stationarity in Reinforcement Learning at the ECML, 2005.
[4]
D. Litman, S. Singh, M. Kearns, and M. Walker. NJFun: a reinforcement learning spoken dialogue system. In Proceedings of the ANLP/NAACL 2000 Workshop on Conversational Systems, Seattle, 2000.
[5]
A. Nilim and L. Ghaoui. Robustness in markov decision problems with uncertain transition matrices, 2004.
[6]
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for pomdps, 2003.
[7]
J. Pineau, N. Roy, and S. Thrun. A hierarchical approach to pomdp planning and execution. In Workshop on Hierarchy and Memory in Reinforcement Learning (ICML), June 2001.
[8]
L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--286, 1989.
[9]
M. Ravishankar. Efficient Algorithms for Speech Recognition. PhD thesis, Carnegie Mellon, 1996.
[10]
N. Roy, J. Pineau, and S. Thrun. Spoken dialogue management using probabilistic reasoning. In Proceedings of the 38th Annual Meeting of the ACL, Hong Kong, 2000.
[11]
J. Williams and S. Young. Scaling up pomdps for dialogue management: The lhsummary pomdpla method. In Proceedings of the IEEE ASRU Workshop, 2005.
[12]
J. D. Williams, P. Poupart, and S. Young. Partially observable markov decision processes with continuous observations for dialogue management. In Proceedings of SIGdial Workshop on Discourse and Dialogue 2005, 2005.

Cited By

View all
  • (2024)A Survey on Dialogue Management in Human-robot InteractionACM Transactions on Human-Robot Interaction10.1145/364860513:2(1-22)Online publication date: 14-Jun-2024
  • (2023)Improved Inference of Human Intent by Combining Plan Recognition and Language Feedback2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS55552.2023.10342380(7976-7983)Online publication date: 1-Oct-2023
  • (2022)Evaluating Human–Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario GenerationACM Transactions on Human-Robot Interaction10.1145/347641211:3(1-30)Online publication date: 2-Sep-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HRI '07: Proceedings of the ACM/IEEE international conference on Human-robot interaction
March 2007
392 pages
ISBN:9781595936172
DOI:10.1145/1228716
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. decision-making under uncertainty
  2. human-robot interaction
  3. model learning

Qualifiers

  • Article

Conference

HRI07
HRI07: International Conference on Human Robot Interaction
March 10 - 12, 2007
Virginia, Arlington, USA

Acceptance Rates

HRI '07 Paper Acceptance Rate 22 of 101 submissions, 22%;
Overall Acceptance Rate 268 of 1,124 submissions, 24%

Upcoming Conference

HRI '25
ACM/IEEE International Conference on Human-Robot Interaction
March 4 - 6, 2025
Melbourne , VIC , Australia

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Survey on Dialogue Management in Human-robot InteractionACM Transactions on Human-Robot Interaction10.1145/364860513:2(1-22)Online publication date: 14-Jun-2024
  • (2023)Improved Inference of Human Intent by Combining Plan Recognition and Language Feedback2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS55552.2023.10342380(7976-7983)Online publication date: 1-Oct-2023
  • (2022)Evaluating Human–Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario GenerationACM Transactions on Human-Robot Interaction10.1145/347641211:3(1-30)Online publication date: 2-Sep-2022
  • (2021)Influencing leading and following in human–robot teamsAutonomous Robots10.1007/s10514-021-10016-745:7(959-978)Online publication date: 1-Oct-2021
  • (2018)Learning Motion Predictors for Smart Wheelchair Using Autoregressive Sparse Gaussian Process2018 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA.2018.8460502(713-718)Online publication date: 21-May-2018
  • (2018)RETRACTED ARTICLE: Speech to speech interaction system using Multimedia Tools and Partially Observable Markov Decision Process for visually impaired studentsMultimedia Tools and Applications10.1007/s11042-018-6264-279:7-8(5023-5042)Online publication date: 23-Jun-2018
  • (2017)Human-robot mutual adaptation in collaborative tasks: Models and experimentsThe International Journal of Robotics Research10.1177/027836491769059336:5-7(618-634)Online publication date: 14-Feb-2017
  • (2017)A smart wheelchair ecosystem for autonomous navigation in urban environmentsAutonomous Robots10.1007/s10514-016-9549-141:3(519-538)Online publication date: 1-Mar-2017
  • (2016)Simultaneous localization and mapping in unknown environment using dynamic matching of images and registration of point clouds2016 2nd International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM)10.1109/ICIEAM.2016.7910967(1-6)Online publication date: 2016
  • (2016)Learning User Intentions in Natural Language Call Routing SystemsRecent Developments and New Direction in Soft-Computing Foundations and Applications10.1007/978-3-319-32229-2_4(37-46)Online publication date: 26-May-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media