skip to main content
10.5555/1734454.1734552acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
research-article

Following directions using statistical machine translation

Published: 02 March 2010 Publication History

Abstract

Mobile robots that interact with humans in an intuitive way must be able to follow directions provided by humans in unconstrained natural language. In this work we investigate how statistical machine translation techniques can be used to bridge the gap between natural language route instructions and a map of an environment built by a robot. Our approach uses training data to learn to translate from natural language instructions to an automatically-labeled map. The complexity of the translation process is controlled by taking advantage of physical constraints imposed by the map. As a result, our technique can efficiently handle uncertainty in both map labeling and parsing. Our experiments demonstrate the promising capabilities achieved by our approach.

References

[1]
A. V. Aho and J. D. Ullman, The Theory of Parsing, Translation, and Compiling. Prentice Hall Professional Technical Reference, 1972.
[2]
S. R. K. Branavan, H. Chen, L. Zettlemoyer, and R. Barzilay, "Reinforcement learning for mapping instructions to actions," in Proc. of the Joint Conf. of the 47th Annual Meeting of the ACL and the 4th Int'l Joint Conference on Natural Language Processing. Suntec, Singapore: Association for Computational Linguistics, August 2009, pp. 82--90.
[3]
P. F. Brown, J. Cocke, S. A. D. Pietra, V. J. D. Pietra, F. Jelinek, J. D. Lafferty, R. L. Mercer, and P. S. Roossin, "A statistical approach to machine translation," Comput. Linguist., vol. 16, no. 2, pp. 79--85, 1990.
[4]
D. Chen and R. Mooney, "Learning to sportscast: a test of grounded language acquisition," in ICML 2008: Proc. of the 25th international conference on Machine learning. Helsinki, Finland: ACM, 2008, pp. 128--135.
[5]
D. Chiang. (2006) An introduction to synchronous grammars. Available: http://www.isi.edu/chiang/papers/synchtut.pdf
[6]
J. Dzifcak, M. Scheutz, C. Baral, and P. Schermerhorn, "What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution," in Proc. of the 2009 IEEE Int'l Conf. on Robotics and Automation (ICRA '09), Kobe, Japan, May 2009.
[7]
S. Friedman, H. Pasula, and D. Fox, "Voronoi random fields: Extracting topological structure of indoor environments via place labeling. in IJCAI, M. M. Veloso, Ed., 2007, pp. 2109--2114.
[8]
J.-S. Gutmann, M. Fukuchi, and M. Fujita, "3d perception and environment map generation for humanoid robot navigation," Int. J. Rob. Res., vol. 27, no. 10, pp. 1117--1134, 2008.
[9]
K.-y. Hsiao, S. Tellex, S. Vosoughi, R. Kubat, and D. Roy, "Object schemas for grounding language in a responsive robot," Connection Science, vol. 20, no. 4, pp. 253--276, 2008.
[10]
A. Lopez, "Statistical machine translation," ACM Comput. Surv., vol. 40, no. 3, pp. 1--49, 2008.
[11]
M. Macmahon, B. Stankiewicz, and B. Kuipers, "Walk the talk: Connecting language, knowledge, action in route instructions," in In Proc. of the Nat. Conf. on Artificial Intelligence (AAAI), 2006, pp. 1475--1482.
[12]
E. Martins, M. Pascoal, and J. Santos, "A new improvement for a k shortest paths algorithm," Investigação Operational, 2001.
[13]
R. J. Mooney, "Learning to connect language and perception," in Proc. of the Twenty-Third AAAI Conf. on Artificial Intelligence, AAAI 2008, D. Fox and C. P. Gomes, Eds. Chicago, Illinois: AAAI Press, July 2008, pp. 1598--1601.
[14]
F. J. Och and H. Ney, "A systematic comparison of various statistical alignment models," Computational Linguistics, vol. 29, no. 1, pp. 19--51, 2003.
[15]
N. D. Ratliff, J. A. Bagnell, and M. A. Zinkevich, "Maximum margin planning," in In Proc. of the 23rd Int'l Conf. on Machine Learning (ICML06), 2006.
[16]
D. Roy, "Learning visually-grounded words and syntax for a scene description task," Computer Speech and Language, 2002.
[17]
D. Roy, "Semiotic schemas: a framework for grounding language in action and perception," Artificial Intelligence, vol. 167, no. 1-2, pp. 170--205, 2005.
[18]
N. Shimizu and A. Haas, "Learning to Follow Navigational Route Instructions," in Int'l Joint Conf. on Artificial Intelligence (IJCAI), 2009.
[19]
M. Skubic, D. Perzanowski, S. Blisard, A. Schultz, W. Adams, M. Bugajska, and D. Brock, "Spatial language for human-robot dialogs," IEEE Transactions on Systems, Man, and Cybernetics, Part C, Special Issue on Human-Robot Interaction, vol. 34, no. 2, pp. 154--167, May 2001.
[20]
S. Thrun,W. Burgard, and D. Fox, Probabilistic Robotics. Cambridge, MA: MIT Press, September 2005, ISBN 0-262-20162-3.
[21]
Y. Wei, E. Brunskill, T. Kollar, and N. Roy, "Where to go: Interpreting natural directions using global inference," in Int'l Conf. on Robotics and Automation (ICRA), 2009.
[22]
Y. W. Wong, "Learning for semantic parsing and natural language generation using statistical machine translation techniques," Ph.D. dissertation, Univ. of Texas at Austin, August 2007.
[23]
Y. W. Wong and R. J. Mooney, "Learning for semantic parsing with statistical machine translation," in Proc. of the main conference on Human Language Technology Conf. of the North American Chapter of the Association of Computational Linguistics. Association for Computational Linguistics, 2006, pp. 439--446.
[24]
J. Y. Yen, "Finding the k shortest loopless paths in a network, Management Science, vol. 17, no. 11, pp. 712--716, 1971.
[25]
B. Ziebart, A. Maas, A. Dey, and J. D. Bagnell, "Navigate like a cabbie: Probabilistic reasoning from observed context-aware behavior," in UBICOMP: Ubiquitious Computation, 2008.

Cited By

View all
  • (2020)Discourse in MultimediaComputational Linguistics10.1162/coli_a_0036045:4(627-665)Online publication date: 1-Jan-2020
  • (2019)Miscommunication Detection and Recovery in Situated Human–Robot DialogueACM Transactions on Interactive Intelligent Systems10.1145/32371899:1(1-40)Online publication date: 17-Feb-2019
  • (2018)Scheduled policy optimization for natural language communication with intelligent agentsProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304222.3304396(4503-4509)Online publication date: 13-Jul-2018
  • Show More Cited By

Index Terms

  1. Following directions using statistical machine translation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    HRI '10: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction
    March 2010
    400 pages
    ISBN:9781424448937

    Sponsors

    Publisher

    IEEE Press

    Publication History

    Published: 02 March 2010

    Check for updates

    Author Tags

    1. human-robot interaction
    2. instruction following
    3. natural language
    4. navigation
    5. statistical machine translation

    Qualifiers

    • Research-article

    Conference

    HRI 10
    Sponsor:

    Acceptance Rates

    HRI '10 Paper Acceptance Rate 26 of 124 submissions, 21%;
    Overall Acceptance Rate 268 of 1,124 submissions, 24%

    Upcoming Conference

    HRI '25
    ACM/IEEE International Conference on Human-Robot Interaction
    March 4 - 6, 2025
    Melbourne , VIC , Australia

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2020)Discourse in MultimediaComputational Linguistics10.1162/coli_a_0036045:4(627-665)Online publication date: 1-Jan-2020
    • (2019)Miscommunication Detection and Recovery in Situated Human–Robot DialogueACM Transactions on Interactive Intelligent Systems10.1145/32371899:1(1-40)Online publication date: 17-Feb-2019
    • (2018)Scheduled policy optimization for natural language communication with intelligent agentsProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304222.3304396(4503-4509)Online publication date: 13-Jul-2018
    • (2018)Scene understanding using natural language description based on 3D semantic graph mapIntelligent Service Robotics10.5555/3287991.328806011:4(347-354)Online publication date: 1-Oct-2018
    • (2018)Efficient grounding of abstract spatial concepts for natural language interaction with robot platformsInternational Journal of Robotics Research10.1177/027836491877762737:10(1269-1299)Online publication date: 1-Sep-2018
    • (2018)Behavioral Indoor Navigation With Natural Language DirectionsCompanion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3173386.3177001(283-284)Online publication date: 1-Mar-2018
    • (2018)End-User Programming of Manipulator Robots in Situated Tangible Programming ParadigmCompanion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3173386.3176923(319-320)Online publication date: 1-Mar-2018
    • (2018)Human–Robot Communications of Probabilistic Beliefs via a Dirichlet Process Mixture of StatementsIEEE Transactions on Robotics10.1109/TRO.2018.283036034:5(1280-1298)Online publication date: 1-Oct-2018
    • (2017)Navigational Instruction Generation as Inverse Reinforcement Learning with Neural Machine TranslationProceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction10.1145/2909824.3020241(109-118)Online publication date: 6-Mar-2017
    • (2017)Situated Tangible Robot ProgrammingProceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction10.1145/2909824.3020240(473-482)Online publication date: 6-Mar-2017
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media