Reformulation Strategies of Repeated References in the Context of Robot Perception Errors in Situated Dialogue (original) (raw)

Abstract

We performed an experiment in which human participants interacted through a natural language dialogue interface with a simulated robot to fulfil a series of object manipulation tasks. We introduced errors into the robot's perception, and observed the resulting problems in the dialogues and their resolutions. We then introduced different methods for the user to request information about the robot's understanding of the environment. In this work, we describe the effects that the robot's perceptual errors and the information request options available to the participant had on the reformulation of the referring expressions the participants used when resolving a unsuccessful reference.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (46)

  1. J. Aberdeen and L. Ferro. Dialogue patterns and misunderstandings. In ISCA Tutorial and Research Workshop on Error Handling in Spoken Dialogue Systems, 2003.
  2. A. Anderson, B. Bader, M., B. E., G. M. E., Doherty, S. Garrod, S. Is- ard, J. Kowtko, J. McAllister, J. Miller, C. Sotillo, H. S. Thompson, and R. Weinert. The HCRC map task corpus. Language and Speech, 34(4):351-366, 1992.
  3. J. Bateman, J. Hois, R. Ross, and T. Tenbrink. A linguistic ontology of space for natural langauge processing. Artificial Intelligence, 174(14):1027-1071, 2010.
  4. M. Brenner, N. Hawes, J. Kelleher, and J. Wyatt. Mediating between qualitative and quantitative representations for task-oriented human- robot interaction. In In Proceedings of the 20th International Confer- ence on Artificial Intelligence (IJCAI-07), pages 2072-2077, 2007.
  5. M. Burgio and P. Knoeferle. Visual attention during spatial language comprehension. PLOS ONE, 10(1), 2015.
  6. M. Burigo and S. Sacchi. Object orientation affects spatial language comprehension. Cognitive Science, 37(8):1471-1492, 2013.
  7. L. Carlson-Radvansky and G. Logan. The influence of reference frame selection on spatial template construction. Journal of Memory and Langauge, 37:411-437, 1997.
  8. H. H. Clark and E. F. Schaefer. Contributing to discourse. Cognitive Science, pages 259-294, 1989.
  9. F. Costello and J. Kelleher. Spatial prepositions in context: The semantics of Near in the presense of distractor objects. In Proceedings of the 3rd ACL-Sigsem Workshop on Prepositions, pages 1-8, 2006.
  10. K. Coventry, A. Cangelosi, R. Rajapakse, A. Bacon, S. Newstead, D. joyce, and L. Richards. Spatial prepositions and vague quantifiers: Implementing the funcitonal geometric framework. In Spatial Cogni- tion IV. Reasoning, Action, Interaction. Springer Berling Heidelberg, 2005.
  11. K. Coventry and S. Garrod. Saying, Seeing and Acting: The Psycho- logical Semantics of Spatial Prepositions. Psychology Press, 2004.
  12. S. Dobnik. Teaching Mobile Robots to use Spatial Words. PhD thesis, The Queen's College, University of Oxford, 2009.
  13. N. Hawes, M. Klenk, K. Lockwood, G. S. Horn, and J. D. Kelleher. Towards a cognitive system that can recognize spatial regions based on context. In Proceedings of the 26th National Conference on Artificial Intelligence (AAAI'12), 2012, 2012.
  14. M. Hoetjes, R. Koolen, M. Goudbeek, E. Krahmer, and M. Swerts. Reduction in gesture during the production of repeated references. Journal of Memory and Language, 79-80:1-17, 2015.
  15. H. Horacek. Generating referential descriptions under conditions of uncertainty. In Proceedings of the 10th European Workshop on Natural Language Generation (ENLG), pages 58-67. Citeseer, 2005.
  16. J. Kelleher and F. Costello. Cognitive representations of projective prepositions. In Proceedings of the Second ACL-Sigsem Workshop of The Linguistic Dimensions of Prepositions and their Use in Compu- tational Linguistic Formalisms and Applications, 2005.
  17. J. Kelleher and F. Costello. Applying computational models of spatial prepositions to visually situated dialog. Computational Linguistics, 35(2):271-306, June 2009.
  18. J. Kelleher, F. Costello, and J. van Genabith. Dynamically structuring, updating and interrelating representations of visual and linguistic discourse context. Artificial Intelligence, 167(1):62-102, 2005.
  19. J. Kelleher and R. Ross. Topology in compositie spatial terms. In D. Rapp, editor, Proceedigns of Spatial Cognition 2010: Poster Presentations, pages 46-50. SFB/TR8 Spatial Cognition, 2010.
  20. J. Kelleher, R. Ross, B. Mac Namee, and C. Sloan. Situating spatial templates for human-robot interaction. In In Proceedings of the AAAI Symposium on Dialog with Robots, 11th-13th Nov. 2010.
  21. J. Kelleher, R. Ross, C. Sloan, and B. Mac Namee. The effect of occlusion on the semantics of projective spatial terms: a case study in grounding language in perception. Cognitive Processing, 12(1):95- 108, 2010.
  22. J. Kelleher and J. van Genabith. A computaitonal model of the refer- ential semantics of projective prepositiosn. In Syntax and Semantics of Prepositions, pages 211-228. Springer, 2006.
  23. J. D. Kelleher. Attention driven reference resolution in multimodal contexts. Artificial Intelligence Review, 25(1):21-35, 2006.
  24. C. Kennington and D. Schlangen. Simple learning and compositional applicaiton of perceptually grounded word meanings for incremental reference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Internatonal Joint Conference on Natural Language Processing, pages 292-301. Association for Computational Linguistics, Beijing, Ching, July 2015.
  25. L. Kunze, C. Burbridge, M. Albert, A. Tippur, J. Folkesson, P. Jensfelt, and N. Hawes. Combining top-down spatial reasoning and bottom- up object class recognition for scene understanding. In In Intelligent Robots and Systems (iROS), pages 2910-2915. IEEE, 2014.
  26. D. J. Litman, M. Swerts, and J. Hirschberg. Characterizing and predicting corrections in spoken dialog systems. Computational Linguistics, 32:417-438, 2006.
  27. C. Liu, R. Fang, and J. Y. Chai. Towards mediating shared perceptual basis in situated dialogue. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 140- 149. Association for Computational Linguistics, 2012.
  28. G. Logan and D. Sadler. A computational analysis of the apprehension of spatial relations. In P. Bloom, M. Peterson, L. Nadel, and M. Garret, editors, Language and Space, pages 493-530. MIT Press, 1996.
  29. R. López-Cózar, Z. Callejas, and D. Griol. Using knowledge of misunderstandings to increase the robustness of spoken dialogue systems. Knowledge-Based Systems, 23(5):471-485, July 2010.
  30. C. Manning, M. Surdeanu, J. Bauer, J. Finekl, S. Bethard, and D. McClosky. The stanford corenlp natural language processing toolkit. In Proceedings of the 52nd Annual Meeting of the Association of Computational Linguistics: System Demonstrations, pages 55-60, 2014.
  31. V. Mast and D. Wolter. A probabilistic framework for object descrip- tions in indoor route instructons. In T. Tenbrink, J. Stell, A. Galton, and Z. Wood, editors, Spatial Information Theory, pages 185-204. Springer, 2013.
  32. E. Ovchinnikova. Integration of World Knowledge for Natural Lan- guage Understanding. Atlantis Press, 2012.
  33. T. Regier and L. Carlson. Grounding spatial language in perceptoin and empirical and computational investigation. Journal of experimen- tal psychology: General, 130(2):273-298, 2001.
  34. M. Richter, J. Lins, S. Schneegans, Y. Sandamirskaya, and G. Schoner. Autonomous neural dynamics to test hypotheses in a model of spatial language. In 36th Annual Conference of the Cognitive Science Society, 2014.
  35. V. Rieser and D. Gkatzia. Generation for things unknown: Accounting for first-time users and hidden scenes. In Proceedings of the 1st International Workshop on Data-to-text Generation, 2015.
  36. R. Ross and J. Kelleher. Putting things "between" perspective. In Proceedings of the 21st Irish Conference on Artificial Intelligence and Cognitive Sciance conference (AICS), September 2010.
  37. R. Ross and J. Kelleher. Using the situational context to resolve frame of reference ambiguity in route descriptions. In Proceedings of the Second Workshop on Action, Perception and Langauge (APL'2), Uppsala, Sweden, November 2014.
  38. N. Schuette, J. Kelleher, and B. Mac Namee. A corpus based dialogue model for grounding in situated dialogue. In Proceedings of the 1st Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Language, Motor Control and Vision (MLIS-2012)., Montpellier, France, Aug. 2012.
  39. N. Schuette, J. Kelleher, and B. Mac Namee. The effect of sensor errors in sitauted human-computer dialogue. In A. Belz, D. Cosker, F. Keller, and W. Smith, editors, Proceedings of the 3rd Workshop on Visiaion and Language (VL'14) at Coling, 2014.
  40. H. Schultheis, S. Bertel, and T. Barkowsky. Moedlling mental spatial reasoning about cardinal directions. Cognitive Science, 38(8):1521- 1561, November/December 2014.
  41. J. Shin, S. S. Narayanan, L. Gerber, A. Kazemzadeh, D. Byrd, and others. Analysis of user behavior under error conditions in spoken dialogs. In INTERSPEECH, 2002.
  42. K. Sjoo. Functional Understanding of Space. PhD thesis, KTH Computer Science and Communications, Sweden, 2011.
  43. M. Spranger and S. Pauw. Dealing with perceptual deviation -vague semantics for spatial language and quantification. In L. Steels and M. Hild, editors, Language Grounding in Robots, pages 173-192. New York: Springer, 2012.
  44. S. Tellex. Natural language and spatial reasoning. PhD thesis, Massachusetts Institute of Technology, 2010.
  45. M. A. Walker, D. J. Litman, C. A. Kamm, and A. Abella. PARADISE: a framework for evaluating spoken dialogue agents. In Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, pages 271-280. Association for Computa- tional Linguistics, 1997.
  46. T. Winograd. Understanding natural language. New York: Academic Press, 1972.