Spoken Conversational Search: Speech-only Interactive Information Retrieval

ABSTRACT

This research investigates a new interface paradigm for interactive information retrieval (IIR) which forces us to shift away from the classic "ten blue links" search engine results page. Instead we investigate how to present search results through a conversation over a speech-only communication channel where no screen is available. Accessing information via speech is becoming increasingly pervasive and is already important for people with a visual impairment. However, presenting search results over a speech-only communication channel is challenging due to cognitive limitations and the transient nature of audio. Studies have indicated that the implementation of speech recognizers and screen readers must be carefully designed and cannot simply be added to an existing system. Therefore the aim of this research is to develop a new interaction framework for effective and efficient IIR over a speech-only channel: a Spoken Conversational Search System (SCSS) which provides a conversational approach to defining user information needs, presenting results and enabling search reformulations. In order to contribute to a more efficient and effective search experience when using a SCSS, we intend for a tighter integration between document search and conversational processes.

References

1. J. F. Allen, D. K. Byron, M. Dzikovska, G. Ferguson, L. Galescu, and A. Stent. Toward conversational human-computer interaction. AI magazine, 22 (4): 27, 2001.
2. S. Azenkot and N. B. Lee. Exploring the use of speech input by blind people on mobile devices. In Proc. SIGACCESS, 2013.
3. C. Baber, B. Mellor, R. Graham, J. M. Noyes, and C. Tunley. Workload and the use of automatic speech recognition: The effects of time and resource demands. Speech Communication, 20 (1): 37--53, 1996.
4. N. O. Bernsen, H. Dybkjær, and L. Dybkjær. Designing interactive speech systems: From first ideas to user testing. Springer, 1998.
5. E. Chang, F. Seide, H. M. Meng, C. Zhuoran, S. Yu, and L. Yuk-Chi. A system for spoken query information retrieval on mobile devices. Speech and Audio Processing, IEEE Transactions on, 10 (8): 531--541, 2002.
6. V. Demberg and A. Sayeed. Linguistic cognitive load: implications for automotive uis. In Proc. of AutomotiveUI 2011, 2011.
7. V. Demberg, A. Winterboer, and J. D. Moore. A strategy for information presentation in spoken dialog systems. Computational Linguistics, 37 (3): 489--539, 2011.
8. L. Dybkjaer, N. O. Bernsen, and W. Minker. Evaluation and usability of multimodal spoken language dialogue systems. Speech Communication, 43 (1): 33--54, 2004.
9. N. M. Fraser and G. Gilbert. Simulating speech systems. Computer Speech & Language, 5 (1): 81--99, 1991.
10. S. G. Hart and L. E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. Advances in psychology, 52: 139--183, 1988.
11. C. Hulme, S. Roodenrys, R. Schweickert, G. D. Brown, S. Martin, and G. Stuart. Word-frequency effects on short-term memory tasks: Evidence for a redintegration process in immediate serial recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23 (5): 1217--1232, 1997.
12. B. J. Jansen and A. Spink. How are we searching the World Wide Web? A comparison of nine search engine transaction logs. Inf. Process. Manag., 42 (1): 248--263, 2006.
13. D. Kelly. Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval, 3 (1-2): 1--224, 2009.
14. J. Lai and N. Yankelovich. Speech interface design. In Encyclopedia of Language & Linguistics (Second Edition), pages 764--770. Elsevier, 2006.
15. G. Marchionini and R. White. Find what you need, understand what you find. International Journal of Human-Computer Interaction, 23 (3): 205--237, 2007.
16. L. J. Najjar, J. J. Ockerman, and J. C. Thompson. User interface design guidelines for speech recognition applications. In Proc. of IEEE VRAIS '98, 1998.
17. N. G. Sahib, D. Al Thani, A. Tombros, and T. Stockman. Accessible information seeking. Proc. of Digital Futures, 12, 2012.
18. J. Schalkwyk, D. Beeferman, F. Beaufays, B. Byrne, C. Chelba, M. Cohen, M. Kamvar, and B. Strope. "your word is my command": Google search by voice: A case study. In Advances in Speech Recognition, pages 61--90. Springer US, 2010.
19. J. R. Trippas, D. Spina, M. Sanderson, and L. Cavedon. Results presentation methods for a spoken conversational search system. In First International Workshop on Novel Web Search Interfaces and Systems (NWSearch'15), 2015.
20. J. R. Trippas, D. Spina, M. Sanderson, and L. Cavedon. Towards Understanding the Impact of Length in Web Search Result Summaries over a Speech-only Communication Channel. In Proc. of SIGIR'15, pages 991--994, 2015.
21. M. Turunen, J. Hakulinen, N. Rajput, and A. A. Nanavati. Evaluation of Mobile and Pervasive Speech Applications, pages 219--262. 2012.
22. S. Varges, F. Weng, and H. Pon-Barry. Interactive question answering and constraint relaxation in spoken dialogue systems, 2006.
23. N. Yankelovich and J. Lai. Designing speech user interfaces. 1998.
24. N. Yankelovich, G.-A. Levow, and M. Marx. Designing speechacts: Issues in speech user interfaces. In Proc. of the SIGCHI'95, pages 369--376, 1995.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • Laboratory experiments

  • Interaction design

    • Interaction design process and methods

      • Interface design prototyping

• Information systems

  • Information retrieval

    • Users and interactive retrieval

      • Search interfaces

  • Information systems applications

    • Multimedia information systems