Beyond Translation: Design and Evaluation of an Emotional and Contextual Knowledge Interface for Foreign Language Social Media Posts

ABSTRACT

Although many social media sites now provide machine translation (MT) for foreign language posts, translation of a post may not suffice to support understanding of, and engagement with, that post. We present SenseTrans, a tool that provides emotional and contextual annotations generated by natural language analysis in addition to machine translation. We evaluated SenseTrans in a laboratory experiment in which native English speakers browsed five Facebook profiles in foreign languages. One group used the SenseTrans interface while the other group used MT alone. Participants using SenseTrans reported significantly greater understanding of the posts, and greater willingness to engage with the posts. However, no additional cognitive load was associated with using an interface that provided more information. These results provide promising support for the idea of using computational tools to annotate communication to support multilingual sense making and interaction on social media.

References

1. Rami Al-Rfou, Vivek Kulkarni, Bryan Perozzi, and Steven Skiena. 2015. Polyglot-NER: Massive multilingual named entity recognition. In Proceedings of the 2015 SIAM International Conference on Data Mining, 586--594.
2. Matheus Araujo, Julio Reis, Adriano Pereira, and Fabricio Benevenuto. 2016. An evaluation of machine translation for multilingual sentence-level sentiment analysis. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, 1140--1145
3. Alexandra Balahur and Marco Turchi. 2014. Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis. Computer Speech & Language 28, 1: 56--75.
4. Mikhail Bautin, Lohit Vijayarenu, and Steven Skiena. 2008. International Sentiment Analysis for News and Blogs. In Proceedings of ICWSM.
5. Charles R. Berger. 2011. Knowledge structures and social interaction. In the SAGE handbook of interpersonal communication, Mark L. Knapp, and John A. Daly (eds). SAGE Publications
6. Kalina Bontcheva and Dominic Rout. 2014. Making sense of social media streams through semantics: a survey. Semantic Web 5, 5: 373--403.
7. Moira Burke, Robert Kraut, and Cameron Marlow. 2011. Social capital on facebook: differentiating uses and users. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 571--580.
8. Jordi Carrera, Olga Beregovaya, and Alex Yanishevsky. 2009. Machine translation for cross-language social media.
9. Paul Ekman. An argument for basic emotions. 1992. Cognition & emotion 6, 3--4: 169--200.
10. Nicole B. Ellison, Charles Steinfield, and Cliff Lampe. 2011. Connection strategies: Social capital implications of Facebook-enabled communication practices. New media & society 13, 6: 873--892.
11. Nicole B. Ellison, D Y.vette Wohn, and Christine M. Greenhow. 2014. Adolescents' visions of their future careers, educational plans, and life pathways The role of bridging and bonding social capital experiences. Journal of Social and Personal Relationships 31, 4: 516--534.
12. Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. 2010. Every picture tells a story: generating sentences from images. In Proceedings of the 11th European conference on Computer vision: Part IV (ECCV'10), Kostas Daniilidis, Petros Maragos, and Nikos Paragios (Eds.). Springer-Verlag, Berlin, Heidelberg, 15--29.
13. Ge Gao, Hao-Chuan Wang, Dan Cosley, and Susan R. Fussell. 2013. Same translation but different experience: the effects of highlighting on machine-translated conversations. In Proceedings of the sigchi conference on human factors in computing systems, 449--458.
14. Ge Gao, Bin Xu, David C Hau, Zheng Yao, Dan Cosley, and Susan R. Fussell. 2015. Two is better than one: improving multilingual collaboration by giving two machine translation outputs. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 852--863.
15. Ruth García-Gavilanes, Yelena Mejova, and Daniele Quercia. 2014. Twitter ain't without frontiers: economic, social, and cultural boundaries in international communication. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing, 1511--1522.
16. Scott A. Hale. 2014. Global connectivity and multilinguals in the Twitter network. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 833--842.
17. Bo Han, Paul Cook, and Timothy Baldwin. 2013. Lexical normalization for social media text. ACM Transactions on Intelligent Systems and Technology (TIST) 4, 1: 5.
18. Cheng-Hsien Han, Chi-Lan Yang, and Hao-Chuan Wang. 2014. Supporting second language reading with picture note-taking. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 2245--2250.
19. Google. Detecting Languages | Google Cloud Translation API Documentation | Google Cloud Platform. Retrieved September 18, 2017 from https://cloud.google.com/translate/docs/detecting-language
20. Google. Cloud Translation API - Dynamic Translation | Google Cloud Platform. Retrieved September 17, 2017 from https://cloud.google.com/translate/
21. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52: 139--183.
22. Ari M J Hautasaari, Naomi Yamashita, and Ge Gao. 2014. Maybe it was a joke: emotion detection in text-only communication by non-native english speakers. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, 3715--3724.
23. Rebecca A. Hayes, Caleb T. Carr, and Donghee Y. Wohn. 2016. One click, many meanings: Interpreting paralinguistic digital affordances in social media. Journal of Broadcasting & Electronic Media 60, 1: 171--187
24. Lichan Hong, Gregorio Convertino, and Ed H Chi. 2011. Language Matters in Twitter: A Large Scale Study. Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, 1: 518--521.
25. IBM. Watson Natural Language Understanding. Retrieved September 17, 2017 from https://www.ibm.com/watson/services/natural-language-understanding/
26. Adam N. Joinson. 2008. Looking at, looking up or keeping up with people?: motives and use of facebook. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 1027--1036.
27. Cliff Lampe, Nicole Ellison, and Charles Steinfield. 2006. A face(book) in the crowd: social Searching vs. social browsing. In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work (CSCW '06). ACM, New York, NY, USA, 167--170.
28. Lina Lee and Alfred Markey. 2014. A study of learners' perceptions of online intercultural exchange through Web 2.0 technologies. ReCALL 26, 3: 281--297.
29. .Hajin Lim and Susan R. Fussell. 2017. Making Sense of Foreign Language Posts in Social Media. Proc. ACM Hum.-Comput. Interact. 1, CSCW, Article 69 (December 2017), 16 pages.
30. Hajin Lim and Susan R. Fussell. 2017. Understanding How People Attend to and Engage with Foreign Language Posts in Multilingual Newsfeeds. In Eleventh International AAAI Conference on Web and Social Media.
31. Sally Maitlis and Marlys Christianson. 2014. Sensemaking in organizations: Taking stock and moving forward. The Academy of Management Annals 8, 1: 57--125.
32. James C. McCroskey and Thomas A. McCain. 1974. The measurement of interpersonal attraction. Speech Monographs 41, 3: 261--266.
33. Joyce S. Osland and Allan Bird. 2000. Beyond sophisticated stereotyping: Cultural sensemaking in context. The Academy of Management Executive 14, 1: 65--77.
34. R. Michael Paige, Helen Jorstad, Laura Siaya, Francine Klein, and Jeanete Colby. 2000. Culture Learning in Language Education: A Review of the Literature.
35. Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, 2--4.
36. Emilee Rader, Alcides Velasquez, Kayla D. Hales, and Helen Kwok. 2012. The gap between producer intentions and consumer behavior in social media. In Proceedings of the 17th ACM international conference on Supporting group work (GROUP '12). ACM, New York, NY, USA, 249--252.
37. Dale H. Schunk. 2003. Self-efficacy for reading and writing: Influence of modeling, goal setting, and self-evaluation. Reading &Writing Quarterly 19, 2: 159--172
38. Tomohiro Shigenobu. 2007. Evaluation and usability of back translation for intercultural communication. Usability and Internationalization. Global and Local User Interfaces: 259--265.
39. Andrew D. Smock, Nicole B. Ellison, Cliff Lampe, and Donghee Y. Wohn. "Facebook as a toolkit: A uses and gratification approach to unbundling feature use." Computers in Human Behavior 27, no. 6 (2011): 2322--2329.
40. TextRazor. TextRazor - The Natural Language Processing API. Retrieved September 17, 2017 from https://www.textrazor.com/named_entity_recognition
41. Stefan Volk, Tine Köhler, and Markus Pudelko. 2014. Brain drain: The cognitive neuroscience of foreign language processing in multinational corporations. Journal of International Business Studies, 45.7, 862--885.
42. Hao-Chuan Wang, Dan Cosley, and Susan R. Fussell. 2010. Idea expander: supporting group brainstorming with conversationally triggered visual thinking stimuli. In Proceedings of the 2010 ACM conference on Computer supported cooperative work (CSCW '10). ACM, New York, NY, USA, 103--106.
43. Charles Wankel. 2016. Developing cross-cultural managerial skills through social media. Journal of Organizational Change Management 29, 1: 116--124.
44. Shaomei Wu, Jeffrey Wieland, Omid Farivar, and Julie Schiller. 2017. Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 1180--1192.
45. Naomi Yamashita and Toru Ishida. 2006. Effects of machine translation on collaborative work. In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, 515--524.
46. Naomi Yamashita and Toru Ishida. 2006. Automatic prediction of misconceptions in multilingual computer-mediated communication. In Proceedings of the 11th international conference on Intelligent user interfaces, 62--69.
47. Kadir Yoğurtçu. 2013. The impact of self-efficacy perception on reading comprehension on academic achievement. Procedia-Social and Behavioral Sciences 70: 375--386
48. Dejin Zhao and Mary Beth Rosson. 2009. How and why people Twitter: the role that micro-blogging plays in informal communication at work. In Proceedings of the ACM 2009 international conference on Supporting group work, 243--252.
49. Bu Zhong, Marie Hardin, and Tao Sun. 2011. Less effortful thinking leads to more social networking? The associations between the use of social network sites and personality traits. Computers in Human Behavior 27, 3: 1265--1271.

Index Terms

• Human-centered computing