Cássio Castaldi Araujo Blaz
ABSTRACT
Sentiment analysis has been adopted in software engineering for problems such as software usability and sentiment of developers in open-source projects. This paper proposes a method to evaluate the sentiment contained in tickets for IT (Information Technology) support.IT tickets are broad in coverage (e.g. infrastructure, software), and involve errors, incidents, requests, etc. The main challenge is to automatically distinguish between factual information, which is intrinsically negative (e.g. error description), from the sentiment embedded in the description. Our approach is to automatically create a Domain Dictionary that contains terms with sentiment in the IT context, used to filter terms in ticket for sentiment analysis. We experiment and evaluate three approaches for calculating the polarity of terms in tickets. Our study was developed using 34,895 tickets from five organizations, from which we randomly selected 2,333 tickets to compose a Gold Standard. Our best results display an average precision and recall of 82.83% and 88.42%, which outperforms the compared sentiment analysis solutions.
- S. Baccianella, A. Esuli, and F. Sebastiani. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proc. of the International Conference on Language Resources and Evaluation (LREC), Valletta, Malta, 2010.Google Scholar
- A. Balahur, R. Steinberger, M. Kabadjov, V. Zavarella, E. Van Der Goot, M. Halkia, B. Pouliquen, and J. Belyaeva. Sentiment analysis in the news. In Proc. of the International Conference on Language Resources and Evaluation (LREC), 2010, Valletta, Malta, volume 10, page 2216, 2010.Google Scholar
- A. Balahur and M. Turchi. Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis. Computer Speech & Language, 28(1):56--75, 2014. Google ScholarDigital Library
- D. Bollegala, D. Weir, and J. Carroll. Cross-domain sentiment classification using a sentiment sensitive thesaurus. IEEE Transactions on Knowledge and Data Engineering, 25(8): 1719--1731, Aug 2013. Google ScholarDigital Library
- J. Bollen, H. Mao, and X. Zeng. Twitter mood predicts the stock market. Journal of Computational Science, 2(1):1--8, 2011.Google ScholarCross Ref
- P. Ekman. An argument for basic emotions. Cognition & emotion, 6(3-4):169--200, 1992.Google ScholarCross Ref
- A. M. El-Halees. Software Usability Evaluation Using Opinion Mining. Journal of Software, 9(2):343--349, feb 2014.Google ScholarCross Ref
- C. Fellbaum. WordNet: An Electronic Lexical Database. Bradford Books, 1998.Google ScholarCross Ref
- N. Godbole, M. Srinivasaiah, and S. Skiena. Large-scale sentiment analysis for news and blogs. In Proc. of the First International AAAI Conference on Weblogs and Social Media (ICWSM), volume 2, 2007.Google Scholar
- E. Guzman, D. Azócar, and Y. Li. Sentiment analysis of commit comments in github: an empirical study. In Proc. of the 11th Working Conference on Mining Software Repositories, MSR 2014, Hyderabad, India, pages 352--355, 2014. Google ScholarDigital Library
- E. Guzman and B. Bruegge. Towards Emotional Awareness in Software Development Teams. In Proc. of the 9th Joint Meeting on Foundations of Software Engineering, pages 671--674, New York, New York, USA, aug 2013. ACM Press. Google ScholarDigital Library
- M. Hu and B. Liu. Mining and summarizing customer reviews. In Proc. of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 168--177. ACM, 2004. Google ScholarDigital Library
- R. Jongeling and A. Serebrenik. Choosing Your Weapons: On Sentiment Analysis Tools for Software Engineering Research. In Proc. of the IEEE International Conference on Software Maintenance and Evolution, pages 531--535, Bremen, 2015. Google ScholarDigital Library
- F. Jurado and P. Rodriguez. Sentiment Analysis in monitoring software development processes: An exploratory case study on GitHub's project issues. Journal of Systems and Software, 104:82--89, 2015. Google ScholarDigital Library
- N. Kobayashi, K. Inui, and Y. Matsumoto. Extracting aspect-evaluation and aspect-of relations in opinion mining. In EMNLP-CoNLL 2007, Proc. of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Prague, Czech Republic, pages 1065--1074, 2007.Google Scholar
- B. Liu. Sentiment analysis and opinion mining. Morgan & Claypool Publishers, 2012.Google ScholarDigital Library
- S. M. Mohammad. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In H. Meiselman, editor, Emotion Measurement. Elsevier, 2016.Google ScholarCross Ref
- M. D. Molina-Gonzalez, E. Martinez-Camara, M.-T. Martin-Valdivia, and J. M. Perea-Ortega. Semantic orientation for polarity classification in spanish reviews. Expert Systems with Applications, 40(18): 7250--7257, 2013.Google ScholarCross Ref
- A. Murgia, P. Tourani, B. Adams, and M. Ortu. Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In Proc. of the 11th Working Conference on Mining Software Repositories, MSR 2014, Hyderabad, India, 2014. Google ScholarDigital Library
- M. Ortu, B. Adams, G. Destefanis, P. Tourani, M. Marchesi, and R. Tonelli. Are Bullies more Productive? Empirical Study of Affectiveness vs. Issue Fixing Time. In Proc. of the IEEE/ACM Working Conference on Mining Software Repositories, Florence, 2015. Google ScholarDigital Library
- D. Pletea, B. Vasilescu, and A. Serebrenik. Security and Emotion: Sentiment Analysis of Security Discussions on GitHub. In Proc. of the IEEE/ACM Working Conference on Mining Software Repositories, pages 348--351, 2014. Google ScholarDigital Library
- G. Qiu, B. Liu, J. Bu, and C. Chen. Expanding domain sentiment lexicon through double propagation. In Proceedings of the 21st International Jont Conference on Artifical Intelligence, IJCAI'09, pages 1199--1204, San Francisco, CA, USA, 2009. Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
- S. Sahibudin, M. Sharifi, and M. Ayat. Combining itil, cobit and iso/iec 27002 in order to design a comprehensive it framework in organizations. In Proc. of the Second Asia International Conference on Modeling Simulation, 2008, pages 749--753, May 2008. Google ScholarDigital Library
- M. Silva, P. Carvalho, and L. Sarmento. Building a sentiment lexicon for social judgement mining. Computational Processing of the Portuguese Language, pages 218--228, 2012. Google ScholarDigital Library
- J. Steinberger, M. Ebrahim, M. Ehrmann, A. Hurriyetoglu, M. Kabadjov, P. Lenkova, R. Steinberger, H. Tanev, S. VÃązquez, and V. Zavarella. Creating sentiment dictionaries via triangulation. Decision Support Systems, 53(4):689--694, 2012. Google ScholarDigital Library
- M. Thelwall, K. Buckley, G. Paltoglou, D. Cai, and A. Kappas. Sentiment in short strength detection informal text. J. Am. Soc. Inf. Sci. Technol., 61(12): 2544--2558, Dec. 2010. Google ScholarDigital Library
- M. Tsytsarau and T. Palpanas. Survey on mining subjective data on the web. Data Mining and Knowledge Discovery, 24(3):478--514, 2012. Google ScholarDigital Library
- D. Tumitan and K. Becker. Sentiment-based features for predicting election polls: A case study on the brazilian scenario. In Proc. of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), 2014, volume 2, pages 126--133, Aug 2014. Google ScholarDigital Library
- A. B. Warriner, V. Kuperman, and M. Brysbaert. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior research methods, 45(4): 1191--207, Dec. 2013.Google Scholar
- J. Wiebe, T. Wilson, and C. Cardie. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39(2-3):165--210, 2005.Google Scholar
Index Terms
- Sentiment analysis in tickets for IT support
Recommendations
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge managementSentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...
Topic sentiment change analysis
MLDM'11: Proceedings of the 7th international conference on Machine learning and data mining in pattern recognitionPublic opinions on a topic may change over time. Topic Sentiment change analysis is a new research problem consisting of two main components: (a) mining opinions on a certain topic, and (b) detect significant changes of sentiment of the opinions on the ...
Social sentiment sensor: a visualization system for topic detection and topic sentiment analysis on microblog
As a new form of social media, microblogging provides platform sharing, wherein users can share their feelings and ideas on certain topics. Bursty topics from microblogs are the results of the emerging issues that instantly attract more followers and ...
Comments