Contextual Sentiment Analysis with Untrained Annotators

Authors: Lucas A. Silva, Carla R. Aguiar

Abstract:

This work presents a proposal to perform contextual sentiment analysis using a supervised learning algorithm and disregarding the extensive training of annotators. To achieve this goal, a web platform was developed to perform the entire procedure outlined in this paper. The main contribution of the pipeline described in this article is to simplify and automate the annotation process through a system of analysis of congruence between the notes. This ensured satisfactory results even without using specialized annotators in the context of the research, avoiding the generation of biased training data for the classifiers. For this, a case study was conducted in a blog of entrepreneurship. The experimental results were consistent with the literature related annotation using formalized process with experts.

Keywords: Contextualized classifier, naïve Bayes, sentiment analysis, untrained annotators.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1091292

References:

[1] Internet World Stats, Usage and Population Statistics. http://www.internetworldstats.com/, January 2013.
[2] Movie Review Dataset. http://www.cs.cornell.edu/people/pabo/moviereview-data/, January 2013.
[3] Natural Language Toolkit. http://www.nltk.org/, January 2013.
[4] Statsoft - Technical Notes about Naive Bayes Classifier. http://www.statsoft.com/Textbook/Naive-Bayes-Classifier, January 2013.
[5] Wordnet, a Lexical Database for English. http://wordnet.princeton.edu/, January 2013.
[6] AlinaAdreevskaia and Sabine Bergler. Mining Wordnetfor Fuzzy Sentiment: Sentiment Tag Extraction from Wordnet Glosses. In 11th Conference of the European Chapter of the Association for ComputationalLinguistics, pages 209–216, 2006.
[7] The Nielsen Company. Global Faces and Networked Places, a Nielsen Report on Social Networkings New Global Footprint. Technical Report,Nielsen Company, March 2009.
[8] Kushal Dave, Steve Lawrence, and David M. Pennock. Mining the Peanut Gallery: Opinion Extraction and Semantic Classification of Product Reviews. In WWW, pages 519–528, 2003.
[9] Emanuel de Barros Albuquerque Ferreira. Análise de sentimentoemredessociaisutilizandoinfluência de palavras. Trabalho de Graduação - Universidade Federal de Pernambuco - UFPE. Departamento de CiênciadaComputação, Dezembro 2010.
[10] VasileiosHatzivassiloglou and Kathleen McKeown. Predicting the Semantic Orientation of Adjectives. In Philip R. Cohen and Wolfgang Wahlster, Editors, ACL, Pages 174–181. Morgan Kaufmann Publishers / ACL, 1997.
[11] VasileiosHatzivassiloglouandJanyceWiebe. Effects of Adjective Orientation and Gradabilityon Sentence Subjectivity. In COLING, Pages299–305. Morgan Kaufmann, 2000.
[12] Minqing Hu and Bing Liu. Mining and Summarizing Customer Reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, pages 168–177, NewYork, NY, USA, 2004. ACM.
[13] Bin Liu. Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data (Data-Centric Systems and Applications). Springer, 2008.
[14] Soo min Kim and Eduard Hovy. Automatic Identification of Pro and Con Reasons in Online Reviews. In Proceedings of COLING/ACL Poster Sessions, pages 483–490, 2006.
[15] Subhabrata Mukherjee. Sentiment Analysis - A Literature Survey, June 2012. Indian Institute of Technology, Bombay. Roll No: 10305061.
[16] Kamal Nigam. Using Maximum Entropy for Text Classification. In IJCAI-99 Workshop on Machine Learning for Information Filtering,pages 61–67, 1999.
[17] Neil OHare, Michael Davy, Adam Bermingham, Paul Ferguson, Praic Sheridan, CathalGurrin, and Alan F. Smeaton. Topic-Dependent Sentiment Analysis of Financial Blogs. In Proc. of CIKM Workshop on Topic Sentiment Analysis for Mass Opinion (TSA ’09), pages 09–16, November2009.
[18] Bo Pang and Lillian Lee. Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval, 2(1-2):1–135, January2008.
[19] Bo Pang, Lillian Lee, and ShivakumarVaithyanathan. Thumbs up? Sentiment Classification Using Machine Learning Techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, EMNLP ’02, pages 79–86, Stroudsburg, PA,USA, 2002. Association for Computational Linguistics.
[20] Anna Stavrianou and Caroline Brun. Opinion and Suggestion Analysis for Expert Recommendations. In Proceedings of the Workshop on Semantic Analysis in Social Media, pages 61–69, Stroudsburg, PA, USA, 2012.Association for Computational Linguistics.
[21] P.D. Turney. Mining the Web for Synonyms: Pmi-ir versus lsa on toefl. In Proceedings of the 12th European Conference on Machine Learning,pages 491–502. Springer-Verlag, 2001.
[22] Peter Turney. Thumbs up or Thumbs down? Semantic Orientation Applied to Unsupervised Classification of Reviews. Pages 417–424, 2002.
[23] J. Wiebe. Instructions for Annotating Opinions in Newspaper Articles. Department of Computer Science Technical Report tr-02-101, Universityof Pittsburgh, 2002.
[24] JanyceWiebe. Learning Subjective Adjectives from Corpora. In Henry A. Kautz and Bruce W. Porter, editors, AAAI/IAAI, pages 735–740. AAAIPress / The MIT Press, 2000.
[25] JanyceWiebe, Rebecca Bruce, Matthew Bell, Melanie Martin, and Theresa Wilson. A Corpus Study of Evaluative and Speculative Language. In Proceedings of the 2nd ACL SIGdial Workshop on Discourse andDialogue (SIGdial-2001), pages 186–195, Aalborg, Denmark, 2001.
[26] JanyceWiebe and Claire Cardie. Annotating Expressions of Opinions and Emotions in Language. Language Resources and Evaluation. In Language Resources and Evaluation (formerly Computers and the Humanities), pages 165–210, 2005.
[27] Theresa Wilson. Fine-grained Subjectivity and Sentiment Analysis: Recognizing the Intensity, Polarity, and Attitudes of private states. PhD thesis, Intelligent Systems Program, University of Pittsburgh, 2007.