Sentiment analysis of reader comments: Automated vs manual text processing

G.K. Gimaletdinova^*, E.Kh. Dovtaeva^**

Kazan Federal University, Kazan, 420008 Russia

E-mail: ^*gim-nar@yandex.ru, ^**emily_dovtaeva@mail.ru

Received December 28, 2020

ORIGINAL ARTICLE

Full text PDF

DOI: 10.26907/2541-7738.2021.1.65-80

For citation: Gimaletdinova G.K., Dovtaeva E.Kh. Sentiment analysis of reader comments: Automated vs manual text processing. Uchenye Zapiski Kazanskogo Universiteta. Seriya Gumanitarnye Nauki, 2021, vol. 163, no. 1, pp. 65–80. doi: 10.26907/2541-7738.2021.1.65-80. (In Russian)

Abstract

The verbal and structural features of the reader comment, a genre of Internet communication, were studied. The method of sentiment analysis (ParallelDots API) was used to reveal and measure the emotive component of the reader comments (N = 3000) in the English and Russian languages. The results obtained were verified by the manual linguistic text analysis. The experts were specialists in the field of philology of the English and Russian languages (N = 6), students of philology, as well as native speakers of the Russian language for whom English is a foreign language, i.e., their level of proficiency is C1 (N = 4). As a result of the comparison of the data collected through the automated and manual text processing, a number of factors that reduce the reliability of the results of automated sentiment analysis of the reader comments were singled out. Difficulties hindering the objective determination of the sentiment by the program were found in the reader comments in both analyzed languages. This is indicative of the structural similarities between the English and Russian reader comments at the lexical and syntactic levels. The feasibility of the mixed automated and manual text processing in order to obtain more detailed and objective data was demonstrated. The results of this work can be used for comparative studies of two or more languages performed by the method of sentiment analysis, as well as for drawing parallels between the lexical, grammatical, and cultural components of languages.

Keywords: text sentiment, sentiment analysis, participatory news article, reader comment, emotiveness

Figure Captions

Fig. 1. Total percentage obtained as a result of the automated and manual sentiment analysis of the reader comments in the English and Russian languages.

Fig. 2. Percentage discrepancy between the positive (according to the software processing) comments and their expert assessment.

Fig. 3. Percentage discrepancy between the neutral (according to the software processing) comments and their expert assessment.

Fig. 4. Percentage discrepancy between the negative (according to the software processing) comments and their expert assessment.

References

1. Topchii I.V. Creative commenting in social media: A review of studies. Izvestiya Ural’skogo Fede­ral’nogo Universiteta. Seriya 1. Problemy Obrazovaniya, Nauki i Kul’tury, 2020, vol. 26, no. 2, pp. 22–28. (In Russian)
2. Gimaletdinova G.K. Linguistic background of participatory news article: On statement of the problem. Politicheskaya Lingvistika, 2012, no 3, pp. 143–148. (In Russian)
3. Nashkhoeva M.R. Linguistic concept of emotions and text emotiveness. Vestnik YuUrGU, 2011, no. 1, pp. 95–98. (In Russian)
4. Pazel’skaya A.G., Solov’ev A.N. The method for identifying emotions in Russian texts. Komp’yuternaya lingvistika i intellektual’nye tekhnologii: Materialy ezhegodnoi mezhdunarodnoi konferentsii “Dialog” [Computational Linguistics and Intelligent Technologies: Proc. Annu. Int. Conf. “Dialogue”], 2011, no. 10, pp. 510–522. (In Russian)
5. Pang B., Lee L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2008, vol. 2, nos. 1–2, pp. 1–135. doi: 10.1561/1500000011.
6. Gimaletdinova G.K. Monologue and dialogue of the reader comments in the English language. Filologiya i kul’tura. Philology and Culture, 2012, no. 3, pp. 30–34. (In Russian)
7. Gimaletdinova G.K., Dovtaeva E.Kh. Sentiment analysis of the reader Internet comment on the political text. Politicheskaya Lingvistika, 2020, no. 1, pp. 42–51. (In Russian)
8. Goroshko E.I., Polyakova T.L. On the development of the genre typology for social media. Zhanry Rechi, 2015, no. 2, pp. 119–127. (In Russian)
9. Goroshko E.I., Zemlyakova E.A. Virtual genre studies: Establishment of a theoretical paradigm. Uchenye Zapiski Tavricheskogo Natsional’nogo Universiteta imeni V.I. Vernadskogo. Seriya “Filologiya. Sotsial’nye Kommunikatsii”, 2011, vol. 24, no. 1-1, pp. 225–237. (In Russian)
10. Dakhalaeva E.Ch. Internet comment and Internet review: Parameters of genre distinction. Sovremennye Problemy Nauki i Obrazovaniya, 2014, no. 6. Available at: http://www.science-education.ru/ru/ article/view?id=16222. (In Russian)
11. Mityagina V.A. Internet commenting as a communicative influencing. In: Zhanry i tipy teksta v nauchnom i mediinom diskurse [Genres and Types of Texts in Scientific and Media Discourse]. Orel, FGBOU VPO “OGIIK”, OOO “Gorizont”, 2012, no. 10, pp. 188–197. (In Russian)
12. Shchipitsina L.Yu. Classification of computer-mediated communication genres based on their functions. Izvestiya RGPU imeni A.I. Gertsena, 2009, no. 114, pp. 171–178. (In Russian)
13. Mel’nik N.V. Linguopersonology of the political Internet comment. Politicheskaya Lingvistika, 2017, no. 5, pp. 47–51. (In Russian)
14. Savel’eva I.V. Linguopersonological potential of the Internet comment. Sibirskii Filologicheskii Zhurnal, 2017, no. 4, pp. 192–201. (In Russian)
15. Stepanova L.N. Commenting in the modern information and communication space: Prospects of linguistic research. Sovremennaya filologiya: Materialy II mezhdunar. nauch. konf. [Modern Philology: Proc. II Int. Sci. Conf.]. Ufa, Leto, 2013, pp. 94–97. (In Russian)
16. Kurash V.V. Stylistic potential of Internet comments. Vestnik MGUP imeni Ivana Fedorova, 2015, no. 2, pp. 99–103. (In Russian)
17. Shkhumishkhova A.R., Kalashaova A.A. Linguistic features of an Internet comment in mass media. Cross Cultural Studies: Education and Science, 2018, no. 3, pp. 387–392. (In Russian)
18. Karpoyan S.M. Epistemic modality and Internet comments. Extended Abstract of Cand. Philol. Diss. Rostov-on-Don, 2014. 26 p. (In Russian)
19. Pasechnaya L.A., Shcherbina V.E. The ways for expressing evaluative attitudes in Internet comments. Izvestiya VGPU, 2020, no. 9, pp. 87–92. (In Russian)
20. Volf E.M. Funktsional’naya semantika otsenki [Functional Semantics of Evaluation]. Moscow, URSS, 2020. 278 p. (In Russian)
21. Lyapun S.V., Sokolova G.V. Verbalization of emotional tension in the Internet comments by readers of the Novaya Gazeta. Vestnik Adygeiskogo Gosudarstvennogo Universiteta. Seriya 2: Filologiya i Iskusstvovedenie, 2016, no. 4, pp. 231–235. (In Russian)
22. Baccianella S., Esuli A., Sebastiani F. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. Proc. 7th Int. Conf. on Language Resources and Evaluation (LREC’10). Valletta, Eur. Lang. Resour. Assoc., 2010, vol. 10, pp. 2200–2204.
23. Semina T.A. Sentiment analysis: Modern approaches and existing problems. Sotsial’nye i Gumanitarnye Nauki. Otechestvennaya i Zarubezhnaya Literatura. Seriya 6. Yazykoznanie, 2020, no. 4, pp. 47–63. (In Russian)
24. Smetanina N.I. Basic tasks of sentiment analysis in social networks. Skif, 2017, no. 15, pp. 96–99. (In Russian)
25. Yurganov A.A. Sentiment analysis as a tool for studying texts. Problemy Sovremennoi Nauki i Obrazovaniya, 2017, no. 29, pp. 39–41. (In Russian)
26. Zvereva P.P. Sentiment analysis of text (based on texts about Russia and the Russians from the New York Times). Vestnik MGOU. Seriya: Lingvistika, 2014, no. 5, pp. 32–37. (In Russian)
27. Kolmogorova A.V. Texts of “Internet confession” with regard to solving the tasks of sentiment analysis. Vestnik NGU. Seriya: Lingvistika i Mezhkul’turnaya Kommunikatsiya, 2019, no. 3, pp. 71–82. (In Russian)

The content is available under the license Creative Commons Attribution 4.0 License.