Predicting webpage credibility using linguistic features

ABSTRACT

The article focuses on predicting trustworthiness from textual content of webpages. The recent work Olteanu et al. proposes a number of features (linguistic and social) to apply machine learning methods to recognize trust levels. We demonstrate that this approach can be substantially improved in two ways: by applying machine learning methods to vectors computed, using psychosocial and psycholinguistic features and in a high-dimensional bag-of-words paradigm of word occurrences. Following Olteanu et al., we test the methods in two classification settings, as a 2-class and 3-class scenario, and in a regression setting. In the 3-class scenario, the features compiled by Olteanu et al. achieve weighted precision of 0.63, while the methods proposed in our paper raise it to 0.66 and 0.70. We also examine coefficients of the models in order to discover words associated with low and high trust.

References

1. B. J. Fogg. Prominence-interpretation theory: explaining how people assess credibility online, 2003.
2. B. J. Fogg, C. Soohoo, D. R. Danielson, L. Marable, J. Stanford, and E. R. Tauber. How do users evaluate the credibility of web sites?: a study with over 2,500 participants, 2003.
3. B. J. Fogg and H. Tseng. The elements of computer credibility, 1999.
4. C. Gaziano and K. McGrath. Measuring the concept of credibility. Journalism Quarterly, 63(3):451--462, 1986.
5. K. D. Giudice. Crowdsourcing credibility: the impact of audience feedback on web page credibility, 2010.
6. C. I. Hovland and W. Weiss. The influence of source credibility on communication effectiveness. Public Opinion Quarterly, 15(4):635--650, 1951.
7. W. D. Indicators. Internet users (per 100 people) | data | table, 2012.
8. D. G. Johnson. Ethics online. Commun. ACM, 40(1):60--65, 1997.
9. E. Kelly and P. Stone. Computer Recognition of English Word Senses. North-Holland, Amsterdam, 1975.
10. K. Krippendorff. Content Analysis: An Introduction to Its Methodology. Sage Publications, 1980.
11. M. R. Morris, S. Counts, A. Roseway, A. Hoff, and J. Schwarz. Tweeting is believing?: understanding microblog credibility perceptions, 2012.
12. R. Nielek, A. Wawer, and A. Wierzbicki. Temporal, cultural and thematic aspects of web credibility. In The proceedings of Social Informatics - 5th International Conference, Lecture Notes in Computer Science, pages 419--428. Springer, 2013.
13. A. Olteanu, S. Peshterliev, X. Liu, and K. Aberer. Web credibility: features exploration and credibility prediction. In Proceedings of the 35th European conference on Advances in Information Retrieval, ECIR'13, pages 557--568, Berlin, Heidelberg, 2013. Springer-Verlag.
14. G. L. Patzer. Source credibility as a function of communicator physical attractiveness. Journal of Business Research, 11(2):229--241, 1983.
15. C. Pornpitakpan. The persuasiveness of source credibility: A critical review of five decades' evidence. Journal of Applied Social Psychology, 34(2):243--281, 2004.
16. J. Schwarz and M. Morris. Augmenting web pages and search results to support credibility assessment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '11, pages 1245--1254, New York, NY, USA, 2011. ACM.
17. J. Sobel. A theory of credibility. Review of Economic Studies, 52(4):557--573, 1985.
18. B. Sternthal, R. Dholakia, and C. Leavitt. The persuasive effect of source credibility: Tests of cognitive response. Journal of Consumer Research, 4(4):252--260, 1978.
19. P. J. Stone, D. C. Dunphy, D. M. Ogilvie, and M. S. Smith. The General Inquirer: A Computer Approach to Content Analysis. MIT Press, 1966.
20. S. Tseng and B. J. Fogg. Credibility and computing technology. Commun. ACM, 42(5):39--44, 1999.

Index Terms

• Applied computing

  • Document management and text processing