One Recommender Fits All? An Exploration of User Satisfaction With Text-Based News Recommender Systems

Abstract

Journalistic media increasingly address changing user behaviour online by implementing algorithmic recommendations on their pages. While social media extensively rely on user data for personalized recommendations, journalistic media may choose to aim to improve the user experience based on textual features such as thematic similarity. From a societal viewpoint, these recommendations should be as diverse as possible. Users, however, tend to prefer recommendations that enable “serendipity”—the perception of an item as a welcome surprise that strikes just the right balance between more similarly useful but still novel content. By conducting a representative online survey with n = 588 respondents, we investigate how users evaluate algorithmic news recommendations (recommendation satisfaction, as well as perceived novelty and unexpectedness) based on different similarity settings and how individual dispositions (news interest, civic information norm, need for cognitive closure, etc.) may affect these evaluations. The core piece of our survey is a self-programmed recommendation system that accesses a database of vectorized news articles. Respondents search for a personally relevant keyword and select a suitable article, after which another article is recommended automatically, at random, using one of three similarity settings. Our findings show that users prefer recommendations of the most similar articles, which are at the same time perceived as novel, but not necessarily unexpected. However, user evaluations will differ depending on personal characteristics such as formal education, the civic information norm, and the need for cognitive closure.

Keywords

• algorithm-based recommenders
• diversity
• news recommender design
• recommender field experiment
• reliable surprise