Bart Knijnenburg | Clemson University (original) (raw)

Uploads

Papers by Bart Knijnenburg

As recommender systems are increasingly deployed in the real world, they are not merely tested of... more As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also “online ” with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems. Categories and Subject Descriptors H.1.2. [Models and principles]: User/Machine Systems–software

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 2010

Providing useful recommendations is an important challenge for user-centric media systems. Wherea... more Providing useful recommendations is an important challenge for user-centric media systems. Whereas current recommender systems research mainly focuses on predictive accuracy, we contend that a truly user-centric approach to media recommendations requires the inclusion of user experience measurement. For a good experience, predictive accuracy is not enough. What users like and dislike about our systems is also determined by usage context and individual user characteristics. We therefore propose a generic framework for evaluating the user experience using both subjective and objective measures of user experience. We envision the framework, which will be tested and validated in the large-scale field trials of the FP7 MyMedia project, to be a fundamental step beyond accuracy of algorithms, towards usability of recommender systems.

Taking a step beyond segmentation, privacy researchers have recently proposed privacy personaliza... more Taking a step beyond segmentation, privacy researchers have recently proposed privacy personalization or adaptation as an approach to assist users in their privacy decision making. Analyzing a number of datasets of users' personal information disclosure behavior, we find an interesting phenomenon regarding privacy personalization: the order in which information is requested has an impact on prediction accuracy. We provide evidence that this happens because certain request orders cause people's disclosure behavior to be less variable and thus more predictable. This is an important phenomenon to study, because if request orders indeed influence the variability and predictability of subsequent requests, then adapting the request order to the user may result in positive feedback loops that promote prediction accuracy. We address several possible explanations for this phenomenon, and we propose a study that will help us find out which of these explanations is correct.

As recommender systems are increasingly deployed in the real world, they are not merely tested of... more As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems.

ABSTRACT In a mixed-methods study on adoption of location-sharing social networks (LSSN), we disc... more ABSTRACT In a mixed-methods study on adoption of location-sharing social networks (LSSN), we discovered that variations in adoption and usage behavior could be explained by one's predisposition to communicate in a certain style. Specifically, we found that certain individuals prefer a communication style we call FYI (For Your Information). FYI communicators like to infer availability and to keep in touch with others without having to interact with them, which is the predominant style in current LSSN. Using structural equation modeling on a U.S. nationwide survey (N=1021), we show how the FYI communication style predicts the adoption of LSSN while also showing a negative effect on one's desire to call someone on the phone. Moreover, we find that as age increases, FYI preference significantly decreases. In a follow-on survey (N=180), we refine the FYI construct and show that it affects users' level of disclosure and participation in social media. Furthermore, we show that it completely mediates the effect of certain Big-5 personality traits on social media participation and LSSN usage. The results suggest that to cater to a wider segment of the population, LSSN (and arguably any social media) should support an active communication style.

Personalization relies on personal data about each individual user. Users are quite often relucta... more Personalization relies on personal data about each individual user. Users are quite often reluctant though to disclose information about themselves and to be "tracked" by a system. We investigated whether different types of rationales (justifications) for disclosure that have been suggested in the privacy literature would increase users' willingness to divulge demographic and contextual information about themselves, and would raise their satisfaction with the system. We also looked at the effect of the order of requests, owing to findings from the literature. Our experiment with a mockup of a mobile app recommender shows that there is no single strategy that is optimal for everyone. Heuristics can be defined though that select for each user the most effective justification to raise disclosure or satisfaction, taking the user's gender, disclosure tendency, and the type of solicited personal information into account. We discuss the implications of these findings for research aimed at personalizing privacy strategies to each individual user.

Abstract Prior research shows that a root cause of many privacy concerns in location-sharing soci... more Abstract Prior research shows that a root cause of many privacy concerns in location-sharing social media is people's desire to preserve offline relationship boundaries. Other literature recognizes lying as an everyday phenomenon that preserves such relationship boundaries by facilitating smooth social interactions. Combining these strands of research, one might hypothesize that people with a predisposition to lie would generally have lower privacy concerns since lying is a means to preserve relationship boundaries. We tested this ...

When disclosing information to a recommender system, users need to trade off its usefulness for r... more When disclosing information to a recommender system, users need to trade off its usefulness for receiving better recommendations with the privacy risks incurred through its disclosure. Our paper describes a series of studies that will investigate the use of feed-forward and feedback messages to inform users about the potential usefulness of their disclosure. We hypothesize that this approach will influence the user experience in several interesting ways.

Abstract Past research on location-sharing technologies and social media has uncovered many types... more Abstract Past research on location-sharing technologies and social media has uncovered many types of privacy concerns such as informational privacy, impression management and interactional privacy. We interviewed 21 users and nonusers of location-sharing technology and found that many of these privacy concerns are actually just symptoms of a higherlevel motivation: the desire to preserve one's existing offline relationship boundaries. We confirmed and generalized this finding through a nation-wide survey (N= 1532) and path ...

Chapter 1 discusses aspects that need to be considered for a successful online evaluation. Format... more Chapter 1 discusses aspects that need to be considered for a successful online evaluation. Formative and summative evaluations have different end-goals. While formative evaluations are used to evaluate, redesign and enhance a product, summative evaluations are used to compare products against a common set of evaluation criteria. A clear understanding of the evaluation criteria is needed because otherwise it will be difficult to select the appropriate data collection method and evaluation metrics. Current literature on recommender systems lacks an understanding of what needs to be measured to assess a recommender system. Technology acceptance models, user experience models and prior research on recommender specific structural models are discussed to get a better understanding of the determinants underlying the user experience while interacting with a recommender system. Next, quantitative metrics to measure algorithm performance as well as factors related to the user experience are r...

An important side effect of using recommender systems is a phenomenon called “choice overload”; t... more An important side effect of using recommender systems is a phenomenon called “choice overload”; the negative feeling incurred by the increased difficulty to choose from large sets of high quality recommendations. Choice overload has traditionally been related to the size of the item set, but recent work suggests that the diversity of the item set is an important moderator. Using the latent features of a matrix factorization algorithm, we were able to manipulate the diversity of the items, while controlling the overall attractiveness of the list of recommendations. In a user study, participants evaluated personalized item lists (varying in level of diversity) on perceived diversity and attractiveness, and their experienced choice difficulty and tradeoff difficulty. The results suggest that diversifying the recommendations might be an effective way to reduce choice overload, as perceived diversity and attractiveness increase with item set diversity, subsequently resulting in participa...

Social Network Sites (SNS) are often characterized as a trade- off where users must give up priva... more Social Network Sites (SNS) are often characterized as a trade- off where users must give up privacy to gain social benefits. We investigated the alternative viewpoint that users gain the most benefits when SNSs give them the privacy they desire. Applying structural equation modeling to questionnaire data of 303 Facebook users, we examined the complex relation- ship between privacy and SNS benefits. We found that SNS users whose privacy desires were met reported higher levels of social connectedness (i.e., perceived relational closeness with others) than those who achieved less privacy than they desired. Social connectedness, in turn, played a pivotal role in building social capital (i.e., the benefits derived from relation- ships with others). These findings suggest that more openness may not always be better; SNSs should aim to achieve ‘Pri- vacy Fit’ with user needs to enhance user experience and en- sure sustained use.

Proceedings of the fourth ACM conference on Recommender systems - RecSys '10, 2010

Proceedings of the sixth ACM conference on Security and privacy in wireless and mobile networks - WiSec '13, 2013

Proceedings of the fifth ACM conference on Recommender systems - RecSys '11, 2011

This paper compares five different ways of interacting with an attribute-based recommender system... more This paper compares five different ways of interacting with an attribute-based recommender system and shows that different types of users prefer different interaction methods. In an online experiment with an energy-saving recommender system the interaction methods are compared in terms of perceived control, understandability, trust in the system, user interface satisfaction, system effectiveness and choice satisfaction. The comparison takes into account several user characteristics, namely domain knowledge, trusting propensity and persistence. The results show that most users (and particularly domain experts) are most satisfied with a hybrid recommender that combines implicit and explicit preference elicitation, but that novices and maximizers seem to benefit more from a non-personalized recommender that just displays the most popular items.

As recommender systems are increasingly deployed in the real world, they are not merely tested of... more As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also “online ” with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems. Categories and Subject Descriptors H.1.2. [Models and principles]: User/Machine Systems–software

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 2010

Providing useful recommendations is an important challenge for user-centric media systems. Wherea... more Providing useful recommendations is an important challenge for user-centric media systems. Whereas current recommender systems research mainly focuses on predictive accuracy, we contend that a truly user-centric approach to media recommendations requires the inclusion of user experience measurement. For a good experience, predictive accuracy is not enough. What users like and dislike about our systems is also determined by usage context and individual user characteristics. We therefore propose a generic framework for evaluating the user experience using both subjective and objective measures of user experience. We envision the framework, which will be tested and validated in the large-scale field trials of the FP7 MyMedia project, to be a fundamental step beyond accuracy of algorithms, towards usability of recommender systems.

Taking a step beyond segmentation, privacy researchers have recently proposed privacy personaliza... more Taking a step beyond segmentation, privacy researchers have recently proposed privacy personalization or adaptation as an approach to assist users in their privacy decision making. Analyzing a number of datasets of users' personal information disclosure behavior, we find an interesting phenomenon regarding privacy personalization: the order in which information is requested has an impact on prediction accuracy. We provide evidence that this happens because certain request orders cause people's disclosure behavior to be less variable and thus more predictable. This is an important phenomenon to study, because if request orders indeed influence the variability and predictability of subsequent requests, then adapting the request order to the user may result in positive feedback loops that promote prediction accuracy. We address several possible explanations for this phenomenon, and we propose a study that will help us find out which of these explanations is correct.

As recommender systems are increasingly deployed in the real world, they are not merely tested of... more As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems.

ABSTRACT In a mixed-methods study on adoption of location-sharing social networks (LSSN), we disc... more ABSTRACT In a mixed-methods study on adoption of location-sharing social networks (LSSN), we discovered that variations in adoption and usage behavior could be explained by one's predisposition to communicate in a certain style. Specifically, we found that certain individuals prefer a communication style we call FYI (For Your Information). FYI communicators like to infer availability and to keep in touch with others without having to interact with them, which is the predominant style in current LSSN. Using structural equation modeling on a U.S. nationwide survey (N=1021), we show how the FYI communication style predicts the adoption of LSSN while also showing a negative effect on one's desire to call someone on the phone. Moreover, we find that as age increases, FYI preference significantly decreases. In a follow-on survey (N=180), we refine the FYI construct and show that it affects users' level of disclosure and participation in social media. Furthermore, we show that it completely mediates the effect of certain Big-5 personality traits on social media participation and LSSN usage. The results suggest that to cater to a wider segment of the population, LSSN (and arguably any social media) should support an active communication style.

Personalization relies on personal data about each individual user. Users are quite often relucta... more Personalization relies on personal data about each individual user. Users are quite often reluctant though to disclose information about themselves and to be "tracked" by a system. We investigated whether different types of rationales (justifications) for disclosure that have been suggested in the privacy literature would increase users' willingness to divulge demographic and contextual information about themselves, and would raise their satisfaction with the system. We also looked at the effect of the order of requests, owing to findings from the literature. Our experiment with a mockup of a mobile app recommender shows that there is no single strategy that is optimal for everyone. Heuristics can be defined though that select for each user the most effective justification to raise disclosure or satisfaction, taking the user's gender, disclosure tendency, and the type of solicited personal information into account. We discuss the implications of these findings for research aimed at personalizing privacy strategies to each individual user.

Abstract Prior research shows that a root cause of many privacy concerns in location-sharing soci... more Abstract Prior research shows that a root cause of many privacy concerns in location-sharing social media is people's desire to preserve offline relationship boundaries. Other literature recognizes lying as an everyday phenomenon that preserves such relationship boundaries by facilitating smooth social interactions. Combining these strands of research, one might hypothesize that people with a predisposition to lie would generally have lower privacy concerns since lying is a means to preserve relationship boundaries. We tested this ...

When disclosing information to a recommender system, users need to trade off its usefulness for r... more When disclosing information to a recommender system, users need to trade off its usefulness for receiving better recommendations with the privacy risks incurred through its disclosure. Our paper describes a series of studies that will investigate the use of feed-forward and feedback messages to inform users about the potential usefulness of their disclosure. We hypothesize that this approach will influence the user experience in several interesting ways.

Abstract Past research on location-sharing technologies and social media has uncovered many types... more Abstract Past research on location-sharing technologies and social media has uncovered many types of privacy concerns such as informational privacy, impression management and interactional privacy. We interviewed 21 users and nonusers of location-sharing technology and found that many of these privacy concerns are actually just symptoms of a higherlevel motivation: the desire to preserve one's existing offline relationship boundaries. We confirmed and generalized this finding through a nation-wide survey (N= 1532) and path ...

Chapter 1 discusses aspects that need to be considered for a successful online evaluation. Format... more Chapter 1 discusses aspects that need to be considered for a successful online evaluation. Formative and summative evaluations have different end-goals. While formative evaluations are used to evaluate, redesign and enhance a product, summative evaluations are used to compare products against a common set of evaluation criteria. A clear understanding of the evaluation criteria is needed because otherwise it will be difficult to select the appropriate data collection method and evaluation metrics. Current literature on recommender systems lacks an understanding of what needs to be measured to assess a recommender system. Technology acceptance models, user experience models and prior research on recommender specific structural models are discussed to get a better understanding of the determinants underlying the user experience while interacting with a recommender system. Next, quantitative metrics to measure algorithm performance as well as factors related to the user experience are r...

An important side effect of using recommender systems is a phenomenon called “choice overload”; t... more An important side effect of using recommender systems is a phenomenon called “choice overload”; the negative feeling incurred by the increased difficulty to choose from large sets of high quality recommendations. Choice overload has traditionally been related to the size of the item set, but recent work suggests that the diversity of the item set is an important moderator. Using the latent features of a matrix factorization algorithm, we were able to manipulate the diversity of the items, while controlling the overall attractiveness of the list of recommendations. In a user study, participants evaluated personalized item lists (varying in level of diversity) on perceived diversity and attractiveness, and their experienced choice difficulty and tradeoff difficulty. The results suggest that diversifying the recommendations might be an effective way to reduce choice overload, as perceived diversity and attractiveness increase with item set diversity, subsequently resulting in participa...

Social Network Sites (SNS) are often characterized as a trade- off where users must give up priva... more Social Network Sites (SNS) are often characterized as a trade- off where users must give up privacy to gain social benefits. We investigated the alternative viewpoint that users gain the most benefits when SNSs give them the privacy they desire. Applying structural equation modeling to questionnaire data of 303 Facebook users, we examined the complex relation- ship between privacy and SNS benefits. We found that SNS users whose privacy desires were met reported higher levels of social connectedness (i.e., perceived relational closeness with others) than those who achieved less privacy than they desired. Social connectedness, in turn, played a pivotal role in building social capital (i.e., the benefits derived from relation- ships with others). These findings suggest that more openness may not always be better; SNSs should aim to achieve ‘Pri- vacy Fit’ with user needs to enhance user experience and en- sure sustained use.

Proceedings of the fourth ACM conference on Recommender systems - RecSys '10, 2010

Proceedings of the sixth ACM conference on Security and privacy in wireless and mobile networks - WiSec '13, 2013

Proceedings of the fifth ACM conference on Recommender systems - RecSys '11, 2011

This paper compares five different ways of interacting with an attribute-based recommender system... more This paper compares five different ways of interacting with an attribute-based recommender system and shows that different types of users prefer different interaction methods. In an online experiment with an energy-saving recommender system the interaction methods are compared in terms of perceived control, understandability, trust in the system, user interface satisfaction, system effectiveness and choice satisfaction. The comparison takes into account several user characteristics, namely domain knowledge, trusting propensity and persistence. The results show that most users (and particularly domain experts) are most satisfied with a hybrid recommender that combines implicit and explicit preference elicitation, but that novices and maximizers seem to benefit more from a non-personalized recommender that just displays the most popular items.