Welfare Properties of Recommender Systems

• Main Author: Zhang, Xiaochen
• Format: Others
• Published: Research Showcase @ CMU 2017
• Subjects: Conflict of Interest; Personalization; Price Discrimination; Recommender Systems; Targetability; Welfare Analysis
• Online Access: http://repository.cmu.edu/dissertations/904; http://repository.cmu.edu/cgi/viewcontent.cgi?article=1943&context=dissertations

Recommender systems are ubiquitously used by online vendors as profitable tools to boost sales and enhance the purchase experience of their consumers. In recent literature, the value created by recommender systems are discussed extensively. In contrast, few researchers look at the negative side of the recommender systems from the viewpoint of policymakers. To fill this gap, I critically investigate the welfare impact of recommender systems (RSs) during my Ph.D. study. The main focus of my Ph.D. dissertation is analyzing whether there exists a conflict of interest between the recommendations provider and its consumers in the electronic marketplace. My dissertation is composed of three parts. In Part I, I evaluate empirically whether in the real world, the profit-driven firm will choose a recommendation mechanism that hurts or is suboptimal to its consumers. In Part II, I analyze the role of personalization technology in the RSs from a unique perspective of how personalization resembles price discrimination as a profitable tool to exploit consumer surplus. In part III, I investigate the vendor’s motivation to increase the level of personalization in two-period transactions. As the RSs are designed by the firm, and the firm’s objective is to maximize profits, the RSs might not maximize consumers’ welfare. In Part I of my thesis work, I test the existence of such a conflict of interest between the firm and its consumers. I explore this question empirically with a concrete RS created by our industry collaborator for their Video-on-Demand (VoD) system. Using a large-scale dataset (300,000 users) from a randomized experiment on the VoD platform, I simulate seven RSs based on an exponential demand model with listed movie orders and prices as key inputs, estimated from the experimental dataset. The seven simulated RSs differ by the assignments of listed orders for selected recommended movies. Specifically, assignments are chosen to maximize profits, consumer surplus, social welfare, popularity (IMDB votes and IMDB ratings), and previous sales, as well as random assignments. As a result, the profit-driven recommender system generates 8% less consumer surplus than the consumer-driven RSs, providing evidence for a conflict of interest between the vendor and its consumers. Major e-vendors personalize recommendations by different algorithms that depend on how much and types of consumer information obtained. Therefore, the welfare evaluations of personalized recommendation strategies by empirical methods are hard to generalize. In Part II of my thesis, I base my analysis of personalization in RSs on a conceptual approach. Under an analytic framework of horizontal product differentiation and heterogenous consumer preferences, the resemblance of personalization to price discrimination in welfare properties is presented. Personalization is beneficial to consumers when more personalization leads to more adoption of recommendations, since it decreases search costs for more consumers. However, when the level surpasses a threshold when all consumers adopt, a more personalized RS decreases consumer surplus and only helps the firm to exploit surplus from consumers. The extreme case of perfect personalization generates the same welfare results as first-degree price discrimination where consumers get perfectly fit recommendations but are charged their willingness-to-pay. As shown in Part II, personalization is always profitable for the monopoly seller. In Part III, I investigate the vendor’s motivation to increase the level of personalization in a two-period transactions. In the first period, consumers do not observe the true quality of the recommendations and choose to accept recommended products or not based on their initial guesses. In the second period, consumers fully learn the quality. The settings of consumer uncertainty and consumer learning incentivize the firm to charge lower-than-exploiting price for recommendations to ensure consumers’ first-period adoptions of the RS. Therefore, uncertainties mediate the conflicts of interest from the vendor’s exploitive behavior even though the vendor might strategically elevate consumers’ initial evaluation to reduce such effect.