Statistical Designs for Network A/B Testing

• Main Author: Pokhilko, Victoria V
• Format: Others
• Published: VCU Scholars Compass 2019
• Subjects: A/B testing; Conditional auto-regressive model; D-optimal design; Mixed integer programming; Social network; Re-randomization; Applied Statistics; Design of Experiments and Sample Surveys; Statistical Methodology; Statistical Models; Statistical Theory; Statistics and Probability
• Online Access: https://scholarscompass.vcu.edu/etd/6101; https://scholarscompass.vcu.edu/cgi/viewcontent.cgi?article=7203&context=etd

A/B testing refers to the statistical procedure of experimental design and analysis to compare two treatments, A and B, applied to different testing subjects. It is widely used by technology companies such as Facebook, LinkedIn, and Netflix, to compare different algorithms, web-designs, and other online products and services. The subjects participating in these online A/B testing experiments are users who are connected in different scales of social networks. Two connected subjects are similar in terms of their social behaviors, education and financial background, and other demographic aspects. Hence, it is only natural to assume that their reactions to online products and services are related to their network adjacency. In this research, we propose to use the conditional autoregressive model (CAR) to present the network structure and include the network effects in the estimation and inference of the treatment effect. The following statistical designs are presented: D-optimal design for network A/B testing, a re-randomization experimental design approach for network A/B testing and covariate-assisted Bayesian sequential design for network A/B testing. The effectiveness of the proposed methods are shown through numerical results with synthetic networks and real social networks.