Design and Secure Evaluation of Side-Choosing Games

• Online Access: http://hdl.handle.net/2047/D20205279

We present an important, general class of new games, called side-choosing games (SCGs), for "gamifying" problem solving in formal sciences. Applications of SCGs include (1) peer-grading in teaching to (2) studying the evolution of knowledge in formal sciences to (3) organizing algorithm competitions. We view SCGs as a new programming language for human computation for formal problem solving and our interest in this paper is on how to evaluate an SCG tournament fairly and effectively. We observe that a specific kind of collusion, where players lie about their strength and sacrifice themselves, could bias the evaluation of SCG tournaments dramatically. Following the idea of Social Choice Theory in the sense of Arrow, we take an axiomatic approach to guarantee that a specific kind of collusion is impossible. We prove the Collusion- Resistance Theorem as a general principle for designing collusion-resistant evaluations for SCG tournaments. The Collusion-Resistance Theorem is surprising: it tells us to be indifferent to wins but to count certain kinds of losses for scoring players and ranking them. If collusion is not an issue, we offer a family of useful ranking functions which are not collusion-resistant. Limit: 18 pages. July 24-28, '16 The Netherlands.