Information Theory for Agents in Artificial Intelligence, Psychology, and Economics

• Main Author: Michael S. Harré
• Format: Article
• Language: English
• Published: MDPI AG 2021-03-01
• Series: Entropy
• Subjects: decision theory; psychology; artificial intelligence; economics; artificial neural networks; neuroscience
• Online Access: https://www.mdpi.com/1099-4300/23/3/310

This review covers some of the central relationships between artificial intelligence, psychology, and economics through the lens of information theory, specifically focusing on formal models of decision-theory. In doing so we look at a particular approach that each field has adopted and how information theory has informed the development of the ideas of each field. A key theme is expected utility theory, its connection to information theory, and the Bayesian approach to decision-making and forms of (bounded) rationality. What emerges from this review is a broadly unified formal perspective derived from three very different starting points that reflect the unique principles of each field. Each of the three approaches reviewed can, in principle at least, be implemented in a computational model in such a way that, with sufficient computational power, they could be compared with human abilities in complex tasks. However, a central critique that can be applied to all three approaches was first put forward by Savage in <i>The Foundations of Statistics</i> and recently brought to the fore by the economist Binmore: Bayesian approaches to decision-making work in what Savage called `small worlds’ but cannot work in `large worlds’. This point, in various different guises, is central to some of the current debates about the power of artificial intelligence and its relationship to human-like learning and decision-making. Recent work on artificial intelligence has gone some way to bridging this gap but significant questions still need to be answered in all three fields in order to make progress on these problems.