A Recipe for the Estimation of Information Flow in a Dynamical System

• Main Authors: Deniz Gencaga, Kevin H. Knuth, William B. Rossow
• Format: Article
• Language: English
• Published: MDPI AG 2015-01-01
• Series: Entropy
• Subjects: transfer entropy; information flow; statistical dependency; mutual information; Shannon entropy; information-theoretical quantities; Lorenz equations
• Online Access: http://www.mdpi.com/1099-4300/17/1/438

Information-theoretic quantities, such as entropy and mutual information (MI), can be used to quantify the amount of information needed to describe a dataset or the information shared between two datasets. In the case of a dynamical system, the behavior of the relevant variables can be tightly coupled, such that information about one variable at a given instance in time may provide information about other variables at later instances in time. This is often viewed as a flow of information, and tracking such a flow can reveal relationships among the system variables. Since the MI is a symmetric quantity; an asymmetric quantity, called Transfer Entropy (TE), has been proposed to estimate the directionality of the coupling. However, accurate estimation of entropy-based measures is notoriously difficult. Every method has its own free tuning parameter(s) and there is no consensus on an optimal way of estimating the TE from a dataset. We propose a new methodology to estimate TE and apply a set of methods together as an accuracy cross-check to provide a reliable mathematical tool for any given data set. We demonstrate both the variability in TE estimation across techniques as well as the benefits of the proposed methodology to reliably estimate the directionality of coupling among variables.