The Design of Global Correlation Quantifiers and Continuous Notions of Statistical Sufficiency

• Main Authors: Nicholas Carrara, Kevin Vanslette
• Format: Article
• Language: English
• Published: MDPI AG 2020-03-01
• Series: Entropy
• Subjects: <i>n</i>-partite information; total correlation; mutual information; entropy; probability theory; correlation
• Online Access: https://www.mdpi.com/1099-4300/22/3/357

Using first principles from inference, we design a set of functionals for the purposes of ranking joint probability distributions with respect to their correlations. Starting with a general functional, we impose its desired behavior through the Principle of Constant Correlations (PCC), which constrains the correlation functional to behave in a consistent way under statistically independent inferential transformations. The PCC guides us in choosing the appropriate design criteria for constructing the desired functionals. Since the derivations depend on a choice of partitioning the variable space into <i>n</i> disjoint subspaces, the general functional we design is the <i>n</i>-partite information (NPI), of which the total correlation and mutual information are special cases. Thus, these functionals are found to be uniquely capable of determining whether a certain class of inferential transformations, <inline-formula> <math display="inline"> <semantics> <mrow> <mi>&#961;</mi> <mover> <mo>&#8594;</mo> <mo>&#8727;</mo> </mover> <msup> <mi>&#961;</mi> <mo>&#8242;</mo> </msup> </mrow> </semantics> </math> </inline-formula>, preserve, destroy or create correlations. This provides conceptual clarity by ruling out other possible global correlation quantifiers. Finally, the derivation and results allow us to quantify non-binary notions of statistical sufficiency. Our results express what percentage of the correlations are preserved under a given inferential transformation or variable mapping.