The Fabric of Entropy: A Discussion on the Meaning of Fractional Information

• Main Author: Zhang, Yuan
• Other Authors: O’Connor, Brian C.
• Format: Others
• Language: English
• Published: University of North Texas 2019
• Subjects: information theory; entropy; data visualization
• Online Access: https://digital.library.unt.edu/ark:/67531/metadc1538775/

Why is the term information in English an uncountable noun, whereas in information theory it is a well-defined quantity? Since the amount of information can be quantified, what is the meaning of a fraction of that amount? This dissertation introduces a quasi-entropy matrix which developed from Claude Shannon's information measure as an analytical tool for behavioral studies. Such matrix emphasizes the role of relative characteristics of individual level data across different collections. The real challenge in the big data era is never the size of the dataset, but how data lead scientists to individuals rather than arbitrarily divided statistical groups. This proposed matrix, when combining with other statistical measures, provides a new and easy-to-do method for identifying pattern in a well-defined system because it is built on the idea that uneven probability distributions lead to decrease in system entropy. Although the matrix is not superior to classical correlation techniques, it allows an interpretation not available with traditional standard statistics. Finally, this matrix connects heterogeneous datasets because it is a frequency-based method and it works on the modes of data rather than the means of values. It also visualizes clustering in data although this type of clustering is not measured by the squared Euclidean distance of the numerical attributes.