Detecting Novel Associations in Large Data Sets

• Main Authors: Reshef, David N. (Contributor), Reshef, Yakir (Contributor), Grossman, Sharon Rachel (Contributor), Finucane, Hilary Kiyo (Author), McVean, Gilean (Author), Turnbaugh, Peter J. (Author), Mitzenmacher, Michael (Author), Sabeti, Pardis C. (Author), Lander, Eric Steven (Author)
• Other Authors: Whitaker College of Health Sciences and Technology (Contributor), Massachusetts Institute of Technology. Department of Biology (Contributor), Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science (Contributor), Lander, Eric S. (Contributor)
• Format: Article
• Language: English
• Published: American Association for the Advancement of Science (AAAS), 2014-02-03T13:18:52Z.
• Subjects: Article
• Online Access: Get fulltext

Identifying interesting relationships between pairs of variables in large data sets is increasingly important. Here, we present a measure of dependence for two-variable relationships: the maximal information coefficient (MIC). MIC captures a wide range of associations both functional and not, and for functional relationships provides a score that roughly equals the coefficient of determination (R[superscript 2]) of the data relative to the regression function. MIC belongs to a larger class of maximal information-based nonparametric exploration (MINE) statistics for identifying and classifying relationships. We apply MIC and MINE to data sets in global health, gene expression, major-league baseball, and the human gut microbiota and identify known and novel relationships.
National Institute of General Medical Sciences (U.S.) (Medical Scientist Training Program)