Learning a Flexible K-Dependence Bayesian Classifier from the Chain Rule of Joint Probability Distribution

• Main Authors: Limin Wang, Haoyu Zhao
• Format: Article
• Language: English
• Published: MDPI AG 2015-06-01
• Series: Entropy
• Subjects: Bayesian classifier; chain rule; optimal attribute order; information quantity
• Online Access: http://www.mdpi.com/1099-4300/17/6/3766

As one of the most common types of graphical models, the Bayesian classifier has become an extremely popular approach to dealing with uncertainty and complexity. The scoring functions once proposed and widely used for a Bayesian network are not appropriate for a Bayesian classifier, in which class variable C is considered as a distinguished one. In this paper, we aim to clarify the working mechanism of Bayesian classifiers from the perspective of the chain rule of joint probability distribution. By establishing the mapping relationship between conditional probability distribution and mutual information, a new scoring function, Sum_MI, is derived and applied to evaluate the rationality of the Bayesian classifiers. To achieve global optimization and high dependence representation, the proposed learning algorithm, the flexible K-dependence Bayesian (FKDB) classifier, applies greedy search to extract more information from the K-dependence network structure. Meanwhile, during the learning procedure, the optimal attribute order is determined dynamically, rather than rigidly. In the experimental study, functional dependency analysis is used to improve model interpretability when the structure complexity is restricted.