Title

Structure Learning Framework Based on Mutual Coherence in Belief Network.

Abstract

In classification, structure prediction from the Bayesian inference model is a highly symbolic formalism for the purpose of retrieving hidden rules in pragmatic situations. Although there are numerous classes of models representing uncertainties; however Bayesian Belief Network (BBN) is the only model explicitly dealing with direct statistical cause-effect relationship based on an established theory of probability. This process comprises of two sequential steps broadly. First step deals with the construction of best suitable structure. The second step is oriented towards parameter learning for the sake of the inference drawn from this structure. In this study, the focus is on the completeness of the first part. We have highlighted some issues related to the hyper-parameters encompassing feature selection versus feature ordering and scoring function which rests at the heart of structure learning. The originality and contribution of this study is bifurcated into two phases.

In the first phase, we have introduced parameter free, decomposable, penalty less factor Non Parametric Factorized Likelihood Metric (NP-FiLM). The data fitting of some of existing scoring metrics are characterized by parameter of external penalty factor; where unfortunately, it is not possible to correctly identify most appropriate penalty factors a prior. On the other hand, some scoring metrics are not potent enough to exhibit balance between overfitting and underfitting of the learnt model. The proposed scoring metric has its root in information theoretic elucidation. The metric is devised to maximize the discriminant function for query variables with respect to the class and other non class variables. We empirically evaluated the proposed metric over an abundant number of natural datasets (fifty UCI dataset). The comparison is made with respect to ten tree classifiers, one regression model and two neural network system. Furthermore, the scoring metric has been examined to six peers scoring metric within the greedy search and hill climbing searching mechanism as well. NPFiLM oriented BBN have been satisfactory found with significant results in a paradigm of classification accuracy with the capability of illustrating the best possible data fitting model in context of hyper-parameters described above.

In the second phase, we have presented an information theoretic criterion Polarity Measure (PM) which is quite useful for feature order sensitive classifiers such as Random Forest (RF) and BBN employing greedy algorithm K2. Both of these classification systems have been shown to be sensitive to the initial ordering of the features. We have illustrated that improvement in classification can be obtained even without ceding variables in case of feature (attribute) ranking sensitive classifiers. We also performed a comparison between BBN and RF classification approaches in the well known feature subset selection and feature ranking problem. The PM measure is devised directly from well renown objective function: conditional likelihood. It posses the capability to discover the degree of explanation made by one feature (attribute)’s state to explain the other feature’s state. The technique has significantly better well performed in BBN and better in RF in comparison to five feature ranking techniques and three well established feature subset selection techniques. The proposed measure PM is quite tractable over large dimensional search spaces with low computational complexity.

Another contribution of this study includes a practical application of structure learning to decision support system for settlements in labor negotiations system and identification of genotype in HCV sequences, where a model learned from the dataset is used to yield swift approximation to counting queries, surpassing in certain aspects other peer state-of-the-art techniques to the same problem.

Download Full Paper