Improving Classification in Bayesian Networks using Structural Learning

Authors: Hong Choon Ong

Abstract:

Naïve Bayes classifiers are simple probabilistic classifiers. Classification extracts patterns by using data file with a set of labeled training examples and is currently one of the most significant areas in data mining. However, Naïve Bayes assumes the independence among the features. Structural learning among the features thus helps in the classification problem. In this study, the use of structural learning in Bayesian Network is proposed to be applied where there are relationships between the features when using the Naïve Bayes. The improvement in the classification using structural learning is shown if there exist relationship between the features or when they are not independent.

Keywords: Bayesian Network, Classification, Naïve Bayes, Structural Learning.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1335262

References:

[1] A. Darwiche, Modelling and Reasoning with Bayesian Networks. Cambridge University Press, Los Angeles, 2009.
[2] J. Pearl, Probabilistic Reasoning in Intelligent Systems: networks of plausible inference. Morgan Kaufmann, 1988.
[3] M. M. Mazid, S. B M. Shawkat Ali and K. S. Tickle, Input space reduction for Rule Based Classification. WSEAS Transactions on Information Science and Applications. 2010, Vol. 7, Issue 6, pp. 749- 759.
[4] J. R. Quinlan, C4. 5: programs for machine learning. San Mateo, CA: Morgan Kaufmann, 2003
[5] I. H. Witten and E. Frank, Generating accurate rule sets without global optimization, in Proceedings of the Fifteenth International Conference, San Francisco, CA, 1998.
[6] V. N. Vapnik, The nature of statistical learning theory: Springer Verlag, Heidelberg, DE, 1995.
[7] S. L. Ang, H. C. Ong, and H. C. Low, Criterion in selecting the clustering algorithm in Radial Basis Functional Link Nets WSEAS Transactions on Systems. 2008, Vol. 11, Issue 7, pp. 1290-1299.
[8] R. Agrawal and R. Srikant, Fast algorithms for mining association rules, in The 20th Int'l Conference on Very Large Databases, Santiago, Chile, 1994.
[9] J. Cheng and R. Greiner, Learning Bayesian Belief Network Classifiers: Algorithms and System, Proceedings of the 14th Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence 2001, pp. 141 - 151.
[10] A. Almonayyes, Multiple Explanations Driven Naïve Bayes Classifier, Journal of Universal Computer Science 2006, Vol. 12, Issue 2, pp. 127- 139.
[11] N. Friedman, D. Geiger and M. Goldszmidt, Bayesian Network Classifiers, Machine Learning, Vol. 29, 1997, pp. 131-163.
[12] Siavash Asadi Ghajarloo. Mining Implicit Knowledge to Predict Political Risk by Providing Novel Framework with using Bayesian Network. World Academy of Science, Engineering and Technology. 2011, Vol. 74, pp. 656-663.
[13] M. G. Madden, On the classification performance of TAN and general Bayesian networks Knowledge-Based Systems, 2009, Vol. 22, Issue 7, pp. 489-495.
[14] K. M. Al-Aidaroos, A. A. Bakar, and Z. Othman, Naive Bayes variants in classification learning, International Conference on Information Retrieval and Knowledge Management 2010, pp. 276-281.
[15] P. Sebastiani, M. M. Abad, and M. F. Ramoni, Bayesian Networks. Data Mining and Knowledge Discovery Handbook, Eds. Maimon, O. and Rokach, L. Part 2, Chapter 10, 175-208, Springer, New York 2010.
[16] M. Scutari, Learning Bayesian Networks with the bnlearn R Package, Journal of Statistical Software, 2010, Vol. 35, Issue 3, pp. 1-22.
[17] M. Scutari and K. Strimmer, Introduction to Graphical Modelling. Chapter for the upcoming Handbook of Statistical Systems Biology Balding, D., Stumpf, M., Girolami, M. eds. 21 pages. 2010.
[18] I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd edition Morgan Kaufmann, San Francisco, 2005.
[19] G. F. Cooper and E. A. Herskovits, Bayesian method for the induction of probabilistic networks from data. Machine Learning, 1992, Vol. 9, pp. 309-347.
[20] A. Frank, and A. Asuncion, UCIMachine Learning Repository
[http://archive.ics.uci.edu/ml], (Irvine, CA: University of California, School of Information and Computer Science, 2010).
[21] G. Yingyu, L. Chunping and Qin,Y. Study on Factors of Floating Women-s Income in Jiangsu Province Based on Bayesian Networks. Advances in Neural Network Research and Applications, Lecture Notes in Electrical Engineering 2010, Vol. 67, Issue 9, pp. 819-827.
[22] R Development Core Team. A Language and Enviroment for Statistical Computing.
[http://www.R-project.org]. R Foundation for Statistical Computing, Vienna, Austria, 201.