Integration of Support Vector Machine and Bayesian Neural Network for Data Mining and Classification

Authors: Essam Al-Daoud

Abstract:

Several combinations of the preprocessing algorithms, feature selection techniques and classifiers can be applied to the data classification tasks. This study introduces a new accurate classifier, the proposed classifier consist from four components: Signal-to- Noise as a feature selection technique, support vector machine, Bayesian neural network and AdaBoost as an ensemble algorithm. To verify the effectiveness of the proposed classifier, seven well known classifiers are applied to four datasets. The experiments show that using the suggested classifier enhances the classification rates for all datasets.

Keywords: AdaBoost, Bayesian neural network, Signal-to-Noise, support vector machine, MCMC.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1077523

References:

[1] T.M. Mitchell, Machine learning. McGraw-Hill, New York, NY, USA, 1997.
[2] E. Al-Daoud, "A comparison between three neural network models for classification problems," Journal of Artificial Intelligence, vol. 2, no. 2, pp. 56-64, 2009.
[3] E. Al-Daoud, "Identifying DNA splice sites using patterns statistical properties and fuzzy neural networks," EXCLI Journal, vol. 8, pp. 195- 202, 2009.
[4] P. Venkatesan and S. Anitha, "Application of a radial basis function neural network for diagnosis of diabetes mellitus," Current Science, vol. 91, no. 9, pp. 1195-1199, Nov. 2006.
[5] R. Bouckaert, "Naive Bayes classifiers that perform well with continuous variables," Lecture Notes in Computer Science, vol 3339, pp. 1089-1094, 2004.
[6] I. Steinwart and A. Christmann, Support Vector Machines. Springer, New York, 2008.
[7] R. Polikar, "Ensemble based systems in decision making," IEEE Circuits and Systems Magazine, vol. 3, pp. 21-45, 2006.
[8] R. M. Neal, Bayesian learning of neural networks, Springer-Verlag, New York, 1996.
[9] Z. Waszczyszyn and L. Ziemiański, "Neurocomputing in the analysis of selected inverse problems of mechanics of structures and materials," Computer Assisted Mechanics and Engineering Sciences, vol. 13, no. 1 pp. 125-159, 2006. www.lce.hut.fi/research/mm/mcmcstuff.
[10] C. W. Hsu, C. C. Chang and C. J. Lin. "A practical guide to support vector classification," Technical report, Department of Computer Science, National Taiwan University. July, 2003.
[11] H. B. Hashemi, A. Shakery and M. P. Naeini, "Protein fold pattern recognition using bayesian ensemble of RBF neural networks" 2009 International Conference of Soft Computing and Pattern Recognition, pp. 436-441. 2009.
[12] CLOP Package http://www.modelselect.inf.ethz.ch