Fuzzy Rules Generation and Extraction from Support Vector Machine Based on Kernel Function Firing Signals

Authors: Prasan Pitiranggon, Nunthika Benjathepanun, Somsri Banditvilai, Veera Boonjing

Abstract:

Our study proposes an alternative method in building Fuzzy Rule-Based System (FRB) from Support Vector Machine (SVM). The first set of fuzzy IF-THEN rules is obtained through an equivalence of the SVM decision network and the zero-ordered Sugeno FRB type of the Adaptive Network Fuzzy Inference System (ANFIS). The second set of rules is generated by combining the first set based on strength of firing signals of support vectors using Gaussian kernel. The final set of rules is then obtained from the second set through input scatter partitioning. A distinctive advantage of our method is the guarantee that the number of final fuzzy IFTHEN rules is not more than the number of support vectors in the trained SVM. The final FRB system obtained is capable of performing classification with results comparable to its SVM counterpart, but it has an advantage over the black-boxed SVM in that it may reveal human comprehensible patterns.

Keywords: Fuzzy Rule Base, Rule Extraction, Rule Generation, Support Vector Machine.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1059795

References:

[1] S. Aeberhard, D. Coomans, and O. de Vel, "The Classification Performance of RDA," Tech. Rep., no. 92-01, 1992.
[2] S. Ali and K. A. Smith-Miles, "Improved Support Vector Machine Generalization Using Normalized Input Space," In: A. Sattar and B.H. Kang (Eds.), Artificial Intelligence, LNAI 4304, Springer-Verlag, Berlin, Heidelberg, pp. 362-371, 2006.
[3] R. Andrews, J. Diederich, and A. B. Tickle, "Survey and critique of techniques for extracting rules from trained artificial neural networks," Knowledge-Based Systems, vol. 8, no. 6, pp. 373-389, 1995.
[4] N. Barakat and J. Diederich, "Eclectic Rule-Extraction from Support Vector Machines," International Journal of Computational Intelligence, vol. 2, no. 1, pp. 59-62, 2005.
[5] C. M. Bishop, Neural Networks for Pattern Recognition. Oxford University Press, New York, 1995, pp. 372-375.
[6] C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining and Knowledge Discovery, vol. 2, pp. 121- 167, 1998.
[7] C. Cortes and V. N. Vapnik, "Support Vector Networks," Machine Learning, vol. 20, pp. 273-297, 1995.
[8] J. Diederich (Ed.), "Rule Extraction from Support Vector Machines," Studies in Computational Intelligence, vol. 80, Springer-Verlag, Berlin, Heidelberg, 2008, pp. 3-30.
[9] R. A. Fisher, "The use of multiple measurements in taxonomic problems," Annual Eugenics, vol. 7, Part II, pp. 179-188, 1936.
[10] G. Fung, S. Sandilya, and R. B. Rao, "Rule extraction from linear support vector machines," in Proceedings of the 11th ACM SIGKDD international Conference on Knowledge Discovery in Data Mining, 2005, pp. 32-40.
[11] S. J. Haberman, "Generalized Residuals for Log-Linear Models," in Proceedings of the 9th International Biometrics Conference, Boston, 1976, pp. 104-122.
[12] S. Haykin, Neural Networks, A Comprehensive Foundation, Macmillan, New York, NY, 1994.
[13] J. Huysmans, B. Baesens, and J. Vanthienen, "ITER: an algorithm for predictive regression rule extraction," in 8th International Conference on Data Warehousing and Knowledge Discovery, Springer Verlag, lncs 4081, 2006, pp. 270-279.
[14] J. Huysmans, R. Setiono, B. Baesens, and J. Vanthienen, "Minerva: Sequential Covering for Rule Extraction," IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, vol. 38, no. 2, pp. 299-309, 2008.
[15] J. -S. R. Jang and C. -T. Sun, "Functional equivalence between radial basis function networks and fuzzy inference systems," IEEE Trans. Neural Networks, vol. 4, pp. 156-158, 1992.
[16] J. Jang, C. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing. Prentice Hall International, 1997, pp. 333-342.
[17] E. Kolman and M. Margaliot, "Are artificial neural networks white boxes?" IEEE Trans. Neural Networks, vol. 16, no. 4, pp. 844-852, 2005.
[18] S. Kumar,Neural Networks: A Classroom Approach. McGraw-Hill, International Edition, 2005, pp. 273-304.
[19] K. R. Muller, A. J. Smola, G. Ratsch, B. Scholkopf, J. Kohlmorgen and V. N. Vapnik, "Predicting time series with support vector machines," in ICANN, 1997, pp. 999-1004.
[20] H. Nunez, C. Angulo, and A. Catala, "Rule Extraction Based on Support and Prototype Vectors," in Studies in Computational Intelligence, vol. 80, Springer-Verlag, Berlin, Heidelberg, 2008 pp. 109-134.
[21] V. J. Sigillito, S. P. Wing, L. V. Hutton, and K. B. Baker, "Classification of radar returns from the ionosphere using neural networks," Johns Hopkins APL Technical Digest, vol. 10, pp. 262-266, 1989.
[22] S. Sivanandam, S. Sumathi, and S. Deepa, Introduction to Fuzzy Logic using MATLAB. Springer Verlag, Berlin, Heidelberg, 2007.
[23] V. N. Vapnik, Statistical Learning Theory. John Wiley & Sons, 1998, pp. 375-520.
[24] W. H. Wolberg and O. L. Mangasarian, "Multisurface method of pattern separation for medical diagnosis applied to breast cytology," Proceedings of the National Academy of Sciences, vol. 87, pp. 9193-9196, 1990.