Bayesian Network Model for Students- Laboratory Work Performance Assessment: An Empirical Investigation of the Optimal Construction Approach

Authors: Ifeyinwa E. Achumba, Djamel Azzi, Rinat Khusainov

Abstract:

There are three approaches to complete Bayesian Network (BN) model construction: total expert-centred, total datacentred, and semi data-centred. These three approaches constitute the basis of the empirical investigation undertaken and reported in this paper. The objective is to determine, amongst these three approaches, which is the optimal approach for the construction of a BN-based model for the performance assessment of students- laboratory work in a virtual electronic laboratory environment. BN models were constructed using all three approaches, with respect to the focus domain, and compared using a set of optimality criteria. In addition, the impact of the size and source of the training, on the performance of total data-centred and semi data-centred models was investigated. The results of the investigation provide additional insight for BN model constructors and contribute to literature providing supportive evidence for the conceptual feasibility and efficiency of structure and parameter learning from data. In addition, the results highlight other interesting themes.

Keywords: Bayesian networks, model construction, parameterlearning, structure learning, performance index, model comparison.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1328876

References:

[1] D. Azzi, I. E. Achumba, V. L. Dunn, and G. A. Chukwudebe, "Intelligent Performance Assessment of Students- Laboratory Work in a Virtual Electronic Laboratory Environment", Paper submitted to IEEE Transactions on Learning Technologies, on 09/Dec./2010, for review.
[2] I. E. Achumba, and D. Azzi, "A Virtual Electronic Laboratory for Genuine Practical Experiences: Description and Evaluation", Paper submitted to IEEE Transactions on Learning Technologies, for review. Revised and resubmitted on 09/Dec./2010.
[3] G. F. Cooper and E. Herskovits, "A Bayesian Method for the Induction of Probabilistic Networks from Data", Machine Learning, vol. 9, no. 4, pp. 309-347.
[4] N. Friedman, "The Bayesian Structural EM Algorithm", Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-98), 1998, pp. 129 - 138.
[5] Y. Pan and E. S. Burnside, "The Effects of Training Parameters on Learning a Probabilistic Expert System for Mammography", International Congress Series, vol. 1268, 2004, pp. 1027 - 1032.
[6] M. de Jongh, M. and M. J. Druzdzel, "A Comparison of Structural Distance Measures for Causal Bayesian Network Models", Recent Advances in Intelligent Information Systems, 2009, pp. 443-456. Retrieved September 07, 2010, from http://iis.ipipan.waw.pl/2009/proceedings/iis09-43.pdf.
[7] L. A. ZADEH, "What is Optimal?", IRE Transactions on Information Theory, 1958, pp. 3. Retrieved September 08, 2010, from http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01057441.
[8] M. J. Druzdzel and H. A. Simon, "Causality in Bayesian Belief Networks", Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2006, pp. 3-11.
[9] N. Fenton and M. Neil, "Comment: Expert Elicitation for Reliable System Design", Statistical Science, vol. 21, no. 4, 2006, pp. 451 - 453.
[10] R. Rajabally, P. Sen, S. Whittle, and J. Dalton, "Aids to Bayesian Belief Network Construction", Proceedings of the 2nd International IEEE Conference on Intelligent Systems, vol. 2, 2004, pp. 457 - 461.
[11] B. Das, "Generating Conditional Probabilities for Bayesian Networks: Easing the Knowledge Acquisition Problem", August 4, 2008. Retrieved on February 24, 2010, from http://arxiv.org/ftp/cs/papers/0411/0411034.pdf.
[12] Druzdzel, M. J., & van der Gaag, L. C. (2000). Building Probabilistic Networks: Where Do the Numbers Come from? (Guest Editors- Introduction). IEEE Transactions on Knowledge and Data Engineering, 12(4), 1-485.
[13] K. Ng and B. Abramson, "Consensus Diagnosis: A Simulation Study", IEEE Trans on Systems, Man, and Cybernetics, vol. 22, no. 5, 1992, pp. 916 - 928.
[14] M. J. Druzdzel and A. Onisko, "Are Bayesian Networks Sensitive to Precision of Their Parameters?", Intelligent Information Systems, 2008, pp. 35 - 44. Retrieved on March 10, 2011, from ftp://ftp.pitt.edu/users/d/r/druzdzel/iis08.pdf
[15] P. Myllymaki, "Advantages of Bayesian Networks in Data Mining and Knowledge Discovery", 2010. http://www.bayesit.com/docs/advantages.html
[16] G. Heinrich, "Parameter estimation for text analysis", Technical Note Version 2.4, vsonix GmbH and University of Leipzig, August, 2008. Retrieved on July 19, 2010 from http://www.arbylon.net/publications/text-est.pdf.
[17] R. G. Cowell, "Parameter Learning from Incomplete Data for Bayesian Networks", 1999. Retrieved from http://www.staff.city.ac.uk/~rgc/webpages/aistats99.pdf
[18] L. Oteniya, "Bayesian belief networks for dementia diagnosis and other applications: a comparison of hand-crafting and construction using a novel data driven technique", Unpublished PhD Thesis, Department of Computing Science, University of Stirling, Stirling, FK9 4LA, Scotland
[19] P. Spirtes, C. Glymour, and R. Scheines, "An algorithm for fast recovery of sparse causal graphs", Social Science Computer Review, vol. 9, 1991, pp. 62-72. Retrieved on July 22, 2010, from http://www.hss.cmu.edu/philosophy/techreports/15_Spirtes.pdf.
[20] F. Sahin, M. C. Yavuz, Z. Arnavut, and O. Uluyol, "Fault diagnosis for airplane engines using Bayesian networks and distributed particle swarm optimization", Parallel Computing, vol. 33, no. 2, 2007, pp. 124- 143.
[21] D. M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian Networks: Search Methods and Experimental Results", 1995. Retrieved from http://research.microsoft.com/enus/ um/people/dmax/publications/aistats95.pdf
[22] D. Heckerman, "A Tutorial on Learning With Bayesian Networks", Technical Report MSR-TR-95-06, USA: Microsoft Research. Retrieved from http://research.microsoft.com/pubs/69588/tr-95-06.pdf
[23] W. Lam and F. Bacchus, "Learning Bayesian belief networks: an approach based on the MDL principle", Computational Intelligence, vol. 10, 1994, pp. 269-293.
[24] H. Akaike, "A new look at the statistical model identification", IEEE Transactions on Automatic Control, vol. 19, no. 6, 1974, pp. 716-723.
[25] G. Schwarz, "Estimation the dimension of a model", Annals of Statistics vol. 6, 1978, pp. 462-464.
[26] D. Heckerman, D. Geiger, and D. M. Chickering, "Learning Bayesian Networks: The Combination of Knowledge and Statistical Data", vol. 20, 1995, pp. 197-243, Kluwer Academic Publishers, Hingham, MA, USA.
[27] W. Buntine, "Theory refinement on Bayesian networks", Proceedings of the 7th Annual Conference on Uncertainty in Artificial Intelligence, Los Angeles, CA, 1991, pp. 52 - 60.
[28] S. Yang and K. Chang, "Comparison of Score Metrics for Bayesian Network learning", IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, vol. 32, no. 3, 2002, pp. 419-428. Retrieved on July 22, 2010 from http://volgenau.gmu.edu/~kchang/publications/journal_pdf/com_score_ matrics.pdf.
[29] G. F. Cooper, "A Bayesian Method for Learning Belief Networks that Contain Hidden Variables", AAAI Technical Report WS-93-02, 1993. Retrieved from http://www.aaai.org/Papers/Workshops/1993/WS-93- 02/WS93-02-011.pdf
[30] M. Forster, and E. Sober, "AIC Scores as Evidence - a Bayesian Interpretation", 2010. Retrieved from http://philosophy.wisc.edu/sober/forster%20and%20sober%20AIC%20 Scores%20as%20Evidence%20jan%2028%202010.pdf
[31] D. M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian networks is np-hard", Technical Report MSR-TR-94-17, Microsoft Research, November, 1994. Retrieved on July 22, 2010 from http://research.microsoft.com/apps/pubs/default.aspx?id=69598.
[32] R. W. Robinson, "Counting labelled acyclic digraphs", 1973. In F. Harary (Ed.), New Directions in the Theory of Graphs, New York: Academic Press, pp. 239-273.
[33] S. Anderson, D. Madigan, and M. Perlman, "A characterization of markov equivalence classes for acyclic digraphs", Annals of Statistics, vol. 25, 1997, pp. 505 -541.
[34] M. J. Druzdzel, "GeNIe: A development environment for graphical decision-analytic models", Proceedings of the Annual Symposium of the American Medical Informatics Association, Washington, D.C., 1999, pp. 1206. Retrieved from http://www.pitt.edu/~druzdzel/psfiles/amia99.pdf
[35] NSC (Norsys Software Corp), Netica-J Reference Manual (Version 3.25), 2008. http://www.norsys.com/netica-j/docs/NeticaJ_Man.pdf
[36] ARG (Automated Reasoning Group) (2004), SamIam. http://reasoning.cs.ucla.edu/samiam/
[37] D. M. Chickering, "The WinMine Toolkit", Technical Report: MSRTR- 2002-103, October 2002. Retrieved from http://research.microsoft.com/enus/ um/people/dmax/WinMine/WinMine.pdf
[38] M. Ramoni and P. Sebastiani, "Learning Bayesian networks from incomplete databases", KMi Technical Report KMi-TR-43, Intelligent Data Analysis Journal, vol. 2, no. 1, 1997.
[39] J. Pearl and S. Russell, "Bayesian Networks", November 2000. Retrieved February 12, 2010, from http://www.cs.berkeley.edu/~russell/papers/hbtnn-bn.ps
[40] MR (Microsoft Research), "WinMine Toolkit Tutorial:, Retrieved on February 02, 2011, from http://research.microsoft.com/enus/ um/people/dmax/WinMine/Tutorial/Tutorial.html.
[41] BNTT (Bayesian Network Toolbox Tutorial), "How to use the Bayes Net Toolbox", October, 2007. Retrieved on February 02, 2011 from http://bnt.googlecode.com/svn/trunk/docs/usage.html
[42] D. Pennock, "Evaluating probabilistic predictions", December, 2006. Retrieved from http://blog.oddhead.com/2006/12/26/evaluatingprobabilistic- predictions/
[43] G. Blattenberger and F. Lad, "Separating the Brier Score into Calibration and Refinement Components: A Graphical Exposition", The American Statistician, vol. 39, no. 1, 1985, pp. 26-32. Retrieved from http://www.jstor.org/stable/pdfplus/2683902.pdf
[44] M. Morgan and M. Henrion, Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis, London: Cambridge University Press, 1990.
[45] P. Doshi, L. Greenwald, and J. Clarke, "Towards Effective Structure Learning for Large Bayesian Networks", AAAI Technical Report WS- 02-14, 2002. Retrieved from http://www.aaai.org/Papers/Workshops/2002/WS-02-14/WS02-14- 003.pdf
[46] NSC (Norsys Software Corp), "Advanced Topics: Testing nets with Cases", 2008. http://www.norsys.com/tutorials/netica/secD/tut_D2.htm
[47] I. J. Good, "Rational decisions", Journal of the Royal Statistical Society, vol. 14, 1952, pp. 107-114. In M. Roulston, "The Logarithmic Scoring Rule a.k.a. "ignorance"", Retrieved from http://www.cawcr.gov.au/bmrc/wefor/staff/eee/verif/Ignorance.html
[48] F. V. Jenson, Bayesian Networks and Decision Graphs. Springer, USA, 2001.
[49] M. S. Roulston and L. A. Smith, "Evaluating probabilistic forecasts using information theory", Monthly Weather Review, vol. 130, 2002, pp. 1653-1660.