Software Product Quality Evaluation Model with Multiple Criteria Decision Making Analysis

Authors: C. Ardil

Abstract:

This paper presents a software product quality evaluation model based on the ISO/IEC 25010 quality model. The evaluation characteristics and sub characteristics were identified from the ISO/IEC 25010 quality model. The multidimensional structure of the quality model is based on characteristics such as functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability, and associated sub characteristics. Random numbers are generated to establish the decision maker’s importance weights for each sub characteristics. Also, random numbers are generated to establish the decision matrix of the decision maker’s final scores for each software product against each sub characteristics. Thus, objective criteria importance weights and index scores for datasets were obtained from the random numbers. In the proposed model, five different software product quality evaluation datasets under three different weight vectors were applied to multiple criteria decision analysis method, preference analysis for reference ideal solution (PARIS) for comparison, and sensitivity analysis procedure. This study contributes to provide a better understanding of the application of MCDMA methods and ISO/IEC 25010 quality model guidelines in software product quality evaluation process.

Keywords: ISO/IEC 25010 quality model, multiple criteria decisions making, multiple criteria decision making analysis, MCDMA, PARIS, Software Product Quality Evaluation Model, Software Product Quality Evaluation, Software Evaluation, Software Selection, Software

References:

[1] Yan, M., Xia, X., Zhang, X., Xu, L., Yang, D., Li, S. (2019). Software quality assessment model: a systematic mapping study. Science China Information Sciences, 62, 1-18.
[2] Blin, M., Tsoukiàs, A. (2004). Multi-Criteria Methodology Contribution to the Software Quality Evaluation. Software Quality Journal, 9, 113-132.
[3] ISO/IEC 25010:2011. Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Product quality models.
[4] ISO/IEC 25012:2008. Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Data quality model.
[5] ISO/IEC 25040:2011. Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Evaluation process.
[6] Ardil, C. (2020) A Comparative Analysis of Multiple Criteria Decision Making Analysis Methods for Strategic, Tactical, and Operational Decisions in Military Fighter Aircraft Selection. International Journal of Aerospace and Mechanical Engineering, 14(7), 275 - 288.
[7] Ardil, C. (2020) Aircraft Selection Process Using Preference Analysis for Reference Ideal Solution (PARIS). International Journal of Aerospace and Mechanical Engineering, 14(3), 80 - 93.
[8] Ardil, C. (2020) Regional Aircraft Selection Using Preference Analysis for Reference Ideal Solution (PARIS). International Journal of Transport and Vehicle Engineering, 14(9), 378 - 388.
[9] Ardil, C. (2020) Trainer Aircraft Selection Using Preference Analysis for Reference Ideal Solution (PARIS). International Journal of Aerospace and Mechanical Engineering, 14(5), 195 - 209.
[10] Saaty, T. L. (1990). How to make a decision: The Analytic Hierarchy Process. European Journal of Operational Research, 48(1), 9-26. doi: 10.1016/0377-2217(90)90057-I
[11] Saaty, T. L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, 1(1), 83-98. doi: 10.1504/IJSSCI.2008.017590
[12] Saaty, T.L. (1980). Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. McGraw-Hill, New York.
[13] Opricovic, S. (1998). Multicriteria Optimization of Civil Engineering Systems. PhD Thesis, Faculty of Civil Engineering, Belgrade (in Serbian).
[14] Opricovic, S. (2007). A fuzzy compromise solution for multicriteria problems. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 15(3), 363–380.
[15] Opricovic, S., Tzeng, G.-H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156(2), 445–455.
[16] Brans JP., Mareschal B. (2005). Promethee Methods. In: Multiple Criteria Decision Analysis: State of the Art Surveys. International Series in Operations Research & Management Science, vol 78, pp 163-186. Springer, New York, NY. https://doi.org/10.1007/0-387-23081-5_5.
[17] Brans, J., Ph. Vincke. (1985). A Preference Ranking Organisation Method: (The PROMETHEE Method for Multiple Criteria Decision-Making). Management Science, 31(6), 647-656.
[18] Brans, J.P., Macharis, C., Kunsch, P.L., Chevalier, A., Schwaninger, M., (1998). Combining multicriteria decision aid and system dynamics for the control of socio-economic processes. An iterative real-time procedure. European Journal of Operational Research 109, 428-441.
[19] Brans, J.P., Vincke, Ph., Mareschal, B., (1986). How to select and how to rank projects: the PROMETHEE method. European Journal of Operational Research, 24, 228-238.
[20] Hwang, C.L.; Yoon, K. (1981). Multiple Attribute Decision Making: Methods and Applications. New York: Springer-Verlag.
[21] Chu, T.C. (2002. Facility location selection using fuzzy TOPSIS under group decisions”, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, Vol. 10 No. 6, pp. 687-701.
[22] Choudhary, D. and Shankar, R. (2012. A STEEP-fuzzy AHP-TOPSIS framework for evaluation and selection of thermal power plant location: a case study from India”, Energy, Vol. 42 No. 1, pp. 510-521.
[23] Zavadskas, E.K., Mardani, A., Turskis, Z., Jusoh, A., Nor, K.M. (2016) Development of TOPSIS method to solve complicated decision-making problems: An overview on developments from 2000 to 2015. International Journal of Information Technology & Decision Making, 15, 645-682.
[24] Roy, B. (1991). The outranking approach and the foundation of ELECTRE methods. Theory and Decision, 31(1), 49–73.
[25] Fei, L., Xia, J., Feng, Y., Liu, L. (2019) An ELECTRE-Based Multiple Criteria Decision Making Method for Supplier Selection Using Dempster-Shafer Theory. IEEE Access, 7, 84701-84716.
[26] Ardil, C. (2017). Applying Fuzzy Logic Theory to Performance Management. PressAcademia Procedia, 5 (1), 153-162 . DOI: 10.17261/Pressacademia.2017.584
[27] Ardil, C. (2018). Multicriteria Decision Analysis for Development Ranking of Balkan Countries. International Journal of Computer and Information Engineering, 12(12), 1118 - 1125.
[28] Ardil, C. (2018) Multidimensional Compromise Optimization for Development Ranking of the Gulf Cooperation Council Countries and Turkey. International Journal of Mathematical and Computational Sciences Vol:12, No:6, 131-138.
[29] Ardil, C. (2018) Multidimensional Compromise Programming Evaluation of Digital Commerce Websites. International Journal of Computer and Information Engineering Vol:12, No:7, 556-563.
[30] Ardil, C. (2018) Multidimensional Performance Tracking. International Journal of Computer and Systems Engineering, Vol:12, No:5,320-349
[31] Ardil, C. (2019) Aircraft Selection Using Multiple Criteria Decision Making Analysis Method with Different Data Normalization Techniques. International Journal of Industrial and Systems Engineering, Vol:13, No:12, 744-756.
[32] Ardil, C. (2019) Aircraft Selection Using Multiple Criteria Decision Making Analysis Method with Different Data Normalization Techniques. International Journal of Industrial and Systems Engineering, Vol:13, No:12, 744-756.
[33] Ardil, C. (2019) Fighter Aircraft Selection Using Technique for Order Preference by Similarity to Ideal Solution with Multiple Criteria Decision Making Analysis. International Journal of Transport and Vehicle Engineering, Vol:13, No:10, 649-657.
[34] Ardil, C. (2019) Military Fighter Aircraft Selection Using Multiplicative Multiple Criteria Decision Making Analysis Method. International Journal of Mathematical and Computational Sciences, 13(9), 184 - 193.
[35] Ardil, C. (2019) Scholar Index for Research Performance Evaluation Using Multiple Criteria Decision Making Analysis. International Journal of Educational and Pedagogical Sciences, Vol:13, No:2, 93-105.
[36] Ardil, C. (2020) Facility Location Selection using Preference Programming. International Journal of Industrial and Systems Engineering, 14(1), 1 - 12.
[37] Ardil, C., Bilgen, S. (2017) Online Performance Tracking. SocioEconomic Challenges, 1(3), 58-72. ISSN (print) – 2520-6621.
[38] Ardil, C., Pashaev, A. M., Sadiqov, R.A., Abdullayev, P. (2019) Multiple Criteria Decision Making Analysis for Selecting and Evaluating Fighter Aircraft. International Journal of Transport and Vehicle Engineering, Vol:13, No:11, 683-694.
[39] Jato-Espino, D., Castillo-López, E., Rodríguez-Hernández, J., Canteras-Jordana, J.C. (2014) A review of application of multi-criteria decision making methods in construction. Automation in Construction, 45, 151-162.
[40] Mardani, A., Jusoh, A., Md Nor, K., Khalifah, Z, Zakwan, N., Valipour, A. (2015) Multiple criteria decision-making techniques and their applications – a review of the literature from 2000 to 2014. Economic Research-Ekonomska Istraživanja, 28, 516-571.
[41] Mardani, A., Jusoh, A., Zavadskas, E.K., Kazemilari, M.; Ungku, N.U.A., Khalifah, Z. (2016) Application of Multiple Criteria Decision Making Techniques in Tourism and Hospitality Industry: a Systematic Review. Transformations in Business & Economics, 15, 192-213.
[42] Mardani, A., Jusoh, A., Zavadskas, E.K.; Khalifah, Z., Nor, K.M.D. (2015) Application of multiple-criteria decision-making techniques and approaches to evaluating of service quality: a systematic review of the literature. Journal of Business Economics and Management, 16, 1034-1068.
[43] Zeleny, M. 1982. Multiple Criteria Decision Making. New York: McGraw-Hill.