Manual to Automated Testing: An Effort-Based Approach for Determining the Priority of Software Test Automation

Authors: Peter Sabev, Katalina Grigorova

Abstract:

Test automation allows performing difficult and time consuming manual software testing tasks efficiently, quickly and repeatedly. However, development and maintenance of automated tests is expensive, so it needs a proper prioritization what to automate first. This paper describes a simple yet efficient approach for such prioritization of test cases based on the effort needed for both manual execution and software test automation. The suggested approach is very flexible because it allows working with a variety of assessment methods, and adding or removing new candidates at any time. The theoretical ideas presented in this article have been successfully applied in real world situations in several software companies by the authors and their colleagues including testing of real estate websites, cryptographic and authentication solutions, OSGi-based middleware framework that has been applied in various systems for smart homes, connected cars, production plants, sensors, home appliances, car head units and engine control units (ECU), vending machines, medical devices, industry equipment and other devices that either contain or are connected to an embedded service gateway.

Keywords: Automated Testing, Manual Testing, Test Automation, Software testing, Test Prioritization.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1110734

References:

[1] E. Aranha, "Estimating test execution effort based on test specifications", Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil, pp. 29-42, January 2009.
[2] M. Cohn, "Agile estimating and planning", Pearson Education, Prentice Hall, New York, USA, 2005.
[3] J. Fernandes and A. Di Fonzo, "When to automate your testing (and when not to)", July 2010.
[4] S. Yoo and M. Harman, "Regression testing minimization, selection and prioritization: A survey", King’s College London, London, England, March 2012.
[5] B. Marick, "When should a test be automated", California, USA, 1998
[11th International Software Quality Week QualWeek 98 San Francisco, 1998].
[6] D. Mosley and B. Posey, "Just enough software test automation", Yourdon Press, Prentice Hall Professional, Englewood Cliffs, NJ, USA, 2002.
[7] V. Motwani, "The when and how of test automation", Bangalore, India (2001).
[Annual International Software Testing Conference in India, 2001].
[8] G. Robert, "Automation selection criteria – picking the "right" candidates, LogiGear Magazine, 2014.
[9] M. Rouse, "What is requirements stability index (RSI)?", TechTarget, 2005.
[10] E. Dustin, J. Rashka and J. Paul, "Automated software testing". Reading, Addison-Wesley, 1999.
[11] S. Desikan and G. Ramesh, Software testing. Bangalore, India, Dorling Kindersley, India, 2006.
[12] S. Covey, A. Merrill and R. Merrill, First things first. Simon & Schuster, New York, USA, 1994.