A Practical Methodology for Evaluating Water, Sanitation and Hygiene Education and Training Programs

Authors: Brittany E. Coff, Tommy K. K. Ngai, Laura A. S. MacDonald

Abstract:

Many organizations in the Water, Sanitation and Hygiene (WASH) sector provide education and training in order to increase the effectiveness of their WASH interventions. A key challenge for these organizations is measuring how well their education and training activities contribute to WASH improvements. It is crucial for implementers to understand the returns of their education and training activities so that they can improve and make better progress toward the desired outcomes. This paper presents information on CAWST’s development and piloting of the evaluation methodology. The Centre for Affordable Water and Sanitation Technology (CAWST) has developed a methodology for evaluating education and training activities, so that organizations can understand the effectiveness of their WASH activities and improve accordingly. CAWST developed this methodology through a series of research partnerships, followed by staged field pilots in Nepal, Peru, Ethiopia and Haiti. During the research partnerships, CAWST collaborated with universities in the UK and Canada to: review a range of available evaluation frameworks, investigate existing practices for evaluating education activities, and develop a draft methodology for evaluating education programs. The draft methodology was then piloted in three separate studies to evaluate CAWST’s, and CAWST’s partner’s, WASH education programs. Each of the pilot studies evaluated education programs in different locations, with different objectives, and at different times within the project cycles. The evaluations in Nepal and Peru were conducted in 2013 and investigated the outcomes and impacts of CAWST’s WASH education services in those countries over the past 5-10 years. In 2014, the methodology was applied to complete a rigorous evaluation of a 3-day WASH Awareness training program in Ethiopia, one year after the training had occurred. In 2015, the methodology was applied in Haiti to complete a rapid assessment of a Community Health Promotion program, which informed the development of an improved training program. After each pilot evaluation, the methodology was reviewed and improvements were made. A key concept within the methodology is that in order for training activities to lead to improved WASH practices at the community level, it is not enough for participants to acquire new knowledge and skills; they must also apply the new skills and influence the behavior of others following the training. The steps of the methodology include: development of a Theory of Change for the education program, application of the Kirkpatrick model to develop indicators, development of data collection tools, data collection, data analysis and interpretation, and use of the findings for improvement. The methodology was applied in different ways for each pilot and was found to be practical to apply and adapt to meet the needs of each case. It was useful in gathering specific information on the outcomes of the education and training activities, and in developing recommendations for program improvement. Based on the results of the pilot studies, CAWST is developing a set of support materials to enable other WASH implementers to apply the methodology. By using this methodology, more WASH organizations will be able to understand the outcomes and impacts of their training activities, leading to higher quality education programs and improved WASH outcomes.

Keywords: Education and training, capacity building, evaluation, water and sanitation.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1124285

References:

[1] B. Broughton and J. Hampshire, “Bridging the gap: a guide to monitoring and evaluating development projects,” Canberra, Australia: ACFOA, 1997.
[2] D.C. Korten, “Third Generation NGO Strategies: A Key to People-centered Development,” World Development, Vol. 15, Supplement. pp. 145-159, 1987.
[3] UN-Water. Global Annual Assessment of Sanitation and Drinking-Water (GLAAS). Geneva, Switzerland: World Health Organization and UN-Water, 2010.
[4] T.K.K. Ngai, O. Mills, G. French, R. De Oliveira, C. Lepore, M. Mattens, T. Sibanda, M. Sweet, and A. Graves, “A global review of capacity building organizations in water, sanitation and hygiene for developing countries,” 36th WEDC International Conference, Nakuru, Kenya, 2013.
[5] B. E. Coff, T.K.K Ngai, T. Hull Bailey, F. O’Hanlon, J. Price, and T. Viecco, “Measuring and reporting the capacity development performance of organisations in water and sanitation,” 37th WEDC International Conference. Hanoi, Vietnam, 2014.
[6] D.L. Kirkpatrick and J. D. Kirkpatrick, Evaluating Training Programs: The Four Levels (3rd ed). San Francisco, CA: Berrett-Koehler Publishers, 2006.
[7] T.K.K Ngai, B.E. Coff, E. Manzano, K. Seel, and P. Elson, “Evaluation of education and training in water and sanitation technology: Case studies in Nepal and Peru,” 37th WEDC International Conference. Hanoi, Vietnam, 2014.