Implementation of a Distant Learning Physician Assistant Program in Northern Michigan to Address Health Care Provider Shortage: Importance of Evaluation

Authors: Theresa Bacon-Baguley, Martina Reinhold

Abstract:

Introduction: The purpose of this paper is to discuss the importance of both formative and summative evaluation of a Physician Assistant (PA) program with a distant campus delivered through Interactive Television (ITV) to assure equity of educational experiences. Methodology: A needs assessment utilizing a case-control design determined the need and interest in expanding the existing PA program to northern Michigan. A federal grant was written and funded, which supported the hiring of two full-time faculty members and support staff at the distant site. The strengths and weaknesses of delivering a program through ITV were evaluated using weekly formative evaluation, and bi-semester summative evaluation. Formative evaluation involved discussion of lecture content to be delivered, special ITV needs, orientation of new lecturers to the system, student concerns, support staff updates, and scheduling of student/faculty traveling between the two campuses. The summative evaluation, designed from a literature review of barriers to ITV, included 19 statements designed to evaluate the following items: quality of technology (audio, video, etc.), confidence in the ITV system, quality of instruction and instructor interaction between the two locations, and availability of resources at each location. In addition, students were given the opportunity to write qualitative remarks for each course delivered between the two locations. This summative evaluation was given to all students at mid-semester and at the end of the semester. The goal of the summative evaluation was to have 80% or greater of the students respond favorably (‘Very Good’ or ‘Good’) to each of the 19 statements. Results: Prior to the start of the first cohort at the distant campus, the technology was tested. During this time period, the formative evaluations identified key components needing modification, which were rapidly addressed: ability to record lectures, lighting, sound, and content delivery. When the mid-semester summative survey was given to the first cohort of students, 18 of the 19 statements in the summative evaluation met the goal of 80% or greater in the favorable category. When the summative evaluation statements were stratified by the two cohorts, the summative evaluation identified that students at the home location responded that they did not have adequate access to printers, and students at the expansion location responded that they did not have adequate access to library resources. These results allowed the program to address the deficiencies through contacting informational technology for additional printers, and to provide students with knowledge on how to access library resources. Conclusion: Successful expansion of programs to a distant site utilizing ITV technology requires extensive monitoring using both formative and summative evaluation. The formative evaluation allowed for quick identification of issues that could immediately be addressed, both at the planning and developing stage, as well as during implementation. Through use of the summative evaluation the program is able to monitor the success/ effectiveness of the expansion and identify specific needs of students at each location.

Keywords: assessment, distance learning, formative feedback, interactive television (ITV), student experience, summative feedback, support