Taking Learning beyond Kirkpatrick’s Levels: Applying Return on Investment Measurement in Training

Authors: Charles L. Sigmund, M. A. Aed, Lissa Graciela Rivera Picado

Abstract:

One critical component of the training development process is the evaluation of the impact and value of the program. Oftentimes, however, learning organizations bypass this phase either because they are unfamiliar with effective methods for measuring the success or effect of the training or because they believe the effort to be too time-consuming or cumbersome. As a result, most organizations that do conduct evaluation limit their scope to Kirkpatrick L1 (reaction) and L2 (learning), or at most carry through to L4 (results). In 2021 Microsoft made a strategic decision to assess the measurable and monetized impact for all training launches and designed a scalable and program-agnostic tool for providing full-scale L5 return on investment (ROI) estimates for each. In producing this measurement tool, the learning and development organization built a framework for making business prioritizations and resource allocations that is based on the projected ROI of a course. The analysis and measurement posed by this process use a combination of training data and operational metrics to calculate the effective net benefit derived from a given training effort. Business experts in the learning field generally consider a 10% ROI to be an outstanding demonstration of the value of a project. Initial findings from this work applied to a critical customer-facing program yielded an estimated ROI of more than 49%. This information directed the organization to make a more concerted and concentrated effort in this specific line of business and resulted in additional investment in the training methods and technologies being used.

Keywords: evaluation, measurement, return on investment, value