The Return on Investment Problem in Simulation

Melissa Mathews identifies common mistakes and describes necessary steps to conduct a ROI evaluation.

For simulationists and administrative leaders of simulation programs, understanding how to show and report the simulation return on investment is critical to the growth and sustainment of the program. Leading programs have a continuous uptake of new simulators, task trainers and program models as well as a reporting process to financial stakeholders. This article will identify the common missed elements in most simulation investment proposals, review key measures within a sound program evaluation and the steps taken to conducting a return on investment evaluation.

Advancing simulation to the next level requires making the case for investment.

Many decisions are made in healthcare thatimpact investments made to improve the education of future healthcareproviders, prevent injury and harm to patients as well as those interventionswhich improve the efficiency and quality of care. Financial and administrativedecision makers in healthcare organizations must make intelligent investmentsto reduce risk to organizations within the healthcare system already stressedto reduce healthcare costs and improve quality. Educational leaders are underpressures to improve achievement scores and pass rates, meet the demands of theclinical skill requirements, increase throughput, decrease cost, improveinstruction and deliver quality trained healthcare professionals.

Making the case for investments insimulation as a tool to assess risk, test systems, and train clinicians isoften difficult for simulationists who often enter the profession as cliniciansor educators with little experience in business and finance. Empoweringsimulationists with information on how to present proposals for simulationprograms and simulation technology investment is critical to improvinghealthcare and reducing healthcare costs.

Several things are often missed in makingthe return on investment argument:

The first step to begin the simulationinvestment proposal with a return on investment is to complete a plan for afull circle program evaluation if the technology will be used for professionaldevelopment. Collecting any preliminary or pilot information, even if anecdotalis key to the planning and measurement reporting to the stakeholders makingdecisions. Capturing the current state and gaps here is critical. WhileKirkpatrick is often the model used for program evaluation, I recommend acloser look at Don Moore in his article “Achieving desired results and improvedoutcomes: integrating planning and assessment throughout learning activities”.Moore captures the relevant key aspects of program evaluation for educationgeared towards healthcare professional development. Most importantly, Mooredrives us to measuring to the level of improving healthcare quality andcommunity health as the ultimate measure of success in the use of simulationtechnology. This outcome is the sharp end of the spear in terms of improvingpatient safety and quality care.

Chart A - Program evaluation Moore versus Kirkpatrick.

Next it is important to determine two ormore approaches to closing the performance gap or solving the problem includingthe current approach with or without simulation or different simulationapproaches. Comparing methods shows the stakeholders and financial decisionmaker’s current approaches and possible alternatives. It is also good to showthe potential benefit and yield for low cost effective interventions versusmore expensive effective interventions.

The return on investment should beconducted for all methods. The standard ROI is calculated by beginning tocollect the costs of training. The standard ROI is calculated as follows:

ROI= (net benefits/costs) X100

where net benefits = benefits-costs.

For a decision maker, a program is a goodinvestment if ROI > 0. The chart below demonstrates and provides examples ofmeasures in a step by step process. Notwithstanding, also providing measures ofintangible benefits.

Tangible benefits are those measured incosts and intangible benefits cannot be measured directly but they do have avery significant business impact. Tangible benefits: Making the processes andpersonnel more efficient and effective while reducing the cost of care andservices. Intangible benefits are those things that are difficult to measuredirectly, such as factors related to:

Patients: personalized healthcare leads to improved outcomes and HCAHPSscores.

Professionals: increase engagement and increase experiential learning opportunities.

Families: enhanced understanding, trust, and confidence.

Proceduralists: simulation rehearsal, preplanning and preparation leads to greaterconfidence and understanding. Opportunities to improve processes and/ortechniques through innovation. Improved communication among multi-disciplinaryteam.

Students/ Residents / Clinical Staff: improved procedure,technique, processes, and disease education.

Researchers/ Innovators: providing the tools and biomedicalengineering that allows the “bench to bedside” discovery.

Educators: providing the instructional scaffoldings that lead to improvededucational outcomes.

Marketing: increase the college or hospital’s brand awareness and image.

FinancialStakeholders: decreasing medical errors, decreasingsafety events, reducing malpractice lawsuits. Build case for malpracticeinsurance reduction and insurance reimbursement and common standards.

Publicbranding and relations: Simulation would indirectlyimpact a college or hospital’s reputation by adding another structural resourcethat leads to an improved process of delivering care which leads to improvedoutcomes. Patient Safety is also improved as it is related to the process ofhealthcare delivery.

Chart B - Steps to Measuring Return on Investment.

In conclusion, providing robust simulation proposals involves comparing choices, reporting and including intangible benefits. An approach to reporting the education effectiveness by an in-depth program evaluation is critical to proving the educational success. Finally following the steps to conducting a return on investment will contribute to producing a case for using simulation technology. Taking the time to write and produce proposals that lay out what will be measured and a plan for reporting is critical to improving the chances for receiving financial support for simulation investments from financial leaders at your organizations. Advancing simulation to the next level will take a commitment to producing proposals that are robust and make the case for investment.

About the Author
Melissa Mathews, BSN, MHPE, RN-BC holds a Master’s degree in Health Professional Education from Vanderbilt University. She is a Certified Healthcare Simulation Educator. She is an experienced educator and innovator with 12 years’ experience in simulation-based education programs; designing, developing and evaluating programs. Her focus is on improving performance, patient safety and quality through simulation. She is a member of the Society for Simulation in Healthcare, sits on several technology advisory boards and is the Co-Chair of the Florida Healthcare Simulation Alliance. She is an Advanced Education Sales Executive at SynDaver.

References
Asche, C. V., Kim, M., Brown, A., Golden, A., Laack, T. A., Rosario, J., ... & Okuda, Y. (2018). Communicating Value in Simulation: Cost–Benefit Analysis and Return on Investment. Academic Emergency Medicine, 25(2), 230-237.
Steven Boggs, M. D., & Okuda, Y. (2014). Cutting costs while maintaining quality: how the VA has leveraged simulation. Physician executive, 40(2), 38.
Moore Jr, D. E., Green, J. S., & Gallis, H. A. (2009). Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. Journal of continuing education in the health professions, 29(1), 1-15.
Twigg, D., & McCullough, K. (2014). Nurse retention: a review of strategies to create and enhance positive practice environments in clinical settings. International journal of nursing studies, 51(1), 85-92.
Kirkpatrick, D. L. (1977). Evaluating training programs: Evidence vs. proof. Training Dev J.

Originally published in Issue 4, 2018 of MT Magazine.