Precision OS delivers accredited curriculum for orthopedic surgical training

By Sébastien Lozé, industry manager, Simulations, Epic Games - Unreal Engine

Orthopedic surgery is no game. Get it right, and a patient’slife can be transformed for the better. Get it wrong, and they could end upworse off than before you started. So how are two former game developers and anorthopedic surgeon working together to more effectively train practitioners?

A very random meeting

Formed in a Vancouver basement two years ago, Precision OSis now a flourishing business that provides orthopedic surgical training invirtual reality (VR). Two of its three founders, CTO Colin O’Connor and ChiefCreative Officer Roberto Oliveira, had both worked in the video game industryfor decades. After years at Radical Entertainment, Black Box Games, andindustry giant Electronic Arts they helped co-found United Front Gamestogether, where they had critical and commercial success with titles likeModNation Racers and Sleeping Dogs.

In 2016, the pair were looking for something new to gettheir teeth into when they had what Oliveira describes as “a very randommeeting” with orthopedic surgeon Dr. Danny Goel, now CEO of Precision OS. Aftera get-together at a local pub, which included a demo from O’Connor of the newlyreleased HTC Vive, the team started building a VR training platform fororthopedic surgery. Initial feedback from surgeons, residents and devicecompanies was positive, and the three went all-in and started the business.

Rethinking surgicaltraining

Traditionally, surgeons have trained using plastic modelsand actual cadavers. As Goel—who continues to practice as a surgeon—explains,neither of these “simulates” real conditions accurately enough. In the case ofcadavers, the condition of the specimen can negatively impact the experience;although variation is at the heart of medicine, the actual core of thesimulation should be consistent. Similarly, plastic models lack the contextualaspects of real-life surgery. The same is true of medical reference books,which can’t deliver the experiential nature of surgery.

“The interesting part about anatomy is when you look in oneof these medical books, everything's clean and broken apart and you see bonesthat are white and muscles that are very well defined,” says Oliveira. “And thenwhen you go into surgery and you look into the approach or the incision, it'scompletely different than what you would expect.”

For Precision OS’s VR training, the goal is to simulate thereal environment as closely as possible, enabling students to experience whatsurgery looks and feels like. They encourage trainees to make mistakes insimulation—without putting patients at risk. To familiarize themselves with theprocedures and thereby recreate them as closely as possible, Oliveira andO’Connor physically stand behind Goel and watch him operate. They also studymedical books and actual cadavers.

And then it’s a case of recreating what they have witnessedin a real-time VR environment. Having had experience with several game enginesin the past, including writing their own at United Front, Oliveira and O’Connorchose Unreal Engine.

“I know for a fact that Unreal supports more things out ofthe box than any other engine out there,” says O’Connor, explaining the choice.“And I wanted to make sure that we hit that triple-A fidelity mark right fromthe outset.”

With his rendering engineering background, the openness ofthe Unreal Engine platform was also a key factor for O’Connor. “I also wantedto make sure that I had access to the code and could go right down to thehardware layer and internal GPU submission calls, to edge every single bit ofperformance out of the VR experience,” he says.

Recreating operatingtheater reality

To create the virtual patient, the team initially purchasedan anatomy model set, but quickly found the limitations of that, so theystarted modeling their own. Recently, in a bid to increase the authenticity,they had a patient scanned. In selecting their candidate, they ensured that,unlike the stock model, he was of the typical age and physique of someoneneeding the operation—that is to say, an older person with a common bodyhabitus.

The accuracy of the simulation is particularly importantwhen dealing with surgery, where a misrepresentation could have significantlygrave consequences. Oliveira and O’Connor use every trick they’ve ever learnedin game development to make sure they are able to represent each step asfaithfully as possible, and Unreal Engine’s deep and broad feature set is partof the solution.

“With the power behind the Blueprint system, the animationsystem, support for morph targets, vertex animation...Unreal just gives us anarray of technology that we can hook into to solve these problems of recreatinga medical environment,” says O’Connor.

“There are ethical considerations to what we are building,”says Goel. “Misrepresentations and over-optimism of VR are critical elementswhen creating something with consequences to actual patients. We are sensitiveto both and are researching all aspects of virtual reality. A second andimportant element to also consider is the point about empathy. It is importantfor the trainees to remember how their practice has implications to patientlives.”

Teaching illustrativeanatomy

Goel also wants to ensure that the module teaches anatomyillustratively, learning he values in his own work on the operating table.

“During surgery, although you can't see each and everymuscle, an experienced surgeon understands the anatomical landscape very well,”he says, “During surgery, I see surgical anatomy but I am thinking aboutillustrative anatomy. Although you never fully see certain nerves, vessels, andmuscles, in my mind I'm imagining where these structures are, to avoid amisplaced retractor or inadvertent injury.

“Anatomy is a three-dimensional concept that we learn in twodimensions. Unless I have regular unencumbered access to a cadaver, how do Ilearn and reinforce my anatomical learning? This point is really important tous here at Precision OS, where we focus on recreating as much of the realisticillustrative anatomy as possible in 3D. Having a deep understanding andappreciation for anatomy is the foundation of surgery in all specialties.”

Beyond the visual

As well as the visual aspect, the application featuresauditory feedback, so that you can hear the anesthetic machine, or the sound ofa drill or mallet as you use it. And haptics are also employed, but only wherethey are critical to support the training. Goel explains that it’s the abilityto make decisions during the training, and to make mistakes, that forms theirdouble-loop simulation experience inherent in what Anders Ericsson has coined“deliberate practice”.

“The decisions you make prior to and during surgery are howwe impact patient outcomes,” he says. “This decision-making process is what weembed within our simulation modules.”

Selecting thehardware

When discussing haptics, both the cost and the portabilityof the hardware required to support them are also factors in the extent towhich they are used. Many of their customers travel with the educational gear,and need to be able to quickly set it up and tear it down. And the team is keento decrease the disparity in health care that exists in different parts of theworld.

“Adding more complex hardware restricts who could haveaccess,” says Goel. “Impacting the health care disparity that exists in certainparts of the world is a major consideration for us at Precision OS. We havetherefore maintained our focus and dedication to creating the most impactfuleducational software while using the most portable hardware, permitting globaldistribution.”

Increased portability is also a goal so that student doctorscan consume the training at their convenience, in their own home, office or school.Currently the hardware implementation is a laptop tethered to a headset, whichadds an element of friction for transportation, but the team is moving tomobile VR devices and devices like the Oculus Quest within the next year.

Training the next generationof surgeons

So has there been resistance to using this kind oftechnology for training?

“It's quite interesting,” says Oliveira. “We see slightresistance sometimes where we don't expect to see it, like in younger studentsor younger doctors, and then sometimes we expect to see it in the oldergeneration and sometimes we get an incredible reaction there. We’ve neverreally had to push the technology. People seem to understand this is the futureof surgical training. Most organizations are just trying to figure out the bestway to introduce it.”

Overall, as the technology becomes more affordable,accessible and portable, it is seeing wide acceptance. The fact that the systemcan also be used to collect performance data and provide metrics for thestudents on the backend is another clear benefit to those whose mandate it isto train and educate the next generation of surgeons.

Today, Precision OS modules are in use by hundreds ofresidents in the 10 North American universities and institutions that weretheir original partners, and, in conjunction with their other customers, arealso available in countries as far apart as Japan, Switzerland, France andAustralia. With their product for the international organization known as theAO Foundation and a new preoperative planning tool, they plan to educatethousands of people from North America, and then tens of thousands globally.

In May of 2019, Precision OS received accreditation from a provider to the Royal College of Physicians and Surgeons of Canada, enabling their training to be used as the performance appraisal component of continuing medical education (CME) for surgeons. This accreditation, coming as it does from a highly regarded organization, is a validation of the company’s efforts to achieve the highest-quality training through VR.

The marriage of medicine and technology is disrupting thewell-entrenched methods to educate physicians and surgeons. Understanding this,the whole team at Precision OS creates content with a heightened sense ofsocial responsibility. “As an operating surgeon, I know the time and energy wespend on the details above could have significant implications,” says Goel.“What and how the trainees learn is of the utmost importance to us. The trustour partners and users have placed in us is reflected in our content for asingle reason—for them to practice with purpose in virtual reality, so they mayoperate with precision, and truly impact patient care worldwide.”