Born in Switzerland: Regulator-Qualified VR Flight Training

“In four years, we will take a look.” That’s a typical response when pilots, instructors and operators first hear about a virtual reality-based simulator, regulator-qualified for training credits. “How about now?” VRM Switzerland has responded. CAT Editor Rick Adams, FRAeS, got an inside look at an historic event.

We’re used to most things touted as ‘revolutionary’ or ‘breakthrough’ not living up to their hype. In an effort to gain attention in an over-saturated stream of media messages, new products or services promise astounding benefits but rarely deliver.

In this case, the reverse is true. Low expectations quickly and thoroughly replaced with ‘Wow. This could change everything.’

When I visited VRM Switzerland in Dubendorf, outside Zurich, I was expecting to see a nascent attempt, a rudimentary iteration of an emerging technology which someday might fit into the mix of tools used to teach pilots how to fly and handle unusual situations.

The conventional wisdom has been that VR is not yet sufficiently mature for the challenges of flight training. Inadequate resolution in the headsets, etc. Fine for maintenance training, maybe systems and procedures, or practising parachute jumping. But at least 2, 3, 5 years away from delivering realistic flying experiences.

With the meeting scheduled for a Saturday, we planned to meet up with friends in the area for dinner and also view the iconic statue of Swiss legend William Tell on the drive home.

The first surprise was learning that the following Monday, an entourage of EASA examiners and 15 highly experienced helicopter pilots from around Europe would be arriving to undertake an exhaustive analysis of VRM’s first Virtual Reality Flight Simulation Training Device (VR-FSTD).

Not an informational visit. A formal review for the purpose of qualifying, for training credit, the world’s first VR-based flight training device.

Algorithmic Reality

VR and its cousins, AR, MR and XR, are the current shiny new thing that just about everyone is trying to apply to just about everything. Including the aviation training community.

We’ve seen any number of demonstrations (at least pre-pandemic, when there were live conferences) of exhibit visitors strapping on a VR helmet and buckling into a motion-mounted seat in a facsimile of flight training.

I learned at VRM Switzerland just how much more sophisticated development needs to be to create an immersive environment that is acceptable to pilots and does not lead to negative training.

For example, something as basic as calculating the eyepoint of the pilot in order to present accurate images in the VR display. In a dome display, such as are widely used in current Level D full-flight simulators, the projected image is two-dimensional; it does not take into account the movement of the pilot’s head. So objects in the field of view may be misrepresented as to their location.

For mirrors mounted on the side of the helicopter, or on a skid, the fixed-eyepoint FFS dome image does not change with head movement either. With three-dimensional VR, the reflected image shifts according to what the pilot would see in the mirrors in the real-world.

In addition, whereas dome sims or even flat-panel flight trainers calculate for a single eyepoint, VR must render separate images for both the left and right eyes of the pilot (which are a couple of centimetres apart). Factoring the headset and possibly two mirrors, that’s up to six separate eyepoint calculations.

Then there’s the motion factor. “Typical motion cueing is not created for 3D visual assistance,” Fabian (Fabi) Riesen, VRM CEO, explained to me. “Using a VR headset in an FFS, the motion cueing will not work. You’ll get sick.” In a high roll rate situation, or for a Vuichard vortex scenario, he added, “an FFS is not good enough.” VRM’s solution is a motion-compensation algorithm designed to minimise disconnects between the pilot’s perception and reality. “There are a lot of small details,” Riesen said.

To determine the locations of the pilot’s moving eyes, there’s an eye-tracking system built into the Varjo VR-3 head-mounted display. VRM has also designed a camera and sensor apparatus on the perimeter of the device which monitors the head and body position. This “pose tracking” data is used to calculate eye position for out-the-window imagery, but also to create a virtual image that enables the pilot to “see” an avatar of his own legs, arms and hands as he moves to touch controls in the cockpit.

The simulator cockpit is a combination of real switches and knobs for haptic feedback as necessary and a virtual representation of instruments and displays.

(In a mixed-reality approach, some developers use headset-mounted cameras to view a realistic or actual cockpit. But this adds yet another two calculations – for the distance between the camera and the pilot’s eye – one of the reasons Riesen says MR “is not good enough that you can qualify it” for flight training.)

Because of the complexities of adapting VR technology for what is perhaps aviation’s most challenging flight environment, VRM Switzerland has created most of the elements of its trainer in-house – electric 6 DoF motion platform, vibration, control loading, 8K image generator, full-body position tracker, 3D aural cueing, cockpit display and haptics, and the integrated software to drive them. Everything except the Varjo headset, which was chosen just prior to the EASA evaluation.

VRM even created its own set of tools to capture the underlying flight model data in an instrumented aircraft.

As well, to address the regulations requiring an FSTDO (Flight Simulation Training Device Organisation) to be responsible for quarterly data measurements – an infrastructure which might make it nigh impossible for small helicopter operators to locate a simulator at their home base, VRM Switzerland created what’s essentially a virtual FSTDO directly with EASA. VRM’s turnkey package also manages maintenance on the trainers, much of it by remote access. The result: pilots can be training whenever they’re not flying.

Unencumbered by Tradition

What’s especially surprising is that the 30 or so mostly young engineers who have developed this breakthrough simulator do not come from traditional aviation training backgrounds. In essence, they started with a classic clean sheet, and have done the fundamental research to figure out what was needed and how to best make it happen.

Riesen explained VRM follows a build-measure-learn methodology advocated by start-up guru Eric Reis. “If something is obvious, there’s no need to reinvent things. If it’s not clear, we build, measure whether the assumption is good or not, and then take learnings from there,” Riesen said. “Then we document the learning so the next time the question comes back, we understand why we chose that way.”

VRM sought extensive feedback from several hundred helicopter pilots during development. “They need to see and feel the helicopter, or they cannot give the right feedback”, noted Riesen. A key resource has been the pilots at Air Zermatt, one of the simulator launch customers: “We decided to go for the most difficult. If we can convince Air Zermatt that we build the best simulators, once we have that we can go in all directions. If they believe it’s good, you know it’s good”.

The first VRM sim, replicating a Robinson R22 single-engine, twin-blade light utility helicopter, was qualified by the EASA team, under current regulations, as an FNPT II, and enables training credits up to five hours for a PPL licence, 10-20 hours for a CPL(H), and five hours of night practice.

As part of their research, four of VRM’s employees, who had never flown before, were trained by a flight instructor on the VR simulator. Then, their first time in a Robinson aircraft, they were able to take-off, hover and land – with no intervention from the flying instructor. When VRM posted a video of the event, a skeptic commented, “This is fake.” Riesen responded, “We know it’s not fake.”

Riesen is confident that under EASA’s proposed new FSTD regulations, NPA 2020-15, expected to take effect in 2022 or 2023, the VRM designs will be qualified to take on a more substantial portion of the training curricula.

Since 2019, VRM has had an Innovation Partnership with EASA, in part to help define the helicopter regulations which will replace CS-FSTD(H). Nominated by the European Helicopter Association, two VRM leaders are on the Rulemaking Task group – Andreas Gafner and Christian Marty. VRM is pursuing a similar collaboration with the US Federal Aviation Administration.

The next aircraft type expected to be qualified is for the Airbus H125 Écureuil (former AS350). Riesen told us the VRM simulator should be qualified at a higher level than the R22 – including proficiency checks during recurrent training. In effect, obviating the need for a much more expensive FFS. Indeed, Riesen said test pilots told them the VR device can perform some tasks which are not possible in current Level Ds.

I saw several VR-FSTDs in various stages of completion at the Dubendorf facility. However, VRM is limited to offering 15 “qualification slots” for 2021, as EASA has only so much examiner team time available for the multi-day process.

Norwegian company Helitrans has already reserved two spots of the H125/AS350 for this year, and confirmed options for the R44, AW09, H135 and H145.

Multiple cameras mounted around the device frame enable an instructor to see the pilot’s view, the entire cockpit, or the aircraft position in the environment. Multi-crew training would be an option with two coupled devices. Image credit: Halldale Media/Rick Adams.

From Students to Teachers

The germination of the company in 1998 was an engineering student project to build a rail/train VR simulator. Riesen said they considered VR technology too limited at the time to consider flight training.

About 2013 they began experimenting with high-fidelity, high-dynamic motion platforms coupled with virtual reality, building a first prototype, and then collaborating with Dr. Guido Schuster at OST, a university in Rapperswil, Switzerland, on research with a goal of “bringing VR together with the human body.”

The first technology challenge was motion compensation. “You need to go deep into the graphic engine,” explained Riesen, “and you need a motion platform you can control at a very high rate.”

The second challenge: “You need to provide, in measures, that we are doing this right. The Qualification Test Guide (QTG) has very precise values. That’s something which did not yet exist [for VR devices], and we needed to develop measurement systems for that.”

For the motion-control element, VRM partnered with a pair of physicists who were developing game software to train hobbyists how to fly remote-control model helicopters. Because of the aggressive manouevering of such scale models, the simulation requires a very high frequency. VRM agreed to licence the IPACS software for commercial applications.

Three years ago, VRM tested the market waters with an F/A-18 fixed-wing fighter VR/motion demo at the ITEC show in Stuttgart, Germany. Soon after, Riesen received a phone call “from someone claiming he is EASA. ‘We saw your video, and we would like to visit you and try it out’, the caller said”. Riesen asked the caller to send an email: “I really thought it was a prank.”

The visit coincided with EASA’s new Rotorcraft Safety Roadmap, which had identified training issues and lack of affordable, realistic training devices as significant contributors to a too-high helicopter accident rate in Europe.

“EASA was proactive to reach out and see what’s around”, Riesen praised, “the opposite of what you often hear about authorities. They were saying we have to address the problem, and let’s think outside the box”.

In December 2018, VRM’s Riesen was a keynote speaker at EASA’s Rotorcraft Symposium in the Agency’s headquarters city of Cologne, Germany.

No Garage, How About the Living Room?

We’ve often heard the proverbial tales of mega tech companies such as Apple which were started in a garage. VRM began in Fabi’s living room.

As a five-year-old, he had all but abandoned his dream of becoming a pilot – on the premise that eyeglasses disqualified him. His childhood friend grew up to be a fighter pilot. But in the interim between the railroad simulator and the prototype VRM aviation device, Riesen earned his fixed-wing PPL, training in Grob, Piper and Cessna aircraft.

He was also exposed to a low-level FTD of the era, which left him wanting more. After acing a VOR intersection task in about 10 minutes, there was nothing more he could learn in the device. (He still has 4.5 hours of paid-for credit, though the device is no longer available.)

Working as a Cisco engineer by day, Riesen cobbled together a home-grown simulator during evenings and weekends… in his living room… over about 18 months. (And he’s still married!) His children even helped with some of the wiring and coding.

Changing the Pilot World?

Has a new Swiss legend been born? Image credit: Halldale Media/Rick Adams.

I first came across VRM (then known as VR Motion) a couple years ago when writing about the early development of the SH09 by Swiss Marenco, now Kopter/Leonardo, an aircraft described as the “Swiss Army Knife of helicopters”. Kopter told me they were planning a virtual reality approach to pilot training, which seemed wishful at the time, given the early negative feedback based on consumer VR headsets. Then Kopter-CEO Andreas Löwenstein told me, “When you come into the market with a quite sophisticated single-engine helicopter, you must be obsessed by safety. We do not have a track record in terms of reliability and safety so we will emphasise from the beginning our pilots must be well trained”. He called VR Motion “a very innovative company bringing new technology to the market; this is a very good means with reasonable cost to have outstanding pilots flying our aircraft.”

Fabi Riesen predicted then that virtual reality technology “is going to change the simulator market”.

I’m inclined to agree, as is EASA and a plethora of savvy pilots. Certainly William Tell (who may only have been virtual himself) altered the course of Swiss history with his convention-defying independent spirit and a very precise flight trajectory. Perhaps VRM is destined to become a Swiss, and aviation training, legend as well.