Mixed Reality to the Rescue

The complete blended HEMS training system, FFS plus MR goggle for the rescue-man role, commenced training at NCCH. Chris Long describes the unique development.

As new technologies reach the market, the innovators embrace them and frequently adapt them to other tasks. A powerful example of this is the training for HEMS operations in Norway, which uses an innovative FFS at the Norwegian Competence Centre Helicopter (NCCH) located at Stavanger airport, Norway.

After a major helicopter accident in 2014, the CAA Norway (Luftfartstilsynet) sought improved training for HEMS (helicopter emergency medical service) crews. Erlend Segtnan, Manager Crew Training at the NLA Norsk Luftambulanse AS, (Norwegian Air Ambulance) and Project Leader, took on the challenge, and with a free hand stepped back from the then-current options and started from the baseline of what was, in his considerable military and HEMS experience, really needed. The conclusion was that a purpose-built FFS was required, which also incorporated a simulated side door for the crewman – in this case the doctor, to control the cable and positioning at the end of that cable of the person being rescued. The desired solution was some form of virtual reality tool to simulate that side-door team member and facilitate training in the essential teamwork so critical to rescue tasks.

The Level D FFS replicates an Airbus H145, one of the mainstays of the Norwegian Air Ambulance fleet. Image credit: NCCH.

The first step was to acquire a Level D full-flight simulator of an Airbus H145 helicopter, popular with HEMS crews because of its twin-engine performance and one of the mainstays of the NLA fleets. Segtnan turned to Reiser Simulation and Training, based in Germany’s Bavaria region, hitherto an important Tier 1 supplier of a wide range of engineering and avionics solutions, but not then known as an integrator of entire systems for an FFS.

Dr. Michael Mayrhofer, strategic advisor of Reiser and Managing Director of NCCH, told CAT that the device was manufactured using a reverse-engineering approach. A major part of that was a comprehensive test-flight campaign on a heavily instrumented H145 to collect all required data for the realistic flight model as well as establishing a benchmark database to check conformity.

The guiding principle was to use COTS processes where possible. Moog provided the motion and vibration platforms, the vibration platform to replicate the inevitable constant vibration felt in a helicopter. It was recognised that image generation and projection systems are areas of technology which are constantly and rapidly developing, so only the latest devices, a Norxe P1 (8 channel IG + IR Projection) / Quantum3D, would do. A large dome was needed to house this – giving a unique shape to this FFS.

As with the aircraft itself, there is an input from a camera mounted under the tail rotor which shows a downward view of both the immediate forward perspective and the image vertically beneath the aircraft. This view is displayed to the pilots and to the instructor; the position of the “rescue man” can be seen and that becomes even more important when, during a normal transit, he trails behind the aircraft. Similarly, the pendulum effect when the helicopter moves away too quickly from a vertical cliff face can be seen as he gets swung toward the cliff. Entanglement in trees and other hazards can also be demonstrated so that they can avoid hauling the rescue man thorough the trees (it has been known!).

The formal certification of the FFS was initially granted in August 2018. Because VR systems were rapidly evolving as well, the search for the latest and most effective technology was deemed to be the Meta 2 MR goggle, which is lightweight, simple to operate, and provides all the functionality needed. This MR system now uses an avatar as the rescue man; this can be controlled by the instructor to make the bold arm gestures which are the primary signalling function for the positioning input from the avatar when he is on a cable which can be up to 60 metres (196 feet) in length.

Essential Teamwork

In the HEMS role, it is immediately obvious that coordination of the crews’ inputs and teamwork are absolutely essential, not only to complete what are sometimes seriously hazardous missions, but to ensure that safety is not compromised during those events. Even though there is careful selection of experienced crew members, the specific techniques and procedures of the NLA have to be honed. This is particularly relevant when the three crew members who are on any given shift of the 24/7 call-out responses may not have worked together. Before this device became operational most of the training had to be live in the aircraft. Conducting winching operations over rough water and close to cliff faces is not the safest of environments, so there is always a significant element of risk – which is why the standard of initial recruits is so high.

Now with the capability of accurately simulating the information flow between all crew members and in all weather and terrain situations, there is a safe route to thorough training, even in the more extreme situations which are frequently encountered in this challenging role. Practice at monitoring the load on the cable, to ensure early identification of either pendulous activity or rotation of the load, can correct that threat.

Most teams in aviation have to work closely together and at least understand the roles of the other team members. CRM is the pathway to that working relationship, but here it is on a completely different level.

In addition, familiarisation with the low-level IFR routing and procedures available in Norway, and with day/night operations facilitated by NVGs, there is a lot to master. Complex navigation techniques (RNP APP LNAV, RNP APP LNAV/VNAV and RNP APP LPV) can all be practised to perfect the skill. The detailed landscape database enables RNP 0.3 flights down to 1200 feet in the many valleys, and IFR approaches to 60 hospitals (the latest mapping has eight laser points per square metre measured across the whole of Norway!). That level of accuracy allows area familiarisation flights and practice at all the heliports from the safety of the FFS.

Crew-Crew & Crew-Ground Coordination

The 75 crews of the NLA operate 21 aircraft and are dispersed round 12 bases in Norway and four in Denmark. The pilots have a minimum of 2000 hours PIC (rotary), IR and preferably experience with NVGs.

As is usually the case with HEMS teams, there is very close liaison and dependence on the ground-based first responders, in this instance volunteers of the Red Cross and the 10 Norwegian Alpine Rescue Groups based around the country.

So – what do the crews think of it? Having started the process relatively recently not everyone has used this process yet, but there is a trend now showing. The opportunity of working with a full-crew complement of the three crew members – pilot/doctor/avatar rescue man is very much appreciated.

Unsurprisingly there are a few who become slightly uncomfortable with the use of VR (that reflects the trend in the general population) but being able to train in a demanding situation without the threat to life and limb is hugely productive. Training is about being able to make mistakes and learning from them. With this platform that can be confidently achieved without risk – much appreciated!

Reiser Simulation and Training’s mixed-reality training for HEMS crews was selected as “ Outstanding VR/AR/MR Application” in Military Simulation & Training (MS&T) magazine’s 2019 industry awards – presented by Halldale Group in December at the I/ITSEC conference.

Published in CAT issue 2/2020