Coming Soon: Smarter Motion and Eye-Tracking Systems

Motion-based training devices remain a mainstay of the commercial aviation training enterprise and are being adapted for the evolving eVTOL market. Concurrently, device providers are increasingly integrating head- and eye-tracking systems into their training devices.

While FTDs become increasingly capable and less expensive than full-flight simulators, don’t write off the ubiquitous FFSs – yet. Indeed, FFSs and their underpinning motion systems will have a firm place in the commercial aviation training enterprise well into the future. In one instance, training organizations and airline operators continue to offload UPRT and associated challenging training tasks from supported aircraft into motion devices. At the same time, motion base suppliers and training device OEMs are improving the life cycle of motion devices – integrating head- and eye-tracking and other technology enablers to add rigor to FFS scenarios, enhancing the life-span and operating characteristics of motion bases.

Insights into trends and developments in motion and tracking systems were gained from four perspectives – at the motion supplier level from Ton Stam, E2M Technologies, the full-flight simulator OEM level from Dr. Christian Theuermann, AXIS Flight Training Systems and Till Borngräber, F.A.S.T. - Group GmbH, and head/eye-tracking specialists Smart Eye - Soli Shahmehr and Björn Lindahl.

“Quite a Large Backlog”

Ton Stam, Sales Director at E2M Technologies, is seeing subtle changes in the motion systems segment of the commercial aviation training enterprise. While Stam said training tasks continue to be off-loaded from Level D full-flight simulators into lower-level flight training devices, he noted “in the end there will always be a place for full-flight simulators – the reason being there will always be a part of the pilot training continuum in which an FFS will be required.”

Looking at the overall market forces on this industry segment, Stam first noted there is “some hesitation and uncertainty” in the airline training community about ordering new FFSs. But the corporate executive then called attention to recent booking announcements by OEMs of FFSs and industry forecasts, and said the FFS market, and by extension the demand for motion systems, “looks healthy at this time.”

E2M is a familiar brand in the commercial aviation training market, with more than 150 motion systems fielded around the globe – primarily to training organizations. Further, Stam reported E2M “has quite a large backlog,” but called attention to another force at play – the challenges to get motion system parts. Noting the availability of system components changes almost weekly, he said that in one case, power cables which connect system actuators have extended lead times. While E2M adroitly seeks alternate and long-term suppliers, the global availability of cables remains uncertain, as electric vehicle makers and other industries compete for these and other motion system materiel. In another effort to improve product life-cycle performance, E2M is expanding teaming agreements with in-country partners to establish part depots, and deliver repair resources and other support in Asia and elsewhere on “very short notice.”

On the motion system community’s near-term horizon, Stam is expecting a new generation of Level D motion systems for the commercial aviation sector. “Our customers are driving this and it’s a total cost, life-cycle issue. They are looking for an ‘install and forget’ system.” Such a system would be expected to have lower energy consumption, be more “user friendly,” have decreased maintenance requirements and like attributes. The executive concluded, “E2M is on track with these new requirements.”

OEMs Weigh In

One OEM, AXIS, thinks the demand for FFSs will increase during the next years, as customers want to improve the training of their pilots. Dr. Christian Theuermann, COO/CFO at the company, pointed out core training events are only possible on a FFS, like UPRT, engine failures, hard landings, turbulence, realistic takeoff / landing training, etc. Of significance, “More than 95% of our customer requests are for a motion-based simulator.”

The corporate executive then focused on specific, emerging requirements helping to generate this demand for FFSs. Windshear was at the top of his list. Theuermann initially noted AXIS windshear training is based on FAA-published models and provides different intensities for different student levels (light, moderate, severe). “Significant motion cues are part of the wind shear simulation as well as a high-fidelity rendering of the aerodynamic model are essential to provide realistic training; these items are crucial to be trained on an FFS.”

The executive also emphasized the AXIS FFS provides an integrated training solution for all required UPRT tasks. “The system incorporates a wake vortex simulation to provide realistic pilot-in-the loop scenarios in different intensities,” he said, and continued, “The number of scenarios is more or less unlimited. The training providers can add as many scenarios in different settings as they need to fulfill the training tasks. Also, pilot out-of-the-loop maneuvers are part of the integrated solution and are tailored to the training needs.”

AXIS works with multiple core motion-system suppliers, including BOSCH/VHT, E2M and Moog. The executive noted, “Our modern and flexible software solution interface is able to integrate different motion systems in a very efficiency way.”

Theuermann then delved into other technology enablers that are helping AXIS FFSs stay ahead of customer requirements. Of importance, the AXIS Evidence-Based Training is based on an AI backend. The executive then noted eye tracking is already possible by combining specialized hardware, software and neuroscientific algorithms, to understand how people cope with increasing mental load, internal and external stressors and sudden changes in their environment. “As ‘visible parts of the brain,’ biologically, eyes are ideal for cognitive monitoring. Through eye tracking it is possible to analyze psychophysiological biofeedback to track changes in attention and perception and even more detailed psychometrics as cognitive load, fatigue measurement and prediction.”

Theuermann added this technology enables customers a better selection of staff and improvement of safety by assessing human factors data and the possibility of evidence-based training by integration of mental load data and insights into the individual processing architecture of pilots. AXIS sees applicability of eye tracking in its training devices through the continuum of pilot training, from cadet pilots up through experienced crews, as well as instructors and target groups outside aviation.

Based on the demand for new pilots around the globe and the aviation market’s return to pre-Covid operational levels, AXIS believes “the commercial aviation FFS market will grow during the next years.”

Getting a ‘Feel’ for eVTOLs

Till Borngräberm, Member of the Executive Board, F.A.S.T. - Group GmbH, provided a third perspective on the motion sector – addressing it from meeting the needs of the emerging eVTOL market.

First, the basics include F.A.S.T.’s flagship MR Series X Flight Simulator, “for which we use an electrically actuated 6 DoF motion base, supplied by VRX Sim. It offers our customers a small form factor, reliable operations and low latencies.   A video of the simulator can be viewed here

The industry executive also emphasized as part of his team’s Mixed Reality Concept, “eye tracking will be a crucial component of our simulators and provide instructors with a new way of interacting with their students. Additionally, we are working on software to further expand on the benefits of eye tracking and use the data in a new and unique way.”

Borngräberm explained motion changes the way pilots interact with their aircraft, especially in VMC [visual meteorological conditions] and close to the ground, both conditions encountered in an eVTOL. “But crucially, eVTOLs are a new generation of aircraft with unique flight dynamics and a new flight feel. Safety should always be at the forefront of eVTOL pilot training, and teaching pilots on how eVTOLs ‘feel’ is just as important as teaching them how to operate the systems. This is why we believe pilot training should always be taught on motion devices. Because of the unique flight characteristics, we are currently developing a new kind of motion device. It will accurately model certain flight dynamics, unique to eVTOLs.”

A partial list of F.A.S.T.’s industry / academic partners include: Elite Simulation Solutions; Technische Universität München; Rotor Consult; FTD Consulting and MOC Simulation Services.

The corporate executive’s roadmap for the next 12 months has many waypoints. A brief summary of activities include: eye tracking improvements; developing a new type of motions system; improved modularity: and further work with its partner OEMs in regards to flight and system modeling, interchangeability of components, and core training goals.

Tracking in Demand

AXIS and F.A.S.T.’s inclusion of head- and eye-tracking systems in their FFSs are the tip of the growing demand for this enabling technology across the commercial aviation market.

Smart Eye’s 3D Head and Eye Tracking products are available for integration into training devices to enable students and their instructors opportunities to better analyze trainee performance during scenarios. Soli Shahmehr, Vice President and Head of Research Instruments at the company, offered the use case of a landing scenario during a simulator session, During the task, Smart Eye enables the instructor to see in real time where the student is scanning – with head movement and eye gaze measured and tracked in 3D (X, Y and Z axes) – while they performed the task. The executive added the major goal “is to help the students optimize their performance during a specific task.”

Smart Eye’s entry into the commercial aviation sector provides another instance of “cross-fertilization” in safety-critical training enterprises, where in this instance, the Swedish-based company is bringing to bear competencies developed in the automotive, aviation and other industries. Shahmehr pointed out, “Smart Eye has more than 20 years of experience in the automotive industry. Today there about 2 million cars on the road equipped with our software. We have more than 200 car models that will come into the market in the near future.”

In the commercial aviation industry, Björn Lindahl, Vice President of Product and Innovation at Smart Eye, called attention to their products having been used in research and development activities, with pilot training being a new business pursuit. “We’re educating the market and, at the same time, we’re learning from the market – especially what it means to work in this part of the aviation industry.” One valuable lesson learned from its emerging work with training enterprises has been the value of its tools during the student’s post-simulator review session. The executive noted Smart Eye is in discussion with a number of airlines and training organizations about its products, even beyond training, Lindahl noted interest by airlines and aircraft OEMs to use Smart Eye solutions on the flight deck during operations.

In terms of other product hardware/software underpinnings, Smart Eye offers a scalable solution, where 1-8 cameras can support wide-ranging horizontal and vertical fields-of-view, up to a maximum of 360 degrees (horizontal).

Smart Eye is strengthening its product line through the integration of AI into its tracking algorithms and through the 2021 acquisition of emotion-detection software startup Affectiva. Lindahl noted the ability to track a student pilot’s emotion is completed by monitoring facial expressions, most significantly when the trainee is “startled or surprised.” Smart Eye develops AI product enablers with its in-house staff.