Your Next Flight Simulator Platform Should be a Game Engine. Seriously.

Walt Yates suggests recent technology developments make a game engine a viable option as a platform for simulation-based training in aviation.

Effective aviation training focuses on improving situational awareness and the decision-making process. It could be argued that no domain of practice has a greater need for visual fidelity and physical accuracy in simulation-based training.

Markedly improved visual fidelity, geographic precision, and performance on affordable hardware is currently available at unprecedented levels for civil and military aviation application developers and users.

Permit me to offer a guide to the types of features that developers of aviation simulations should look for before adopting a game engine as a platform, as there are products available today that meet, and even exceed, all these requirements. In addition, the most popular game engines are currently offered for very modest prices, sometimes even for free.

Flexibility

Perhaps the most important advantage of using a game engine for development of simulation-based training is flexibility.

Off-the-shelf game engines, which have been used to create various kinds of games and experiences, provide a depth of flexibility unseen in traditional simulation environments. This includes the flexibility to access vastly expanded ecosystems of 3D assets; flexibility in computer platforms, hardware, and display options; and flexibility in the size, shape, and location of the deployed solution.

Flexible Asset Sourcing

A game engine, by nature, accepts 3D assets in a variety of formats, and also provides a built-in, highly accessible authoring system for designing scenarios. Custom assets can be built by in-house designers with off-the-shelf digital content creation (DCC) tools, while less specialized assets – terrain, vegetation, civilian vehicles, and so on – are readily available from numerous third-party sources.

A game engine’s authoring system provides a great deal of flexibility in the way training can be designed and deployed. By nature, a game engine includes tools for developing a user interface, meaning that the training solution itself can be customizable. For example, a training module can include an option to customize a scenario with a particular combination of weather, terrain, vegetation, and runway length, or to emulate a specific geographic location.

If you have an existing pipeline for simulation development, a game engine can, in most cases, replace or augment one or more steps without disrupting the rest of your workflow. If your existing pipeline already includes a source for 3D assets, those same assets can be ported to the game engine via one of many file formats designed for portability.

Flexible Hardware and Display

Traditionally, simulators have been of bespoke design, which limits the upgrade path. With commercially available XR equipment (Oculus, Vive, HoloLens, etc.) and other readily available hardware, users have a lot more options for developing new training systems and improving existing solutions.

Some modern game engines can output to many types of display devices, from enormous domes and LED screens to VR headsets, AR glasses, and even mobile devices. Advantages of such a wide range of broadcast options include automatic asset quality-scaling based on the hardware’s features, and the ability to adopt a “create once, deploy many” strategy for a number of possible training solutions on different media.

The game engine you choose should also support a wide variety of display hardware so that you may choose the most effective medium to develop and deliver your training without the constraint of limited hardware compatibility. The rise of OpenXR is bringing a level of standardization between the devices, removing the burden to adapt to any single set of hardware drivers and interfaces while applications are being developed.

Likewise, game engines are designed to run smoothly on a wide range of hardware and OS configurations, taking advantage of the most powerful and modern systems while still performing adequately on cheaper or aging hardware. If you have access to VR goggles and a PC, chances are you can already create a modest VR-based training simulation with your existing hardware.

Flexibility in Space

Leveraging the power of a game engine to drive head-mounted displays for aviation simulators provides a significant conservation of floorspace and reduces the cost of facilities needed to host pilot training simulators. The objective is not to replace large projection dome simulators with head-mounted VR systems, but to increase the availability of VR training so that students naturally gain a higher level of skill mastery by the time they reach the point of using one of a smaller number of projection dome full-cockpit simulators. A space that could only accommodate one projection dome simulator may be able to host half a dozen part-task trainers using VR.

One such VR system for helicopter pilot training developed by VRM Switzerland has been qualified by the European Union Aviation Safety Agency (EASA).

Open Source

An open and extensible architecture begins with freely available source code. An open-source platform vastly simplifies the challenges of verification, validation, and accreditation (VV&A) for mission-critical applications.

From the perspective of cybersecurity, the availability of source code means that automated source-code review can be performed at any interval. Such a review, a necessity for aviation simulation both in civilian and military domains, ensures the integrity of applications while maintaining the smallest possible attack surface.

Accuracy

Until recently, game engines have existed in a space of compromise between speed and precision. Historically, game engines have used limited precision in the calculation of geographic coordinates, which do not provide the necessary precision for navigational vectors over large distances of hundreds or thousands of miles. This limitation has had important consequences for aviation training simulation, and has, in the past, essentially relegated many game engines to very limited utility for pilot training.

Advances in game engines means they now may incorporate better precision coordinates natively, simplifying development and removing the burden of workarounds. Such accuracy, and the use of a flexible geo-coordinate system, are essential to effective simulation-based training.

Game engine-based deployability was recently showcased in Epic Games’ Project Anywhere, a cloud-based demo that gave vIITSEC 2020 attendees the opportunity to explore high-resolution 3D terrain and building data in real time from their computers, tablets and smartphones. Project Anywhere leveraged the GPU power from NVIDIA through a Microsoft Azure infrastructure and the Cesium for Unreal plugin.

In addition to accuracy over long virtual distances, the use of validated physics models for aerodynamic performance is also critical to ensuring that the lessons imparted to trainees are truly representative of how their aircraft will perform in the real world. Ideally, your game engine of choice features transparency at the algorithmic level, so accuracy can be assessed and refined whenever necessary.

The modular nature of game engines makes a “BYO physics model” approach possible, where you can simply swap in your project’s own existing physics engine or model. Your model may control not only simple object interaction, but also advanced weather and atmospheric modeling, and even more esoteric physical phenomena such as electromagnetic interference from weather situations.

Realism and Visual Fidelity

Though often overlooked in the discussion of fidelity-to-task of round-earth coordinate systems versus flat-earth systems, representation of specular effects from the solar ephemeris is an important part of realism for trainees. Attenuation and diffusion of light through clouds, smoke, rain, and snow should be accurately represented.

The development of realistic environments and scenarios for VR and AR pilot training, whether through physics, 3D assets, or artificial intelligence, has been recognized as one of the top trends for aviation training going into the next decade.

The engine should be able to import and render cinematic-quality 3D assets and textures without loss of performance and provide accurate rendering of the subtlest visual cues onto display systems from curved LED monitors and projection screens to head-mounted XR. As aviation training systems continue to embrace the use of head-mounted XR devices, those systems will require fully native support from the image generator.

Affordability, Availability, Agility

Historically, the first cost of developing an aviation training simulator is to license the simulation platform on which the application is developed. Before any user-specific content is developed, the licensing of a simulation platform may consume a significant portion of the software development budget. Conversely, the most popular game engine, Unreal Engine, is available for free.

In addition, the selection of the simulation platform can constrain the customer to rely on the platform provider for asset and content development. With a game engine, learning modules are readily available on demand through videos and documentation, and an active community of millions of experienced developers regularly communicate with one another to share knowledge. The millions of user-developers spanning the globe, and from a large variety of backgrounds, creates a competitive market for contracting custom development of applications without the bottleneck that can occur with proprietary simulation platforms.

The use of a game engine also enables smaller or less-specialized teams to produce useful,

working simulation-based training. Individual developers across the globe are already experimenting, and are showing the types of simulations they can achieve from scratch using just home-office hardware, projects that come with their own tools for producing flight simulations, and accurate, effective simulations that take just an hour to develop.

Thus, not only can the use of a game engine as a platform increase development speed and decrease costs, it also frees teams to experiment and produce working demos quickly for peer review. Lowering the costs associated with exploratory development means that new products can be brought to market with reduced investment, while at the same time greater risks can be taken in finding those new products.

A recent example of such a defense aviation solution is Talon Simulations’ Strike full-motion simulator, which has the flexibility to integrate artificial intelligence and data analytics. The system takes advantage of VR’s small footprint to lower the size and cost constraints of a dome experience.

These smaller, more agile systems are ideal for interim training, to keep pilots’ skills fresh between visits to the dome. For example, HTX Labs helps US Air Force pilots practice responding to cockpit emergencies such as a sudden fire or hydraulic malfunction. With this system, pilots are able to practice responding to emergencies over and over until they can perform the appropriate steps quickly and confidently, all from the safety of a VR simulation.

Image credit: HTX Labs.

Training-from-Home Ready

The impact of the Covid-19 pandemic on simulator-based pilot training highlights the flexibility of using a game engine for development in rapidly changing or unpredictable circumstances. While there may have been modest interest in ultra-lightweight, “personal” flight training systems before the pandemic, such systems are now very desirable for safety reasons.

The ability to rapidly modify existing training systems means that the hardware and software configuration for small-scale use can be orders of magnitude simpler to create and deploy. In an uncertain world, a platform that facilitates rapid response to new challenges is more than a cost-cutting and time-saving convenience – it can facilitate the continued training of pilots while minimizing health risks.

Unreal Engine

The author of this paper is, in particular, a supporter of game engine Unreal Engine as a platform for simulation-based aviation training, having experienced all these benefits first-hand. However, other game engines exist which can and should be considered as a means to make the development of aviation training faster, less expensive, more accessible, and more flexible.

About the Author

Walt Yates served 27 years in the US Marine Corps at the rank of Colonel with specialties in field artillery and acquisition management and modeling and simulation. He currently works for Epic Games, the creators of Unreal Engine, as a Simulation Solutions Advisor.