Leveraging Game Tech for MilSim Software

There has recently been a significant push from the gaming world into military simulation with the rise of game engines such as Unity and Unreal into the training domain. MS&T Special Correspondent Andy Fawkes reports.

Leveraging the significant growth and investment of the gaming ecosystem, game engines are reducing the simulation barriers to entry for a large number of training use cases, particularly for XR applications. They are also now supported by dedicated support services for their military customers. MS&T spoke to simulation and training industry leaders for their views on the development and usage of dedicated military simulation software and the benefits for their customers.

From Custom-Made to Customisation

“For me, this trend is just a logical progression,” said Phil Perey of the influence of gaming and other information technologies on training and simulation. Perey is CAE’s Head of Technology for Defence & Security, and has 35 years in the simulation and training industry in various engineering capacities.

CAE was one of the first companies to replace bespoke graphics cards with gaming cards, or GPUs, with 16 gaming GPUs used for one channel, where now two gaming cards can be used for a Level D simulator at a higher resolution. CAE continue to track gaming GPU performance: “We're following that progression and every time we can consolidate, we take that step and say okay, what is the sweet spot between performance and the rendering capacity,” Perey said.

For the next iteration of technology, Perey believes the “eventual adoption towards gaming engines is the next logical step. We have been exploring this capability now for a number of years, trying to understand what are the capacities of gaming tech and how does gaming tech apply now”. Perey continued, “There's always things you need to do on top of whatever the technology direction is going, so it's really how best to do that”.

Brookline, Massachusetts-based MetaVR has also looked to exploit the broader IT and gaming worlds from the launch of its first main product, Virtual Reality Scene Generator (VRSG), in 1996. MetaVR co-founders Richard Rybacki and W. Garth Smith first worked together at a DoD consulting firm in the mid-1990s on a simulation intranet research project. Witnessing the growing market for PC-based 3D gaming, they saw potential value in developing a real-time texture-mapping visualisation/simulation application for Windows PCs, enabling users to visualise large and detailed geospecific terrains at real-time interactive frame rates on a commercially available PC.

Creating and maintaining 3D content is often included in the licences for military simulation software. Image credit: MetaVR.

VRSG is a Microsoft DirectX-based render engine, and early adopters of MetaVR's technology included the US Air Force and US Army for training fixed-wing pilots and UAV operators. It is those kinds of training requirements that have driven the development of VRSG and set it apart from game engines.

Smith told MS&T: “As VRSG is designed to fulfil training scenarios controlled by the user, it has to be extremely robust. In a game-style trainer, the game engine developer has a certain level of control over the ‘level’ layout, and the player’s camera location, hence can guarantee maximum frame rate. We cannot take these steps, as we don’t anticipate/control the camera path or user perspective. Instead, we produce VRSG in a way that it can support almost limitless use, meaning it must be extremely robust as our customers can (and do) test its limits ruthlessly”.

Smith continued, “It’s very far horizon, round-earth terrain format contributes to its ability to maintain fidelity both on the ground and in the air – making it extremely valuable for joint/close air support type training missions.”

Multi-national Bohemia Interactive Simulations (BISim) has taken a somewhat different path. Formed 20 years ago to exploit the then hugely successful PC-based modern combat simulation game Operation Flashpoint, BISim modified games software for military training use. Since the first release of its VBS (born Virtual BattleSpace), the software has gone through multiple development cycles and although it still draws heavily on game technology, it is not now a modified game as was VBS.

BISim’s latest flagship product is VBS4, a whole-earth virtual and constructive simulation and simulation host released last August. Development of some of its core whole-earth technology commenced in 2013 and early prototypes were released in 2017 under contract to the US Army.

Also drawing on the early 2013 work is VBS Blue IG. First released in 2018 as image generator software, it provides a baseline global terrain that is procedurally enhanced from real-world data which can be blended or replaced by users’ satellite/high-resolution terrain data.

BISim’s customer base is now diverse with products used in over 60 countries and by over 300 integrators, including the US Army, US Marine Corps, Australian Defence Force, Swedish Armed Forces, French MoD and UK MoD.

Tailoring for Training Long Term

MetaVR’s VRSG has training-specific features that go beyond visualisation and enable its military customers to customise their simulators to meet specific training purposes. As an example, Smith told us, “Our Scenario Editor allows users to create custom environments as well as scripted vehicle motions; our Network API supports DIS and CIGI, but it also adds a layer of even more focused customisation to support fine-detailed simulation (e.g. controlling helicopter rotor speed); and we also have an extensible effects library”. VRSG maintenance licence customers get access to all MetaVR content, new and legacy, including continuously updated 3D models and terrain.

Maintaining and updating content is also a key theme for CAE. Perey told MS&T, “I think we have to recognise the heterogeneous nature of the devices that are out there now and that will always be the case. So, it's not just about the real-time side of things, it's about the content creation, and what we need to do is embrace the portions in the gaming industry that allow large-scale automated content creation through use of AI and cloud computing.”

Using the example of an enemy city of three million buildings: “I don't want to have designers clicking on every building footprint in creating models for it; I want to use AI in the same way the gaming industry uses it for large-scale content creation.” CAE’s Immersive Group is “exploring and developing such capabilities,” Perey told us.

BISim have developed their software in close cooperation with military users. Group CEO Arthur Alexion said, “It would take a 200-person team over seven years to build another VBS4, but it’s not just the amount of time and money invested in development, it’s the fact that development has been done with subject matter experts”. This development has resulted in an 18,000 3D-model library, 200 mil-sim specific use cases, and technology to serve, store and enhance the baseline global terrain.

More than “game engine” visualisation, military simulation software often includes capabilities such as training tools, scenarios, AI and 3D terrain, and military content. Image credit: BISim.

Alexion continued: “If you licence a game engine, you need to build out the rules of interactions in your ‘virtual world’ (what we call the Simulation Engine) and you need to build out all the assets that you want (e.g. vehicles, avatars, sounds, animations). With VBS4 you get an already built-out virtual world and a library of assets all pre-configured to work with the Simulation Engine. The overarching point here is VBS4 is much, much more than a game engine.”

Another key advantage BISim claim is that, as customers fund VBS4 enhancements, BISim manages the merging of the new capability into new software versions together with backward compatibility.

New Classes of Training Devices

As training requirements and technology change, military simulation software companies cannot keep still. MetaVR’s Smith told MS&T, “We are constantly working to adapt to meet emerging customer requirements and take advantage of the newest visualisation technologies. We see the market shifting toward mixed-reality more and more, and to this end we released the latest version of VRSG (version 6.5) in February 2021.”

VRSG now supports Varjo VR and MR headsets, the Valve Index VR system, and the HP Reverb VR system. It also includes support to foveated rendering which with eye-tracking allows the hardware to render the scene at full fidelity in the area that the HMD wearer is looking.

This trend toward support to XR is also echoed in BISim’s plans. The next release of VBS4 (21.1) is planned for later this summer and will include UI extensions and other features for use in a VR environment. Other customer-driven enhancements will be VBS4’s ability to handle more entities and improved physics for large-scale simulation. Alexion told MS&T, “We will continue to see a simplification in content development pipelines. Our aim is to reduce the need for GIS experts to produce new terrains and models in the training environment. Similarly, we see a continued drive toward removing the barriers to entry between trainees and military staff and the training system.”

An example, “VBS Plan, VBS4’s mission planning mode, allows command staff to create new complex scenarios without requiring specialist training. In the next VBS4 version, VBS Plan will enable distributed complex collaborative mission planning in VR.”

Another trend of importance to Alexion is the cloud. “As cloud technologies continue to advance and governments become more comfortable with operating in the cloud, we expect to see greater opportunity for our product to utilise cloud tech, especially where there’s a requirement for large-scale simulation.”

CAE’s Perey sees advances in technology, such as in XR, leading to new classes of training devices and ways of training and education. He cited “immersive training devices”, or ITDs, a term that originated in the USAF, and explained, “I like this notion of a new type of device because that is so important in our customers’ vision. A lower-end device but a compliment to higher-end devices.” Perey continued: “I've adopted the term internally and with customers because I think there really is a need. The traditional full-flight simulator with a motion system is a fantastic device; it's about as immersive as it gets. But you've serialised the students through it because you only get two students in there at a time. How do I parallelise that?”

Referring to Pilot Training Next’s (PTN) requirement to shorten the training time, “I can't have many more instructors, so that pushes you into more self-paced training delivery.”

ITDs have the potential to support such ways of training through practicing proficiency skills in “in a very different way,” Perey said.

A Mixed Market?

Militaries will be exploiting both game engines and military simulation software. Essentially a mixed market of software. Asked what militaries at an enterprise level should do to better support simulation software, Smith told MS&T, “Several commercial standard formats already exist which are well suited for 3D terrain construction for real-time visualisation.”

What is needed “is a meta terrain description such that one standardises on how the source data is projected, sampled, triangulated, decimated, and on a Level of Detail (LOD) scheme such that the final terrain configuration is deterministic and then would correlate regardless of the image generation software.”

Bohemia’s Alexion recognises the mixed market as different technologies suitable to different applications. For S&T assets, he said, “It’s important for customers to understand what they have and manage those well. They should keep reuse of these assets in mind, but not constrain functionality and training effectiveness by handbraking technology advances and innovation. There isn’t a one-size fits all technology standard, in our view.”

Good Times, Same Challenges?

There is not much doubt that the $175bn gaming ecosystem is supporting many technological advances and this looks set to continue for the foreseeable future. However, whatever the simulation software technology, game engine based or specialised military, there are enduring issues in the procurement of training systems and services: technical support, security of supply, legacy systems, changing operational requirements, and maintaining the availability of instructors.

MetaVR’s Smith told MS&T that training systems need to be robust, proven and supported and “we implore the simulation community decision makers to look beyond highly optimised demonstrations and consider how the software will work in fielded systems in the hands of the warfighter.”