XR ROI for Pilot Training Shows Early Promise

The collaborative US DoD-industry webinar, “How the USAF trains pilots of the future with XR,” provided wide-ranging insights on emerging enabling technologies for military pilot training.

The cost of training an Air Force pilot remains an expensive proposition – providing ‘a target’ for increased efficiencies when and where possible. In terms of 2018 dollars, the estimated total expense (in millions of dollars) to obtain a basic, qualified pilot:

Competing challenges are encouraging the US Air Force to add mission-ready pilots at reduced costs and faster. Indeed, Air Force Major Alex Horn, a program manager at Defense Innovation Unit (DIU), noted the service’s pilot recruitment and accession has been a “pretty critical problem that has emerged in the past decade or so.”

Horn, who is also a liaison to Air Education Training Command, provided one vital insight on the disparity between fast-jet and air transport training costs. “The simulator systems for the C-17 were so advanced that we complete almost all of our initial qualification in the simulator and we only get three flights before we’re qualified as a C-17 co-pilot. For single-seat aircraft – fighter and bomber – that’s not really possible, but you can see the dramatic difference in the cost of training when you can put most of the training in a virtual world.”

Beyond front-end training costs is the mission readiness challenge. The service has a pilot shortfall of about 2,000-3,000 pilots and is also experiencing turbulence in its training aircraft fleet. In two instances, the venerable T-1 is being divested and  the next-generation T-7A program is ever so slowly righting itself.

Of further significance, the program manager noted his office hopes to drive the lighter weight, less expensive, yet capable simulation devices and accompanying technologies undergoing evaluation in service programs into different aircraft programs “to make it a lot cheaper to train all of these pilots.”

DIU’s Mantra: Go Innovate

AETC was the service’s focal point in 2018 to start exploring the art-of-the-possible in terms of harnessing commercial off-the-shelf technologies for pilot training. After launching Pilot Training Next that year as an initial project to achieve training innovation for accession pilots, the service elevated its efforts in 2020 to the Pilot Training Transformation program. Horn labeled PTT as one of DIU’s “exemplar projects, where the service has been able to field some really exciting technology and evolve it to keep pace with advancements in this technology domain, and do so better, faster and cheaper.” Three representative, high-level innovations emerging from these projects include:

Of interest, the US Navy followed in the wake of the Air Force and launched similar efforts that have included Project Corsair – Naval Aviation Training Next’s prototype flight training syllabus for advanced tactical jet pipeline students to gain efficiencies and cost reductions in its undergraduate training programs.

Fast forward to 2023, when DIU, the Air Force and their industry partners continue to raise the bar in gaining efficiencies and reducing the cost of early pilot training phases – with a heavy emphasis on advanced learning technologies. In the latest case, Vertex Solutions was selected again by DIU for an advanced pilot training program: a Virtual TRaining for Air Dominance (VTRAD) mixed reality (MR) simulator to support the US Air Force’s new Fighter/Bomber Fundamentals tactical jet training course, training the next generation of fast-jet pilots. This project is envisioned to transform the common immersive training devices (cITDs) Vertex delivered under the PTT program into a MR simulator with integrated hardware avionics and a new advanced T-38C aircraft model. Vertex will produce an upgrade kit to modify existing cITDs into the VTRAD configuration providing higher-fidelity training at lower cost. “Fighter and bomber pilot communities can now interact with critical hardware avionics components integrated into Vertex Solutions’ proven cITD platform currently in use at all undergraduate pilot training locations,” a company statement read.

A VertexSolutions-supplied video of its Multi-place Mixed Reality Flight Simulator Prototype may be viewed below.

Challenges, Opportunities

Other current and emerging opportunities exist to help advance and accelerate USAF’s efforts to transform its pilot training – across many technology sectors.

As the government-industry team advances into MR-based devices to support individual and team training, Horn said that instead of just focusing on the hardware solution and making sure we have the immersive technology right, “we are also working to prototype a new synthetic environment to be able to support the high-fidelity modeling graphics that would be required to do fast-jet training.” Sweden-based Metrea Simulations, two months into a DIU contract to advance this capability, has been doing so in an Unreal Engine 5 environment, “but heavily modified to support this advanced training.” The DIU program manager emphasized there is the opportunity “for a lot of third-party developers to come in and help build a huge ecosystem to really scale up the capability with AI agents, and other aircraft developers modeling content, etc.”

In a second technology thrust to add more rigor to undergraduate training scenarios, Vertex Solutions’ TJ Moser, a retired career Air Force pilot, added “the next thing is working these devices into a secure environment.” While Varjo’s hardware- and software-compliant HMDs offer one entrée into higher-level security environments, there is also the challenge of operating on a secure cloud computing environment. DIU’s Horn explained that it was “intentional that the services started with undergraduate pilot training in an unclassified environment, so we could capitalize on all the flexibility of the COTS components and not have to worry too much where we are sourcing some of the content. We are now looking toward developing those ‘air gap solutions.’”

One concurrent project to permit training in a higher-level security setting is DIU’s efforts to seek partners through NATO and, as an adjunct, to develop an F-16 training version. The program manager continued, “We look to begin on some of those accreditation and compliance issues for the hardware and software prototypes to support classified training in those places – soon.”

Another interesting convergence in next-generation pilot training is in the broad area of visuals. DIU’s Horn noted that his organization “would like to… run the simulation environment on a high-end PC like an NVIDIA 4090, or similar product for these new devices – getting away from the server rack and the costs associated with procuring and maintaining that. So the devices of the future would be able to run on a PC, or several PCs – one per HMD – or some similar configuration.” Recalling that a recent DIU solicitation said “utilize commercial game engines to the maximum extent possible,” Horn added, “We’re not discarding the idea of traditional image generators” – yet. John Burwell, Global Simulation Manager and Lead at Varjo, and webinar moderator, noted that in his site visits he still sees “lots of traditional image generators in use. Aechelon’s pC-NOVA is supporting a number of Navy programs. Quantum3D is around supporting a variety of different projects.” Of importance, he pointed out most of the IGs have evolved to support virtual reality and mixed reality solutions.

One instance of the advancement in USAF pilot training enablers is the migration of lightweight, less-expensive but capable devices, from the common immersive training devices to multi-place mixed reality devices permitting individual and aircrew training.

Source: VertexSolutions

Promising Early ROI Results

The DoD-industry team has gained early, impressive returns on investment to further encourage the continued funding of these programs. These and other ROIs should provide compelling reasons to invest in XR and related technologies in commercial aviation training and adjacent high-risk sectors.

DIU’s Horn noted in initial testing the undergraduate pilot projects have been “able to reduce aspects of the early syllabus – getting to your initial solo from one-third to two-thirds faster. And just by accelerating the early pipeline, you’re able to achieve hundreds of millions of dollars in annual savings. Just seeing this as an augmenting capability, to really enhance the access to virtual training and simulators, is hands down an overall win for the pilot training community.”

Vertex Solutions’ Moser presented two other outcomes of migrating early instructional content to technology-enabled learning. Noting that when tests were done, “in 14 of 14 categories, T-6 students outperformed their non-immersive peers,” and added, “The Air Force is also essentially awarding pilot training wings 4-6 months earlier. While there is still training they have to do ‘to declare,’ they are still certifying them as able to fly in the National Air Space as pilots.”

A huge takeaway from the discussion on ROIs is that after reviewing the outcomes of PTN, PTT and other programs, there should be no argument that DoD can use these and other augmenting strategies to obtain efficiencies early in the training pipeline – in terms of training better, faster, when able, and at lower cost. The challenge will be to expand similar training constructs further into a pilot’s training continuum.

While the FY2024 US DoD budget remains in deliberation, it’s important to note the USAF is investing in PTT – for FY2023 $15.3 million was appropriated for Pilot Training Transformation.

The July 26 webinar video may be viewed below.