Virtual Maintenance Training

Military programs and other activities using virtual and adjacent technologies for maintenance training are in their formative stages. The military-industry team is accelerating that pace, writes Group Editor Marty Kauchak.

Virtual reality (VR) is the military-industry team’s stepping-off point to efficiently and effectively engage service men and women, and their industry counterparts, in an immersive learning environment. The most significant activity supported by virtual training is occurring in the maintenance mission. As VR-enabled courses begin to populate maintenance curricula, there are “growing pains,” with shortfalls emerging in training devices, course design and other instructional foundations. Training program managers are correcting these challenges to bolster their current course offerings and take the technology baseline to the higher plateaus of augmented reality (AR)- and mixed reality (MR)-supported instruction. MR is typically conceived as a combination of VR and AR.

Virtual Training as Reality

Virtual maintenance training (VMT) is a reality in a number of global military services and is an integral part of new weapons platforms’ life cycles.

In one instance, the US Navy’s Naval Air Warfare Center Training Systems Division (NAWCTSD) developed and fielded a Virtual Reality E-28 Arresting Gear Trainer, with a monitor-based version that is in use in the fleet and available through Navy E-Learning.

Kevin Mikalsen, the Director of Global Marketing at the DiSTI Corporation, noted another good example of his company’s virtual maintenance trainers based on the US Army Air National Guard UH-72 Lakota airframe. “Originally designed as a desktop classroom platform with both student and instructor stations and mobile tablet devices, we’ve continued our R&D development to include procedural vignettes for different training situations where augmented or virtual reality will enhance the student’s immersion or engagement with the lesson objectives.”

Beyond the US, the Chilean Air Force is developing its own in-house VMT production team using DiSTI’s VE Studio for their development platform. “They bought VE Studio, training and professional services from DiSTI to build their Bell 412 VMT. They are now quite proficient with the tools and capable of developing their own VMTs – to the point of building additional trainers and offering their services to other branches of the Chilean Armed Forces.”

The VMT sector is also advancing, in part, through “cross pollination” – taking best-of-breed solutions from the adjacent civil sector, as well as commercial technologies.

Boeing’s Pete Boeskov, Chief Technologist for Training & Professional Services, noted his company has extensive experience creating a variety of VR systems for pilots, maintenance personnel and crew. He said, “Using VR through a commercially available headset, students can train anytime, anywhere using Boeing’s system that guides them through the curriculum and provides personalized training and records progress.” Boeskov provided one snapshot of VR’s maturity, noting Boeing has currently deployed VR training on multiple contracts. However, “Although contracts have focused on the procedural aspects for VR training, they have been implemented for very diverse use cases.”

Delving into VR’s technology underpinnings, Boeskov identified important returns on investment, initially pointing out the current state-of-the-art in virtual reality is immersive, allowing the student to learn the layout of the aircraft, the individual components, and where all of the relevant elements are spatially without having to use any aircraft hardware. “Aspects such as ‘muscle memory’ can be trained in a way that computer-based training or its equivalent cannot achieve. Boeing is collaborating with its customers continuously to define the right VR applications and deploy this capability into the hands of our warfighters,” he added.

Boeing see evolutions in broad XR (extended reality) applications (enabled in one instance by HoloLens) across AR/Mixed Reality. Image credit: Boeing.

Another significant trend is that the technology baseline is advancing through collaboration – pairing platforms with other companies’ materiel. In one premier case, DiSTI’s Mikalsen pointed out the company’s development platform, VE Studio, is using APIs (application programming interfaces) so its customers “can choose from a number of options, including augmented reality devices such as Microsoft HoloLens and mixed reality VR headsets from Microsoft, HP’s tether-less VR-Ready backpack, Oculus Rift and HTC VIVE and, per the client’s request, add VR to any existing or future VMT as part of the deliverable and include it in the procedural lesson. “The user experience and supporting device is an output extension for us, much like a touch screen monitor, laptop or mobile device,” Mikalsen noted.

VR and other learning technologies have the attention of senior Pentagon officials. Secretary of the Navy Richard V. Spencer (in light blue shirt) is shown wearing VR headgear while reviewing a flight deck trainer developed at NAWCTSD. Image credit: US Navy

Challenges to Broader VR Use

VR is advancing incrementally and deliberately as a learning technology, across military services in the US and elsewhere. So, what is holding back VR from achieving wider acceptance and use?

One vital customer perspective was furnished by Commander Henry L. Phillips IV, Ph.D., a member of the US Navy’s Medical Service Corps. Assigned as military director for Research and Engineering (4.0) at NAWCTSD, the technology expert pointed out that improvements in COTS Virtual Reality hardware, commonly used in military systems, are never going to be driven by the military, since it makes up such a small proportion of the market compared to the gaming industry. Further, “Visual resolution in VR headgear is still far lower than with the naked eye, even with 4K visuals. However, this is arguably less of an issue for most aviation maintainer communities than for aviators, since generally, maintainers don’t need to be able to focus on extremely small, distant objects to train for their jobs.”

However, there are two general limitations of VR headsets that are definitely relevant to maintainers, “Current COTS VR products are designed for clear display at a single focal point, meaning that when users shift their eyes instead of moving their heads to track something, what they focus on will be blurry and distorted,” the senior officer pointed out. And “the biggest limitation in current VR gear, though, is the risk of ‘vergence-accommodation conflict’ in users after extended wear. When you’re wearing VR headgear, your eyes focus at a point on the screen, whether you’re looking at an image rendered as a near object or a far object. Your brain doesn’t like this. When users wear VR gear for extended periods, they can experience discomfort, fatigue, eyestrain, nausea, and visual problems after removing the gear. This is a problem that the industry is undoubtedly going to solve, but until it does, having users in VR headsets for hours at a time carries some risk.”

AR’s Emergence…

As significant, VR’s other evolving technology sibling, AR, is another learning enabler for next-generation weapons platforms, including the Bell V-280 Valor, one of two Joint Multi-Role Technology Demonstration demonstrators for the US DoD. The V-280 Valor has been flying since late last year, and is making rapid progress through flight test, including flights with Army test pilots.

As of this mid-July, Bell’s Michael Moody, the Manager for Integrated Product Support, Future Vertical Lift, and Dan Rowe, the Principal Logistics Engineer for Future Vertical Lift, told MS&T, Team Valor and its US government partners have worked together to achieve more than 115 hours of rotor turn time to date and nearly 40 hours of flight time with the V-280 aircraft. The V-280 has been able to execute demonstration flights showcasing both low-speed agility and cruise speeds in excess of current helicopter capabilities.

“The V-280 flights represent a maturation of many affordable design and manufacturing technologies incorporated in the aircraft. In parallel, Bell is focused on maturing revolutionary technologies that reduce the sustainment cost of the aircraft. These technologies capitalize on Bell’s application of the digital thread throughout the design, assembly, and maintenance of the V-280,” the two program executives added.

To that end, Bell is developing both VR and AR applications to reduce sustainment cost, ease the burden on the maintainer, and improve the accessibility of training.

Bell’s development of AR-enhanced maintenance and training is a result of the original equipment manufacturer exploiting the V-280 ‘Digital Thread’ and using software that allows for the manipulation and viewing of 3-D CAD data. The two Bell SMEs explained: “Digital Thread is a 3-D digital and data design method used to create the V-280 Valor. This allows Bell to deliver ‘manipulatable’ 3-D renderings of components to the maintainer on tablets at the aircraft to increase their efficiency and understanding of tasks on this revolutionary aircraft. We focus on making more relevant information available at point-of-use to execute a task while making it less burdensome than traditional work instructions. Developed in collaboration with the maintainers, this type of data is used daily in the performance of prescribed aircraft maintenance activities. Feedback from these journeyman professionals has been extremely positive in how these products have quickly enabled mastery of maintenance procedures.”

Additionally, taking advantage of the digital thread architecture enables Bell to rapidly update training curriculum (and other support products) required by any subsequent design improvements to the aircraft throughout the program’s lifecycle. The subject matter experts concluded that using digital design methods and advanced manufacturing methods, the V-280 program continues to reduce program risk and advance technology readiness to deliver a minimized lifecycle and sustainment cost. And finally, “Bell is developing AR to deliver point-of-need training, enable task rehearsal, ensure maintainers get maintenance task ‘reps’ on aircraft that rarely fail, and provide multiple options for training of soldiers wherever they may be.”

…and Challenges

Similarly, from the perspective of the military-industry team, AR also has challenges during usage which are preventing its more rapid and wider-spread use.

Bells’ Moody and Rowe were initially asked how these emerging technologies must evolve to permit wider use in maintenance training for the V-280 and other programs. They offered, “All applications of VR and AR must be able to handle large files of detailed component design, procedural instructions, and other enablers similar to ‘YouTube’ videos to enhance instruction, task rehearsal, etc. All the application devices (goggles, visors, etc.) must be rugged and durable for daily use in an austere environment.”

NAWCTSD’s Phillips shared his observations about AR for maintenance training, summarizing the biggest limitations in current Augmented Reality applications for maintainers remain durability, field of view (though this is improving), object recognition, and usability in daylight conditions, and even conforming to information security standards and protocols.

Phillips concluded, “These are all problems that I am confident will be solved in time, but they remain significant barriers to widespread deployment for now.”

Optimism Ahead

Additional optimism about the technology baselines for VR and AR, and even mixed reality, evolving and improving, was voiced by Boeing’s Boeskov. He called attention to evolutions in broad XR (extended reality) applications across AR/MR; the biggest over the next 12-24 months being continued collaboration with its customers to define and deploy relevant VR training content to determine how it can augment and even replace traditional training mediums that exist today in approved curricula. “Additional efficacy data and user feedback can accurately inform the right methodology to achieve the most value-added benefits from VR training,” he noted and predicted that “evolutions in commercial hardware, such as display resolution and field of view, over the next 12-24 months, can likely mitigate these current limitations.”

DiSTI’s Mikalsen recalled one impression from recent industry gatherings he attended. “We are listening to our customers and many of the requests for proposals (RFPs) we are now seeing are specifying the need for VR/AR/MR training as part of the deliverables. We’ve seen the demand grow since last year’s I/ITSEC, and while at ITEC in Germany and TSIS [NDIA’s Training & Simulation Industry Symposium] in Orlando this year, there was strong demand for demonstrating our capabilities for future program RFPs.”

Originally published in Issue 4, 2018 of MS&T Magazine.