US Navy Cautiously Embracing XR

From VR flight training devices to AR gunnery trainers, the US Navy is incorporating X-Reality technologies to help modernize its training programs. US Editor Chuck Weirauch reports.

Navy commands and other organizations are becoming more aware of the increasing number of technology options available for training. Captain Tim Hill, Commander, Naval Air Warfare Training Systems Division (NAWCTSD), shared some of the Service’s current thinking and XR programs during a roundtable discussion with MS&T that included some of the Division’s program managers.

Flight: Enhanced Chair-Flying

One example of how the Service is adopting XR technologies into its training programs is the T45 Goshawk VR and mixed-reality part-task trainer. This trainer was initially developed by Bohemia Interactive Solutions under a 2018 NAWCTSD contract. Since then, 11 T45 training devices that are variants of the original trainer have been deployed to several naval air stations, with four similar T6 Texan II VR devices located near Chief of Naval Training (CNATRA) headquarters in Corpus Christi, Texas. All have been used by initial Navy student pilots on an informal, free-play basis and not in formal courses. NAWCTSD has completed a formal cost-effectiveness study on the trainer, but that report was not available for public release at press time.

However, there has been considerable positive feedback from student pilots who have been involved in the initial experimentation with the VR trainers.

“The way that these devices are being used today is what I would call enhanced chair-flying,” Hill explained.” By that I mean when we were at flight school and getting ready to do an event, they would teach us how to do chair flying, or sitting in a chair and mentally going through the process of that event, trying to draw a sight picture. And some of the comments we have gotten back from students using the T6 and T45 VR devices is that now they do not have to build that type of mental picture because of the virtual reality or augmented reality technology we have there (in those devices).”

The Navy is also working to develop a TH57 training helicopter VR and mixed-reality training device, “but we are not there yet,” Hill pointed out. NAWCTSD panel discussion members also stated that there is “a lot of value and potential” in these trainers, but that they do have some limitations to overcome, such as visual acuity of the headset, for example. It was brought up during the discussion that VR does offer a 360-degree situational awareness for the student, an advantage that needs to be considered against some of those limitations.

Carrier Launch Team Trainer

One of the success stories in employing VR reality for training that Hill likes to cite is a Command prototype, originally named the Flight Deck Crew Refresher Training Expansion Packs (Flight Deck TEPs) system. This trainer R&D was funded by the Office of Naval Research (ONR). The system features a VR headset for the aircraft carrier catapult officer, or “shooter,” as a part of an expandable framework of game-based 3D immersive technologies. The virtual environment provided for the shooter via the VR HUD simulates the entire carrier flight deck and all on-deck crew members of the launch team. The trainer is also networked with the Primary Flight Control (Pri-Fly) team, including the Air Boss and Mini-Boss team, and the Landing Signal Officer (LSO) in the bridge control tower. They observe and coordinate flight deck operations through several monitors in the tower, rather than employing a VR or AR headset for situational awareness during the training exercise.

This prototype has since been further developed, with expansion packs created for other flight deck launch and recovery personnel, such as arresting-gear team members. It has also been delivered to the Fleet at A schools at Naval Air Engineering Station Lakehurst in New Jersey and at the LSO school at NAS Oceana in Virginia, according to Courtney McNamara, NAWCTSD’s Advanced Gaming Interactive Learning Environment (AGILE) Team Lead. The Service is also trying to get the trainer considered as a training program of record for the Air Bosses and Mini Bosses.

Hill likes to use this trainer to highlight the Navy’s approach to incorporating a mix of new and more conventional technologies into its training initiatives. That is because the Flight Decks TEPs system incorporates new VR as well as 3D and 2D technologies, along with gaming technologies in one unified system. The point is that training for some deck crew launch team member positions does not require, or may not even benefit from, XR technology applications, Hill said.

“We are talking about extended reality all the way from 2D to 3D through the continuum to virtual reality, with augmented reality falling into the middle of the spectrum,” Hill stated. “From there, you get into the conversation of what is the right kind of training that you want to deliver – the knowledge transfer that you want to make happen.”

“The Flight Deck Crew Refresher Training Expansion Packs (TEPs) system is really a good model of what we are trying to do at the Command and the Navy is trying to do as a whole in introducing these technologies,” Hill continued. “Really, we are trying to take a fundamental approach every time we put something out into the field. We are looking at what does the customer or user need to know to do a task and get proficient at that task, what is the best way to deliver that training, and to let the student get the reps and sets to get proficient.”

This kind of training delivery technology analysis – what technologies should be applied to a particular training objective – is becoming more important as the kind and number of technologies to be applied to the training goal increases.

“The fear of using some of these newer technologies is gone,” Hill said. “We have seen more times than not that customers have come to us, saying that ‘I want VR or AR’ or whatever. So sometimes we have to talk them off the ledge, because we understand the requirements, and we can say that maybe that is not what they really want based on what they are trying to do. So it’s not the question as to whether VR or AR will do the training or not. It’s what is the best bang-for-the buck to deliver it. But we are seeing more customers certainly exploring those options for training and for job aids as they are coming up for refreshes or new requirements.”

Assorted AR Applications

One area where the Navy has successfully employed AR technology is the Unified Gunnery System Augmented Reality (GunnAR) system developed by the Service’s Battlespace Exploitation of Mixed Reality (BEMR) Lab. Gunners wear an AR helmet provided by Daqri, while a shipboard gunnery liaison officer sends commands to the gunner’s tablet. Those commands are converted to visual images projected onto the Smart Helmet’s transparent lens. The system is much more effective and accurate than sending those commands via conventional shipboard voice systems, developers say.

The Weapons Augmented Reality Scoring System employs a Microsoft HoloLens system to project images of enemy ships onto the HUD display for live-fire exercises, with simulated target hits and misses shown on the HUD eyepiece. NAWCTSD has developed some AR gunnery training systems for several helicopter platforms for NAVAIR that provide an outside-the-cabin view.

This year, ONR, the Naval Surface Warfare Center, the Army and industry collaborated to develop the Tactically Reconfigurable Artificial Combat Enhanced Reality (TRACER) project. This system incorporates AR and VR technology to train Service security forces.

Navy medical personnel have been experimenting with AR to train for medical procedures.

RRL Apps Underway

With the Navy’s Ready, Relevant Learning (RRL) initiative now underway, MS&T asked the NAWCTSD discussion panel members if and how that effort might take advantage of XR technologies to enhance the learning process for sailors. It has been postulated by some educators that XR technologies can “revolutionize” the learning process. The long answer from the panel was “not yet,” but there could be some application of these emerging technologies in the near future.

“We have so far delivered one operational specialist rating training program in RRL,” Hill reported. “There was a lot of game-based, gaming engine software that was a part of that trainer that provided a lot more immersive experience. But it was 3D on a 2D platform, and nothing particularly high-end beyond that you can get in a gaming console today. But the feedback we have gotten so far is that it draws the students in considerably more than PowerPoint or flash cards. There is some gamification in it as well, and certainly that‘s the kind of environment that a lot of our recruits have grown up in terms as to what is familiar to them.”

That statement might indicate to some that immersive XR technologies have a bright future in RRL. However, it was brought up during the roundtable that because of the hardware and infrastructure requirements for AR, VR and MR, they might be somewhat of a drawback. The training objectives might not call for such capabilities that the emerging technologies offer, such as a 360-degree situational awareness viewpoint, for example.

“As far as XR, we are still operating at the left-hand end of that beyond a 2D type of technology,” Hill pointed out. “But I think that there is an average chance that we will see some sort of augmented reality in the not-to-distant future, depending on what job services we are talking about. That’s because some of these training objectives might be accomplished as both a training aid and a job aid, with such things accomplished with something such as a tablet at a much lower price point.”

The Future XR Navy

While there are many situations where XR technologies might not be the most applicable means to provide the most cost-effective training solutions for the Navy, Hill said that as the Service gains more knowledge and experience with them, the more “bullish” the Navy has become on their application for training.

“This kind of technology can open up a really more cost-effective way to give people more reps and sets,” Hill summed up. “There is no way we can afford to buy the number of operational flight trainers that we need in aviation, for example. It’s just not affordable from an infrastructure perspective, and there is certainly no way we could take those out on the ship with us.”

“The kind of (XR) technologies we are talking about here are absolutely more affordable and more portable to the environment out at sea, where we usually have room just for the footprint of a gaming computer,” Hill continued. “If we are able to get to a place where we can overcome some of the obstacles – or in some cases, just recognize what some of the limitations are, it will be a whole new world for us in terms of more proficiency for pretty cheap, relative to what we have been used to in the past. And that part is really where we are a little bit more bullish than in previous years on these technologies.”

Published in MS&T issue 6/2019