‘But It’s So REAL’ - I/ITSEC Showcased an Array of Groundbreaking XR Training Products

From life-size virtual helicopters to a haptic suit that lets soldiers feel faux gunshot wounds, I/ITSEC 2019 demonstrated the rapid evolution of high-tech XR training aides. Rona Gindin reports.

We say this with ultimate and pervasive respect for the seriousness of training troops around the world: I/ITSEC 2019 looked like a gamer’s nirvana. Throughout the trade show floor in December, participants wearing cutting-edge goggles flew virtual military aircraft, explored enemy terrain, observed full-size helicopters and patched up bleeding soldier mannikins – virtual, augmented, or a mix of one or both of those with real life.

XR Basics – Plus the Promise of 5G

This surge of immersive technology comes under the banner XR, for Extended Reality. XR encompasses virtual reality (VR) – complete virtual immersion; augmented reality (AR), realistic digital elements overlaying actual physical items; and mixed reality (MR), which combines actual bodies or objects with virtual elements. The terms were used inconsistently on the show floor, since the XR arena is so new, so here we’ll focus on the capabilities instead. Also, many developers were careful to note their software or hardware is “agnostic,” able to work with a variety of complementary products.

Many of the 550 companies exhibiting at 431 booths are testing, introducing or fine-tuning these XR products and their kin so military outfits can train their troops better or at least more cost-efficiently. And our team of editors couldn’t begin to experience every offering while vying with the population of 17,400 attendees, so this round-up can’t be comprehensive. We can say, though, that the arcade appeal of domes, simulators and blank spaces – even a Holodeck – brought to life by combos of software and headset technology led to never-ending queues at booths offering a chance to test-try so many simulated vehicles, ships and aircraft.

Cost and efficiency are at the crux of the surge toward XR. The US Army calls this virtual world “synthetic training environments”. While leaders are especially excited about the rapid influx of options, they’re equally eager for the chance to integrate many of these products with 5G – the promise of an unbelievably fast “fifth generation” cellular network that is due to proliferate in the coming years. 5G will enable network capacity amid phones, computers, devices and other machines with a fluidity never seen before.

5G will enhance virtual training operations exponentially, panelists said in a talk called ‘5G – From Hype to Reality,’ explaining the processing will be “faster than the human mind”. With 5G, latency will be very low, various technologies will blend more easily, and, as a result, servicemen and women in far-apart locations will be able to train together, in seemingly real time. Training costs will be lower as a result, because, as one speaker said, “We’ll bring training to the soldier, not the soldier to the training.” Soldiers can virtually enter scenarios in dense urban areas or challenging rural ones without the need to replicate situations or equipment in physical space, nor to transport soldiers from one location to another. Instead, they can meet up in a virtual room no matter where each is stationed.

Mixed Reality Blends Hands-on & Virtual Training

Amid I/ITSEC’s exhibitors, the visions of landscapes, weapons and modes of transportation varied widely. At some booths, the visuals were nearly cartoon-like. They were sufficient for learning basic tasks such as cleaning an aircraft or looking for danger on rooftops. Others were ultra-high-definition, populated with images of actual buildings, vessels, vehicles or aircraft, and, as a result, convincingly realistic.

One of the most sophisticated introductions was the XR-1 Developer Edition by Varjo Technologies, a Finnish XR headset maker. Wearing the XR-1 goggles, users can see even minute details in a virtual dashboard, streaming reality with only a 20-millisecond latency, which is practically undetectable. Most notably, the XR-1 has integrated eye tracking, allowing trainees to seamlessly move from high-resolution virtual objects, which are colored realistically, to real-life ones – hence, mixed reality.

On a more basic level, Inlusion Netforms demonstrated how XR using Epic Games' Unreal Engine can help train those who would de-ice an aircraft. A trainee can go into any conference room, put on a helmet, and learn the task step-by-step. Guided by a virtual manager, the trainee will virtually bring the de-icing fluid to the correct temperature, check the thermometer, then move on to another section of the plane.

All Hands On Deck

Some virtual training programs involve controllers, similar to those used with home gaming systems, while others create a more tactile experience – some with bare hands on realistic replicas of items such as guns or cockpit controls, others with virtual replicas together with haptic clothing. Haptic apparel has built-in sensors that allow users to feel life-like reactions to virtual objects. They also send electronic reports on the user’s reactions, which enable trainers to collect useful data and give helpful feedback.

BeBop Sensors touted its haptic gloves, which are wireless and made with “smart fabric” on the bottom two knuckles of each finger. That allows trainees to flick virtual switches and press virtual buttons, feeling the interaction. A military outfit can use it for maintenance training, in place of controllers, so trainees articulate their fingers and build up muscle memory.

Manus pairs its haptic gloves with full-body gear in the Manus Polygon, designed for multiplayer training. Polygon can train soldiers to, as an example, breach a door with hostiles behind it. Each trainee can see the virtual versions of the door, one trainee kicking in the door, the door swinging open, maybe even the rest of the house.

Let’s not forget feet. At Cyberith, the Virtualizer Elite 2 launched this past spring. It’s essentially a round treadmill meant to be used with VR programs. It has six optical motion sensors, which track trainees as they walk, as well as rotation and height sensors. Together with electric motors, it reacts to users’ movements. In a multiplayer military police simulation, say, the motion platform will detect direction, acceleration and speed, then provide data for analysis.

Then there’s the Teslasuit, a full-body haptic suit described as “highly integrated smart clothing.” It can be programmed to deliver authentic physical feelings, even pain, without harming the trainee. Introduced at the partner booth of VR/AR maker Brightline Interactive, the Teslasuit has an on-board mini-computer that delivers sensations via small electrical impulses, essentially telling the trainee how to do each specific step of an operation, such as storming a house. Trainees might feel a sensation every time they turn the wrong way, which when repeated helps develop muscle memory. At a demonstration, a gentleman wearing the Teslasuit reported feeling the recoil from shooting his weapon, and the sensation of being shot in the arm. The suit can also be programmed to replicate hot or cold weather, rain and wind. The suit can collect data to improve simulations in the future. Teslasuit was the honoree in the 2019 CES Innovation Awards.

While we’re talking about wearables, let’s deviate from XR briefly to take note of Chiron Global Technologies’ digitized armor. Soldiers can train virtually first, then don the Chiron-X1 or, soon, the X2, which will allow them to experience what a true enemy assault might feel like – and that means striking vital targets including the head, neck and throat at full force. What’s more, sensors embedded into the armor link to a medical database, which later indicate what injuries each body would have after each impact. The new product will add in “slicing, dicing and ballistic injury,” a spokesperson said, noting, “The paradox to training with lethal techniques is that you can’t do it without killing people. With this armor they can.”

Mixing the Virtual & Real Worlds

Simthetiq specializes in digital twins, which are detailed, realistic, digitized 3D models of more than 3,000 military vehicles and aircraft. To help others understand, the company invited visitors to don goggles and sit in front of authentic reproductions of the controls of a life-size helicopter. Then, in what is actually an empty space except for the controls, testers were invited to step out of the helicopter to view the entire aircraft, peek into the back seat, see the underside as the entirety was “lifted” to the ceiling, and look down upon it as it was lowered into the floor. All virtual.

Kratos Defense & Security Systems invited I/ITSEC attendees to visit its Holodeck (named after the iconic Star Trek feature), a mixed-reality training option formally branded the Reconfigurable Virtual Collective Training System (RVCTS). In a single room, participants wearing helmet-mounted display VR goggles sat in the seats of a helicopter pilot, co-pilot and gunner while a hoist module also enabled hoist or rescue training via virtual reality. They viewed the simulated scenario (gunnery, pilot engagements, etc.) while operating a real weapon replica or cockpit controls, while communicating with the rest of the crew. In the Holodeck, observers saw only three blue-hued wall areas plus the physical replicas.

Augmented reality will be used to train medics when Orlando’s Design Interactive launches its new Optimus product in 2022. Medic trainees will don Microsoft’s HoloLens 2 while perched over a full-size synthetic-tissue mannequin that has every realistic body part except skin. When wearing the smartglasses loaded with Optimus, Combat Life Saver trainees who are learning the Tactical Combat Casualty Care (TCCC) curriculum might see Rescue Randy mannikins retrofitted by SynDaver to include high-fidelity synthetic tissue analogs – only the mannikin will be contextualized with visual injury overlays and spatial audio cues. In other words, the body might appear to be virtually dressed in uniform and bleeding from a leg wound. As the trainees practice applying a tourniquet, they might hear the sounds of gunfire – as they would in real life. The reason: even well-trained medics who are adept at performing a procedure might freeze in action when faced with actual sights and sounds on the battlefield, so this is designed to prepare them better. The company is currently creating instructional content.

Plug & Play, or Make Your Own

Since buyers approach I/ITSEC at a spectrum of budget levels, several vendors provide make-your-own options for XR tutorials. In the simplest, trainers film what they’re doing through smartglasses, edit in text and cut out excess footage, then share the video with newcomers.

Bohemia Interactive Simulations (BISim) and partners demonstrated an Apache XR-based trainer with a high-detail cockpit from Bugeye Technologies, a Varjo XR-1 HMD, a simulation host from Zedasoft and BISim's VBS Blue IG as the virtual environment. This enables trainees to see the physical cockpit, controls, console dials and buttons, and their own hands while also seamlessly viewing the virtual environment merged with the physical cockpit.

LLS  has a cross-platform 3D modeling web-based editor that allows drag-and-drop creation of training videos for everything from changing a tire to handling a crisis such as a rocket falling on a house. It recently began creating 3D models specifically for the military.

Recreational gaming is a significant driver of XR technology and as evidence of this Epic Games Unreal Engine was being exploited across many of the stands in XR applications. As previously mentioned Inlusion, but also Applied Research Associates (ARA) for a joint terminal attack controller VR trainer, SimCentric Technologies team-based VR trainer, Presagis' Dismounted Immersive Ground Motion VR trainer, Quantum3D's VR parachute trainer, and HTX Labs' C-130 transport aircraft wheel and tire change simulator were examples of Unreal-based XR systems.

WorldViz has VR training software called Vizible that is fully drag-and-drop. A trainer in California, for example, can invite a trainee in another location to walk through a procedural task such as removing a radio transmitter from an aircraft’s nosecone. They can work together virtually using a lifelike model of the aircraft. “It’s like iMovie for VR,” a spokesperson said. One trainee can learn by wearing an immersive headset, or a group can learn together via a projection on a wall.

Based on exhibits at I/ITSEC 2019, it’s clear that military units and private companies will blend XR training, in its many forms, with hands-on training  in the near future more than ever before – even before 5G is standard. Buckle up for an exciting ride.