R&D Mecca in the American Heartland

The University of Iowa may not have the visibility of some aviation schools, but the institution is well known for important research for the civil and military sectors. Robert W. Moorman spoke with their principals.

Often research of new technology serves as a prologue to a corresponding need for new or better training. Such could be the case with various research programs at the University of Iowa’s (UI) Technology Institute (ITI), part of the College of Engineering.

Professor Karim Abdel-Malek, PhD, Director of the Institute, leads the Virtual Soldier Program (VSR), which conducts research and development in human modeling and simulation. Malek’s involvement began in 2003/2004 following the SAE Digital Human Modeling (DHM) conference in Ann Arbor, Michigan.

ITI, formerly the Center for Computer-Aided Design, is a leader in modeling and simulation. ITI’s research focuses, in part, on creating interactive, intelligent and predictive human models that operate in a virtual environment and benefit those in the real world.

The Institute’s mission is to deliver “accurate research to enable validated modeling and simulation of human activities, biomedical engineering, ergonomic analysis, human performance analysis, human systems integration and training.”

Research topics include armor and soldier performance, immersive virtual reality, motion capture, predictive dynamics, physiology, whole body vibration, predictive dynamics, spine modeling and motion prediction.

Avatars for Army Fitness

In 2019, the US Army contracted ITI to provide a third-party evaluation of the six-part Army Combat Fitness Test (ACFT). VSR provided a peer review of the Army’s methods in developing the assessment, collecting data from Iowa ROTC cadet volunteers who performed the ACFT and common soldier tasks.

To help in the evaluation, VSR enlisted ‘Santos’ and ‘Sophia,’ human male and female avatars, to perform biomechanical analyses to supplement the Army’s data. A software system accompanies both models.

“We are currently studying the effectiveness of ACFT in preparing modern-day soldiers,” said Malek.

The ACFT measures soldiers in deadlift, leg tuck, sprint-drag-carry, two-mile run, standing power throw and hand-release push-ups. Baselines were established at three levels based on the physical demands. “The drills predict whether the soldier has the strength requirements needed to perform duties in a combat environment,” according to ITI. Active duty soldiers are required to take the ACFT twice annually; Reserve and Guard soldiers must take the test once a year.

ITI’s evaluation of the ACFT consists of three phases: Phase 1 included a Baseline Soldier Physical Readiness Requirements Study (BSPRRS) at the Army Center for Initial Military Training at Fort Eustis, Virginia. ITI was contracted to validate the BDSPRRS. ITI completed and validated the peer review of the BSPRRS in 2020. Full implementation of ACFT was to have occurred in April 2022.

For Phase 2, ITI collected data from Iowa Army ROTC cadets performing simulated common soldier tasks, such as dragging a litter and hauling supplies. ITI researchers imported data into Santos and Sophia to model strength requirements and “determine if successful completion matches the strength needed to perform combat duties.”

Phase 3, which is ongoing, calls for ITI to evaluate the “cumulative effects of repetition activities,” as well as gauging muscle strength, fatigue and energy expenditure metrics.

ITI found the ACFT to be a “better predictor of fitness levels” than the Army’s old physical readiness program, which required two minutes of push-ups, two minutes of sit-ups and a two-mile run. The new test consists of six comprehensive drills to better prepare soldiers for combat.

Virtual Soldier Receives High Marks

“The University of Iowa Technology Institute has been useful in providing an independent, scientific review of the ACFT’s ability to predict a soldier’s performance of physically demanding common soldier tasks,” said Brig. Gen. John Kline, Commander for the US Army Center for Initial Military Training, which developed the ACFT. “With their use of digital twins and predictive analytics, we can further verify the science behind the ACFT and likely improve Army Holistic Health and Fitness programs intent on increasing Army readiness for years to come.”

The science behind the software in Santos and Sophia avatars is human kinematics, which considers the position, velocity and acceleration of a subject and dynamics, which considers inertias, forces and moments of the motion, said Malek. “What we’ve done over the last 18 years is take a virtual human for male and female and advance soldiers capabilities and answer questions not yet answered,” said Malek. The US Army funded Malek’s program early on to advance the avatar technology.

Simulating Semper Fi Grunts

The Army is not the only military branch to seek help from the Institute. The Institute transitioned its comprehensive “Grunt Sim” program for the US Marines. GruntSIM is a load-bearing simulation that enables so-called trade-off analysis. The program provides “a robust and validated environment for simulating tasks, loading the soldier with varying equipment and observing whether it can be done, calculating biomechanics and physiological performance data.” said Malek.

Avatars are loaded up with various equipment – night vision goggles, weapon, backpack, and body army. Then the virtual soldier is given a specific task and evaluated on that task, calculating biomechanics and physiological performance data.

The instruction process for the avatars is relatively simple. “Hey, Santos, I want you to climb this wall and run ten miles,” said Malek. “At the end of the tests, we will know the energy expenditure and stresses on the body.”

The same task is given to Sophia for female soldiers. Taking into account the changes in body types and strengths of the male and female models, the strategy toward accomplishing various tasks is similar, noted Malek.

The Marines use this program as a procurement tool for potential recruits and to provide additional information on new equipment and technology.

Toward Safer Aircraft Ejections

Some of ITI’s work involves military aviation. Some time ago, the Institute provided research on pilot performance for the Naval Air Systems Command (NavAir). Using the avatars, a cockpit was configured with various levers, push buttons and other controls. Santos and Sophia were tasked to assess touch, reach and other important aspects of a pilot’s performance while restrained.

In a separate ongoing project for an unnamed military contractor, ITI was contracted to assess pilot ejection by replicating the significant forces sustained on the body during the ejection process. “Our team is currently under contract to assess pilot ejection by replicating the blasts/forces sustained on the human during the ejection process,” said Malek.

Military pilots can suffer significant spine injuries when they eject from the aircraft. While ejection seats have saved thousands of pilot lives, there is ample radiographic evidence of vertebral fracture in 30%-70% of pilots after ejection, according to the BMJ Emergency Medicine Journal. ITI is working with the client to improve seat ejection design.

Virtually Aiding Commercial Research

Private-sector companies such as Ford, GM Chrysler, Caterpillar and John Deere have shown interest in virtual human tests of their products in the virtual space. And ITI is tapping into that trend.

“There is a strong interest in the Digital Twin idea,” said Malek, referring to the virtual replication of, say, a factory, engine or anything new being designed. “Allowing testing, assembly, visualization processes and everything else in the virtual space is a powerful idea because it reduces cost and time,” he added. “Less cutting of metal, less cost of manufacturing until the perfect solution is found.

Digital twins, like Santos and Sophia, reduce development and production costs without having to rely on the physical object.

Flight Testing Pods

In its 24-year history, the UI-affiliated Operator Performance Laboratory (OPL), part of ITI, has accumulated 2,700 hours of flight tests and participated in 400-plus projects.

OPL has an impressive record. The laboratory worked closely with Collins Aerospace on the development and testing of the Common Range Integrated Instrumentation System (CRIIS). The technology provides Time, Space and Position Information (TSPI) and systems evaluation data to support testing for a variety of aircraft. Over 250 CRIIS pods and internal mounts have flown on the F-15, F-16, F/A-18, F-22 and F-35 fighter aircraft.

The system is interoperable with the US Navy’s Tactical Combat Training System (TCTS-II), developed and built by Collins Aerospace and partner Leonardo DRS. OPL provided tests and evaluation data on CRIIS. Collins Aerospace supports CRIIS with a 24/7 service call center.

OPL has tested and flown Air Combat Maneuvering Instrumentation (ACMI) pods, a generic term for systems like CRIIS, for the US military since 2010. ACMI pods are used by the military to record an aircraft’s inflight data.

OPL Director Tom “Mach” Schnell, PhD, is a Professor in Industrial Engineering with a specialization in Human Factors/Ergonomics at UI. A veteran pilot, Schnell was the test pilot who flew the government acceptance flights on ACMI pods in 2015 at Eglin Air Force Base in northwest Florida.

“The biggest new thing these days is called tactical autonomy,” said Schnell. “Which means teaming humans with machines in a tactical aviation environment.” He gave an example: “When you fire a simulated weapon on the adversary, TCTS will tell the pilot if it was a good or bad shot.”

One of OPL’s MI-2 twin-turbine helicopters on a mission to test an obstacle detection LIDAR (on left outrigger) and a Collins Aerospace weather radar (inside of radome). Image credit: University of Iowa.

Brownout and HMDs

Years ago, ITI was asked to explore ways of dealing with “brownout”-related helicopter crashes. OPL’s brownout research, funded by Airbus, resulted in a sensor-display prototype system that used Light Detection and Ranging (LIDAR) technology to “see and remember” key features of a landing zone on short final before the brownout occurs “and provide a compelling 3D-conformal symbology on a Helmet Mounted Display (HMD),” said Schnell.

LIDAR is also used for airborne laser mapping.

In a separate project, then-named Rockwell Collins (now Collins Aerospace, a unit of United Technologies) worked with OPL to evaluate its’ Synthetic Vision Aviation Backbone (SVAB) on OPL’s MI-2 helicopter. OPL installed Collins avionics for the Blackhawk helicopter in the MI-2. OPL tested Collins’ Synthetic Vision Systems (SVS) software integrated into the avionics.

“To fully cope with brownout, a helicopter needs all three legs,” said Schnell. Which include the use of use sensors, displays and advanced control laws.

Sensors provide the “see and remember” data before and during the brownout, said Schnell. Although expensive, Millimeter Wave Radar technology is showing promise as a see-and-remember tool.

In the display leg, “we deal with advanced see-through Helmet Mounted Displays (HMDs) that now offer color symbologies and large fields of view” to aid the pilot flying in degraded visual environments.

OPL has worked with the USAF 711th Human Performance Wing (711- HPW) at Wright Patterson AFB near Dayton, Ohio on testing HMDs. The lab also worked with the FAA’s William J. Hughes Technology Center at Atlantic City International Airport (ACY), on civilian rotorcraft applications.

While HMD advanced technology is impressive, formal training on the device is lacking, said the veteran pilot. “I don’t know of anyone providing a formal HMD training program,” Schnell said. “As a consequence, military pilots go to their assignments and learn on the go. A huge training gap exists with regards to HMDs. There should be a drive to create HMD training programs.”

Schnell on Training & Simulation

On flight training: “I am concerned that pilot training in the military especially is going more and more to simulation,” said Schnell. “In a simulator, students can’t get true air sense. Students take higher risks in the simulator because they can’t die.”

Schnell said some of his tech-savvy freshman college students, who’ve stated a desire to fly for the military, seem overconfident in their abilities.

OPL’s leader said aircraft-training sector should tap the brakes on the apparent overreliance on technology and return to an instructor-led process that also embraces technology as a training tool.

“That is exactly how I feel,” said Schnell. ”No doubt there is a huge value in simulation as it relates to pilot training. But under no circumstances should you offer fewer flight hours in the actual aircraft in favor of simulators.”

While the university doesn’t offer a professional pilot training program, it is involved in research comparing how novice and trained pilots handle the workload in the cockpit.

“Novices work hard on everything,” said Schnell. “Their flying is not bad. Novice pilots often ignore a secondary task because we told them to concentrate on flying the aircraft. Whereas experienced pilots do the secondary task and also fly the airplane.”

On the trend toward virtual instructors: “I think it is a shortcut,” said Schnell. “It would have to mature a lot before it’s ready for prime time.”

That said, virtual reality goggles might not replace expensive cockpit systems, but “the technology brings realistic applications,” said Schnell.

Based at the Iowa City Airport, OPL’s testing fleet of aircraft consists of two, tandem-seat, single turbojet engine AV-L-29 Delfin jet trainers, an Aero L-39 Albatros jet, two, twin turbine GTD-350 powered MI-2 Hoplite Soviet-made helicopters and two piston-powered general aviation aircraft, a Beechcraft A-36 Bonanza and Cessna 172. Both L-29s, the NI-2s and the Bonanza serve as aircraft-in-loop (AIL) simulators.

The OPL has a non-motion 737 simulator as well. The device is for “mode confusion research,” said Schnell. Also on site is a Software Integration Lab (SIL) helicopter simulator. “None of our simulators are certified to any type of level; they are research devices,” said Schnell.

A surgeon wears the night vision loupe to conduct simulated trauma surgery on a helicopter. The main purpose of the NV Loupe is to perform emergency surgery on a wounded soldier in the dark as a helicopter leaves a war zone. Image credit: University of Iowa.

Life-saving in the Loupe

Associate Professor Yang Liu joined UI’s College of Engineering in 2019. He wears several hats. He teaches Electrical and Computer Engineering; he’s a researcher for ITI, the Institute for Biomedical Imaging, Center for Bioinformatics and Computational Biology; and is a faculty member of the Iowa Informatics Initiative.

His special fields include Augmented Reality, Surgical Visualization, Imaging Systems Biophotonics, Computer Vision and Computer-assisted Surgery.

Lately, Professor Liu has been involved in R&D of a novel, life-saving Night Vision Loupe for magnification, surgical guidance and decision support. Performing any kind of surgery onboard a helicopter or fixed-wing aircraft under the best of conditions is challenging. With no light? Unthinkable. The Night Vision Loupe adds a dimension to onboard care where the impossible is possible.

During a recent interview, Professor Liu showed MS&T a video of inflight resuscitative thoracotomy in complete darkness on a dead pig model using the Night Vision Loupe. The current Loupe is not battery powered. Yet, battery power is a consideration for future iterations, Liu said.

Research and development of the loupe technology is multi-institution effort and sponsored by the National Institute of Health.

Liu leads the ITI team. Other partners include the Madigan Army Medical Center at Joint Base Lewis McChord, led by Jason Bingham, MD FACS, Director of Surgical Research; and Unify Medical Inc., led by Maziyar Askari Karchegani, PhD. Ongoing for nearly three years, the partners are in the process of commercializing this novel surgical aid.

“The Night Vision Loupe allows one to perform a variety of surgeries at night,” said Liu. “If someone has a traumatic injury, this Loupe is a unique way of saving someone’s life.” With this device, “you can train residents, with a minimal learning curve,” he added.

The Loupe is part of a wave of new smart eyewear technology that has varied applications, including avionics, simulation and consumer products. “We are at a technology inflexion point,” said Liu, “where AI and camera images are getting smaller, cheaper and better.”

Liu’s team is involved with other medical augmented reality and 3D display research. At present, they’re working on developing fluorescence imaging eyewear with 3D display. “We are also working on multimodal imaging and 3D display (e.g. CT and fluorescence imaging merged), he said.

Many of the advances in aerospace technology often begin at the university level through strategic research and development partnerships with the civil and military sectors, aircraft manufacturers, avionics suppliers as well as the medical community. In its 41-year-plus history, the UI-affiliated ITI has become an integral part of this phenomenon through human modeling and simulation and systems and sensors and other types of valuable research.