The Academic Pipeline

The use of simulation and virtual reality in education and training today is widespread and expanding. Where will the next generation of “simulationists” come from to continue the evolution? Robert W. Moorman spoke with world-leading universities. Part 1 of 2.

In the earliest days of simulation, the technology was largely aviation focused. But in recent years, simulation and virtual reality tools have become an integral part of various safety-critical fields, ranging from aerospace and defense to healthcare to energy to manufacturing. That has spawned a greater need for more simulationists, or simulation engineers, to design, develop and incorporate these real-life-like tools into education and training, as well as instructors to teach them. Degree programs in simulation engineering, computer simulation, cyber security and related disciplines are being offered at more colleges and universities worldwide.

Heather Bellini, Business Unit Leader, Telecommunications, Media and Technology, Goldman Sachs Research, noted that augmented and virtual reality could “transform how we interact with almost every industry today.” The 2016 Goldman Sachs report expected the VR/AR segment to become an $80 billion market by 2025.

Here are a few of the institutions leading the development of next-generation technologies and technologists.

Cranfield University

Cranfield University’s campus at Shrivenham, the Defense Academy of the United Kingdom, is the de facto full-time academy for the Ministry of Defence’s (MoD) post-graduate education in technology management and leadership. The students are UK senior military officers, Non-Commissioned Officers (NCOs) and MoD civilians primarily. Occasionally, the school educates individuals from other countries. A post-graduate school, the main campus is at Cranfield, Bedfordshire.

Dr. Jonathan Searle, a 40-year veteran educator, is the course director for the modeling and simulation unit. He is responsible for the delivery of a suite of education courses focusing on modeling, simulation and operational research. In military speak, Searle is in charge of producing Suitably Qualified and Experienced Personnel (SQEP) as it relates to simulation at introduction, practitioner, or expert – post-graduate levels.

“We see a growing awareness of the role of simulation in the military coupled with a challenge we face in the UK in terms of resourcing,” said Searle. The challenge is finding trained instructors to teach these courses, particularly in technology subjects, he said.

In some ways, demand is outpacing supply. “We have a paradox here,” he said, “these technologies are becoming increasingly important to us. But we in the Armed Forces [in the UK] are finding it difficult in investing in the people who understand the importance of simulation technology and who are in a position to purchase, operate and manage it effectively.”

Addressing one possible misconception, Searle emphasized: “We’re not educating the simulationists of tomorrow. We’re educating facilitators, leaders in simulation and analysis, planning, experimentation, acquisition, virtual engineering and prototyping.” Often, these are people who write the requirements for simulation-related contracts, he added.

The courses offered at the Cranfield campus vary from a one-week introductory course, which the school runs multiple times per year, to a five-week course as well as two masters-level courses, of which one is Defence Simulation and Modeling.

Most students, who are mid-career, work in the training application area of simulation. A smaller number are engaged in operational research, while others become specialists in areas such as medical services.

MOVES Institute

The nearest US post-graduate equivalent to the MoD-aligned Cranfield is the Modeling Virtual Environments and Simulation (MOVES Institute) at the Naval Post Graduate School (NPS) in Monterey, California. The school has its roots in the NPSNET Research Group founded in 1986. NPSNET was the first low-cost government-owned, SIMNET- and DIS-compatible visual simulator.

MOVES Institute Faculty, students and staff model a variety of VR headsets created over the past three decades. At center is founder Michael Zyda. Image credit: Javier Chagoya/US Navy.

MOVES, as an academic program, was founded in 1996 with the launch of the Master of Science program, followed by the Doctoral program in 1999. The Institute combines analysis by the Operations Research Department and the simulation, training, and software development expertise of the Computer Science Department.

The Department of Computer Science at NPS offers the MOVES degree. Most of the graduates are military officers from the US Marine Corps. Years ago, the US Navy had that distinction. In recent years, MOVES hosted officer students from Germany, South Korea and Singapore.

A noteworthy new addition to the MOVES curriculum is two Artificial Intelligence courses. The first is an introductory AI course tailored to the needs of simulationists.

In addition to a general overall coverage of AI, “we spend a lot of time covering search algorithms, as those are vital to entity movement in simulations,” said Dr. Chris Darken, MOVES Academic Chair, NPS. “We also spend a lot of time on neural nets and reinforcement learning, as these are basic technologies underlying a lot of simulation-relevant current AI research.”

In the advanced course, instructors focus on creating AI for simulation entities. In the most recent offering of the course, instructors devoted considerable time on a system for automated assault planning, which has been advanced at MOVES through various theses.

Darken said the school offers a “basic application of neural network-driven reinforcement learning to simulations, as that technology is highly relevant and progressing rapidly.”

University of Central Florida

The University of Central Florida’s (UCF) School of Modeling, Simulation and Training school, part of the Science program, is considered one of the larger and more varied simulation-related schools in the world. In its 17-year history, the school graduated several hundred students with an MS, some with PhD degrees. [At present, there is no related undergraduate degree available.]

The MST school grew from a need by the US Department of Defense for improved training of warfighters and commanders. Students come to the “interdisciplinary school” with backgrounds in psychology, education, computer science, engineering, science and philosophy and other disciplines.

The University of Central Florida enrolled more than 10,000 graduate students this year. Modeling, simulation and training students can earn MS or PhD degrees or graduate certificates in Modeling and Simulation of Behavioral Cybersecurity and Modeling and Simulation of Technical Systems. Image credit: UCF.

The school provides practical courses that relate to simulation, including mathematics, simulation techniques, coding, continuous and discreet-event simulation, hands-on classroom simulation development as well as human factors to give students a big picture perspective.

The school takes a broad-brush approach to educating the simulationists of tomorrow and those that will hire them. It lays the foundation. “We launch them off to specialize in whatever area they want to,” said R. Paul Wiegand, Associate Research Professor. “These students will know a lot about simulation and then go on to specialize in a specific area of simulation, as opposed to knowing a lot about mechanical engineering and be able to run a simulator.”

Wiegand, who co-manages the Advanced Research Computing Center at UCF, and directs the Natural Computation & Coadaptive Systems lab, believes healthcare, cyber security, psychological assessment and treatment and education are key areas of growth for simulation and virtual reality-aided education. Simulations in education will only get better, as will related technology, he noted. Displays are getting better, in terms of technology, as is immersive virtual reality. So too are physiological measurement devices related to simulation, such as for eye-tracking or measuring cognitive load, a necessary component of training.

Expect too an affordable combination of classic scientific simulation methods with new innovative elements of simulation and virtual reality in the future, added Wiegand. While students come from varied backgrounds, “we get a lot of people from the US Army and Navy and contractors to those branches.”

The Army Research Laboratory’s modeling and simulation group sends students to the school to obtain their Master of Science degrees as part of their tour of duty. The school also hosts students from defense-related companies such as Raytheon, Lockheed Martin and Siemens. The companies also hire graduates from UCF’s simulation school.

There is the added benefit for those students enrolled in the sim school. Next to UCF, in Orlando’s Research Park, is the National Center for Simulation. In addition, there are DoD operations as well as private-sector companies that support DoD and related private-sector companies in non-DoD-related businesses. There’s also the UCF-run high-technology incubator in the Park.

“Students are able to get their degree at UCF and then literally walk to one of the largest simulation hubs in the world for internships or employment,” said Sean Helton, Vice President of Marketing for Enterprise Florida, a public-private economic development organization.

In a related development, UCF’s computer science department built a tool to help first-year teachers in urban areas manage the classroom. The program, Teach Live, has been commercialized, said Wiegand.

Among the notable students from the program, who went on to high-level positions with DoD are: Dr. Syed Mohammad, Director of the Modeling and Simulation Technology Center for the US Department of Homeland Security; Dr. Sae Schatz, Director of the Advanced Distributed Group under the Office of the Secretary of Defense, Personnel and Readiness department; and Dr. Bruce Caulkins, former Chief of the Cyber Strategy, Plans, Policy and Exercises Division (J65) within the US Pacific Command, US Army, now retired and back at UCF as a Research Assistant Professor in the MST program.

University of Southern California

Dr. Albert “Skip” Rizzo, Director, Medical Virtual Reality for the Institute for Creative Technologies, University of Southern California, conducts research on the design, development and evaluation of virtual reality systems for clinical assessment and rehabilitation treatment. Asked where is the greatest need for new simulationists and related VR software programs and technology, Rizzo responded: “Healthcare is a giant area, particularly in mental health rehabilitation and medical training.”

Simulationists are providing enhanced simulation and virtual reality programs for medical professionals, psychologists, social workers and others connected to the health sector.

When Rizzo started in the 1990s, simulation and virtual reality was an immature tool for the treatment, training and teaching in the clinical arts. In the past 15 years, there have been huge advances in the virtual human interaction dimension. Simulation and virtual reality help train people to develop better job interviewing skills. This can be done through role-playing using actors or with VR tools. This training is especially useful for anxiety-prone individuals, who could have social interaction difficulties or troubled juveniles with little or no job experience.

Rizzo said technology has advanced to the point where trainers can build credible simulated worlds with interactive methodologies to help in a variety of ways. With the help of these tools, it is now possible for a clinician to help a PTSD (post-traumatic stress disorder) sufferer confront and reprocess difficult memories through simulation.

BraveMind, developed with the University of Southern California, is an interactive, immersive virtual reality system utilizing exposure therapy for PTSD. Image credit: USC.

One application for simulated education/training came from Professor Rizzo’s work in treating PTSD. The program is particularly useful for soldiers returning from war. The team created 14 different simulated worlds in Iraq and Afghanistan. In addition, the team provided simulations for soldiers on the front end: preparing them for possible emotional challenges in a combat environment. The team built the Stress Resilience in Virtual Environments (STRIVES) program, which consists of five to seven-minute virtual narratives in which a soldier dons a headset, goes on a virtual mission, and then something bad happens.

“The simulation is an emotional obstacle course, a pre-exposure to what they could face in combat,” said Rizzo. At the end of the traumatic episode, out walks a virtual mentor, who guides the soldier through emotional coping strategies to manage stress.

Other simulation-related work is being done at USC. Jernej Barbic, associate professor of computer science, USC, Los Angeles, helps conduct cutting-edge research that led to the development of a simulation of a human hand’s musculoskeletal system in motion. Using magnetic resonance imaging (MRI), with visual effects technology, USC researchers, comprising two computer scientists and a radiologist, developed a realistic model of the human hand’s musculoskeletal system in motion. The innovation could have teaching applications in hand surgery or as a basic tool for an anatomy class.

USC provides a Master of Science in Computer Science degree. Graduate students learn to solve scientific and engineering problems using high-end computers, high-speed networks, and advanced scientific visualization. Achieving the MS degree requires 32 total units for two different tracks: Computer Science, which includes computer animation and simulation; and Computational Science/Engineering, which includes Basics of Atmostic Simulations for Engineers. At USC, there are also courses for BS, MS and PhD programs in simulation.

Purdue University

Juan Wachs is Purdue University’sJames A. and Sharon M. Tomkins Rising Star Associate Professor of Industrial Engineering in the School of Industrial Engineering. He’s worked extensively in the simulation and augmented reality fields as it relates to healthcare business mainly. His work also deals with the fields of telecommunications and information technology.

“We use simulation technology to replicate or convey expertise and instruction from a remote location to a hospital, battlefield or other location,” said Wachs.

Wachs led the Purdue team in 2018 that developed a way for experienced surgeons and physicians worldwide to help less-experienced doctors in war zones, natural disasters and rural areas perform complicated procedures. The Purdue technique uses augmented reality tools to connect health care professionals with highly skilled surgeons and physicians. The AR headset worn by the mentee in the field is designed to replace telestrator technology, which uses a separate video screen and freehand sketches, according to the university.

“Even without having experienced doctors physically co-located in the field, we can help minimize the number of casualties with this technology while maximizing treatment at the point of injury,” added Wachs.

The school has collaborations with numerous hospitals, including the Indiana University School of Medicine Hospital in Indianapolis. It has worked with US Navy and Army medical personnel, surgeons mostly, who want to adopt this technology to train surgeons.

This year, the Intelligent Systems and Assistive Technologies (ISAT) Lab at Purdue, with the help of enterprising graduate students, led an experiment proving that its improved AR telementoring system can significantly increase safety in remotely directed medical procedures. The experiment used an updated version of ISAT’s System for Telementoring with Augmented Reality (STAR). The experiment compared STAR to another telementoring approach, audio-only communication. Twenty corpsmen (mentees) were guided by the remote team of expert surgeons in Indianapolis to successfully complete a cricothyroidotomy (an emergency airway puncture technique, commonly known as a tracheotomy) on a patient simulator.

The telementoring system is not yet on the market. “But we expect this technology will be used by various branches of the military [initially],” said Wachs.

Unlike some innovations that briefly burn bright, then taper off, simulation, augmented and virtual reality as educational tools will continue to improve and expand, according to numerous experts. A significant challenge to consider is what is the future role for human instructors, and how much of what they do now will be supplanted by virtual teachers. More on that dilemma in Part II.

In Part II, MS&T will explore how simulation and virtual reality educational tools are being used at Virginia’s Old Dominion University; Netherland’s University of Technology, aka TU Delft, Full Sail University; KTH Royal Institute of Technology in Sweden and elsewhere.