Tech Deep Dive: Ground-Breaking Developments in the Synthetic Terrain Sector

The military-government team is expanding the use case for synthetic terrain for training and adjacent missions throughout the spectrum of multi-domain operations.

To support the military services’ “train as you will operate” construct, synthetic terrain products and their technology underpinnings are accelerating at a dazzling pace. While products and systems in this sector continue to be developed with affordability, fidelity and on-time delivery and other traditional ROIs as guidelines, synthetic terrain content is evolving to better support new and evolving requirements: for unmanned system operations; to accurately reflect and integrate temporal changes on the mission landscape; and other outcomes.

At the same time AI, digital twin, data analytics, tiles and other evolving enablers also familiar to teams in adjacent S&T sectors, are helping advance synthetic terrain deployment.

This article includes insights from one military service and two randomly selected S&T companies on current and near-term activities in the synthetic terrain space.

US Army Update

Ryan McAlinden, One World Terrain Operations Lead at Army Futures Command, provided a state-of-the-art report on one military service’s use of synthetic terrain.

The technology expert first noted current terrain in US Army training/simulations is produced using existing aerial imagery, land classification and other data sources from the government, commercial and open-source communities. And while accuracy can vary, in all cases the resolution and currency requirements are met to satisfy Army maneuver training tasks. But McAlinden added, “With the shift to All-Domain Operations, there has been an imperative to ensure the resulting ‘digital dirt’ has the requisite attributes, detail and fidelity to operate across all domains.” While the service executive noted the Army believes the current offerings (pipelines, data, standards) are sufficient to meet mission training requirements throughout the warfare continuum, he significantly pointed out, “demand for this same type of data operationally (C2, intel) continues to accelerate.”

Recalling the earlier imperative for including high-fidelity terrain in Army virtual training systems was to streamline and reduce the time/cost for producing 3-D terrain for Army training systems, he added it was a manpower-intensive, arduous process that took months to build a single location. “Now we can do it automatically in a few weeks,” he emphasized and further reflected, “On top of this, each time a new simulation/training system came to be, the process had to be repeated for every location because none of the formats were compatible.” Fast forward to today, where the pipeline has also been instrumented with high-resolution, precision source data that means the resulting terrains have fidelity and clarity that weren’t possible even a few years ago. “This allows trainees to experience a very close approximation to the real-world when training” McAlinden said.

Industry Insights

Synthetic terrain is not the exclusive purview of the US Army.

Indeed, Randall Toth, VP of Geospatial Products, and Earl Laamanen, Senior Engineering Manager of Geospatial Products, at Bohemia Interactive Simulations (BISim) noted their company’s synthetic terrain product portfolio with VBS, Terratools, Mantle and other offerings, is in use by over 60 nations worldwide, including commercial and, with US DoD and other international, defense customers. BISim’s synthetic terrain capabilities support these diverse end users’ Live, Virtual, and Constructive (LVC) training systems

The two corporate technology experts also shared what they are observing in terms of their diverse customer base’s requirements.

While interoperability, correlation, and reusability of terrain datasets between heterogeneous sets of distributed training systems was at the top of their list, the technology experts also called attention to the generation of Geospatial Digital Twins – to allow immersion in accurate and current representation of the world for mission rehearsal and other operational needs. “Digital twins are typically generated using satellite imagery captured just hours ago, or live drone platforms surveying terrain at the point of need,” they explained and added, “The input data is quickly translated into 3-D simulated environments with accurate representation of location, structures, people, and weather.”

Position Navigation and Timing systems, through industry standard terrain data formats provided by the commercial industry and authoritative US government agencies.

It's informative to also gain insights on one representative product supporting the synthetic terrain sector – BISIm’s Mantle in this case.

Toth and Laamanen said Mantle is a step change in how BISim approaches synthetic terrain. “We’ve taken the best parts of our desktop terrain generation (TerraTools) and runtime publishing of procedurally enhanced terrain (VBS Blue engine) and assembled them as components in a terrain server that provides automated workflows that can address these needs,” they pointed out and added, “By streaming and automatically exporting terrain to a range of formats, it’s possible to deliver terrain and terrain changes much more efficiently than before.”

Cobra Simulation is also expanding the synthetic terrain technology envelope with its unique competencies.

Kenny Mitchell, whose community leadership positions include CTO at the company, initially observed many simulation and training operations of personnel are ground based, e.g., on foot and in land vehicles, but using scanning and reconstruction technologies or 3D manual art production approaches are very costly to create environments with sufficient detail to be convincing at ground eye level. “So, while Cobra Simulation's dome displays provide immersive views of all types of training and simulation scenarios, our CobraWorld technology targets through ground eye level scenarios that previously have been prohibitively expensive and time consuming to create convincingly.”

The technology subject matter expert also provided more context to customer requirements for terrain content, first noting the ability to quickly transform data such as aerial LiDAR (light detection and ranging) into useable content for visual simulations and piece the results together in tiles, is a common use case requirement of our end users. “Picking out ground features from real terrains and cityscapes and making them appear realistically in real-time rendering is a common unique priority for clients.”

As Mitchell noted progress in this technology niche, he also called attention to differences between state-of-the-art synthetic terrain and military customers’ requirements. “For example, comparing with Google Earth’s polygonal-based projected reconstruction, you see unrealistic coarse triangular artifacts, and this severely impacts immersion and thus the trainee's focus on the operation at hand and also results in unrepresentative lines of sight crucial to aiming at targets, creating positional awareness, and other outcomes.”

Enter Cobra Simulation and its latest efforts to close this technology gap. The executive noted the company’s “CobraWorld technology employing what we call ‘voxspar connected volume primitives,’ leapt ahead of technologies like NeRFs (neural radiance field) and gaussian splatting to focus on the key need for a rapid terrain scan to deployed content workflow.” This patented method allows rapidly captured, sparse terrain scans to maintain geometry connectivity and fill details from the sparse points effectively to reconstruct representative terrains with low data overhead. Mitchell continued, “This sparse set of connected data points with low memory usage further has the virtue of minimizing bandwidth needs of graphics hardware and yields a particularly salient balance of efficient local processing of each point with global connectedness.”

Representative Near-term Industry Technology Thrusts

Beyond the synthetic terrain sector advancements previously cited, consumers of synthetic terrain content will see more near-term sector developments.

Future Command’s McAlinden outlined three technology thrusts his offices will be pursuing: automation of 3D terrain production to further reduce human-in-the-loop manipulation of the data; AI-enabled data cleaning – with machine learning to clean up the datasets; and natively sim-compatible 3D terrain. “Many foundational 3D terrain datasets cannot be used in Army sims lack attribution, structure and form. Developing techniques to make this data sim-compatible ‘out of the box’ are important particularly as we seek to reduce the human labor required to generate these terrains,” the service technology leader explained.

Cobra Simulation, with suitable investment, has an aim which follows the completed phase 1 of an Epic Games’ Epic MegaGrant project. The goal: to extend the voxspar evaluation scheme to employ learned inference directly in each reconstruction shader to apply generative AI with real-time game performance in mind. The outcome provides a wider range of environment terrains, buildings and vehicles until any previously unseen scanned location can be effectively and rapidly built for simulation uses, such as for reconnaissance and remote intelligence operations.

Eyes are also on BISim, with Toth and Laamanen stating a key focus is to continue listening to the company’s customers’ needs and “pain points” to ensure they develop solutions and products that solve critical problems and increase operational readiness. “Within our pipeline lies a cloud-based software deployment, increased capability to import and process additional terrain data sources, and last, but not least, an expansion of the number of supported export terrain datasets and formats.”

Help Wanted

Beyond current and planned near-term projects and activities in this space, the US Army still has other opportunities the S&T community may support.

At the top of the Futures Command technology leader’s list was continued approaches towards automation for the production of clean, segmented, attributed 3D terrain. Noting foundation data from the current pipelines can be “dirty,” McAlinden added techniques that improve the quality and pedigree of this data are required, specifically for enhanced virtual training environments. “Additionally, the community still suffers from a ‘source deficit,’ relying heavily on large-scale government and commercial source data providers. Small, organic, easily deployable collection platforms (i.e. drones) are needed to augment these national assets with more precision, accuracy and currency,” the program lead concluded.

Sector Watch

This article provides a glimpse of activities in the fast-moving synthetic terrain space. MS&Twill continue to monitor and comment on other technology and business developments advancing the sector.

Community members are invited to reach out to the author at marty.kauchak@halldale.com with their new and planned activities to strengthen synthetic terrain technology.