Has the Military Been Building the Metaverse?

It seems that most tech company CEOs are mentioning the metaverse this year. What is the metaverse, why is it important and is the military already building it? MS&T Editor Andy Fawkes reports.

Towards the end of the 20th century as interactive 3D virtual worlds and connectivity of computers advanced, many science fiction writers and filmmakers envisaged a future where the real and virtual worlds would come together and sometimes be indistinguishable. In Ender's Game (1985) the protagonist thought they were in a simulation whilst they were actually in combat, and in the Matrix (1999) the majority of humans were unaware they were in a simulation. A term that has generally come to define such convergence is the “metaverse” first coined by Neal Stephenson in his 1992 science fiction novel Snow Crash. In the novel, humans spend their time in a “computer-generated universe that their computers draw on their goggles and pump into their earphones … an imaginary place known as the Metaverse”. The attributes of the digital avatars controlled by the humans were based on their social economic status and wealth, an extension of the physical world in digital format. In a recent interview the author told the Economist, “I was just trying to imagine what a popular medium would look like centered on the use of 3D graphics technology and the metaverse was my best guess as to what something like that might look like.”

An Interoperable, Persistent, Connected Virtual World

So, what does the metaverse look like from a 2020 perspective? Although many point to the 2018 film Ready Player One and its OASIS (Ontologically Anthropocentric Sensory Immersive Simulation) as the latest inspiration, it is challenging to describe the metaverse. Venture capitalist and former Amazon executive Matthew Ball wrote this year “just as it was hard to envision in 1982 what the Internet of 2020 would be, we don’t really know how to describe the metaverse”.

However, Ball believes it “brings all of the technological revolutions of the past 40 years into a far more interoperable version of the world” with persistency and real time operation. The metaverse should bring together the current virtual worlds that have been built as walled gardens with their own laws of physics, currencies, and customs and link them to the real world. Users will be able to create content for use or sale across the metaverse and interact with others in an experience that spans both the digital and physical worlds. As a consensus view, Wikipedia states that the metaverse is a “concept of a future iteration of the internet, made up of persistent, shared, 3D virtual spaces linked into a perceived virtual universe.”

Although virtual worlds such as Second Life provide many elements of the metaverse with persistency, user created content and an economy, it is now principally the games industry that is bringing the metaverse concept to a more mainstream audience. Epic Games’ CEO Tim Sweeney told the LA Times this year “just as every company a few decades ago created a webpage, I think we're approaching the point where every company will have a real-time live 3D presence, through partnerships with game companies or through games like Fortnite and Minecraft and Roblox … it will be the beginning of tearing down the barriers not just between platforms but between games.” If Sweeney’s vision comes to fruition then games engines such as Unreal and Unity have the potential to be the foundational layer of a new, shared simulated reality. Some financial experts have factored in Unity’s recent IPO success down to the potential of the metaverse.

A Social Utility

Roblox is an online game platform and game creation system which has grown significantly in 2020 with Roblox saying three-quarters of American 9- to 12-year-olds play its games. During lockdown, birthday parties and classes have been held on the platform and in his 2020 keynote CEO David Baszucki spoke to his vision of a metaverse as a “social utility” with “me and my digital self as an avatar, and my friends, both the ones I find on Roblox as well as my real life friends coming together, connecting people around the world globally.”

Gaming platform Roblox launched 'Party Place' in July 2020 catering for the growing demand for dedicated places to host virtual events such as birthdays. Image credit: Roblox.

As for non-gaming virtual worlds, Nvidia’s CEO Jensen Huang told the October 2020 GPU Technology Conference “the metaverse is coming. Future worlds will be photorealistic, obey the laws of physics or not, and inhabited by human avatars and AI beings.” Huang showed a demo of the Omniverse, where engineers can work on designs in a virtual environment.

It is not only advancing software technology that are providing inspiration for the metaverse. XR technologies can provide a more immersive “window” into the virtual world but also can blend the real and virtual worlds. Since 2016 the AR game Pokémon Go has bridged the two worlds with 2020 reportedly its most successful year with $1bn in income. As technologies such as eye tracking and haptics evolve, our levels of immersion and interaction with the metaverse will only enhance.

What About the Military?

MS&T readers might be wondering what has the metaverse got to do with the military? On one level, the military have been tracking games industry developments for over two decades now but also there is a recognition that new recruits will be digital natives and gamers. This is much more the case now as children have had to spend months away from their friends and places of education due to the pandemic. Looking at Fortnite, Minecraft and Roblox, children have perhaps been living in an early form of the metaverse and this experience will likely stay with them a lifetime.

It might be easy to jump to the conclusion that military thinking is only just playing catch up with the metaverse. However, over 40 years ago in 1978, USAF Captain Jack Thorpe envisaged networks of simulators that would be used for combat planning, rehearsal, and execution. In his paper “Future Views: Aircrew Training 1980-2000” he foresaw that simulation and mission systems would come together and would “tightly align training systems with real combat readiness and make them indistinguishable”. Thorpe went on to lead DARPA’s SIMNET program which by the end of the 1980s culminated in over 200 local and wide area networked simulators of tanks and aircraft connected within the US and across to Europe. To reduce network traffic, a single basic unified “world model” was shared across all the simulators, each of which would calculate their own status and broadcast corrections to the world model as necessary. The approaches developed at the time led to the Distributed Interactive Simulation (DIS) protocol, still in use to this day, and together with the more advanced High Level Architecture, different military simulations can now be linked to provide a richer collective training or mission preparation experience.

One of DARPAs SIMNET networked tank-simulators at the Fort Knox US Army Armor Center 1986. Image credit: The US National Archives.

It was not only the networking of virtual worlds. In the early 1990s, together with the US Navy, DARPA networked the U.S.S. Wasp, in port at Norfolk, Virginia, with Marine Corps SIMNET simulated helicopters at Fort Rucker, Alabama, simulated Army tanks, Navy staff command centres, and an observation node at the Institute for Defense Analyses. The Wasp crew could “detect” the simulated helicopters and the pilots could see the simulated ship. This live and virtual integration developed into the LVC or Live-Virtual-Constructive construct and using protocols such as DIS it is now possible for militaries to introduce simulated entities and effects routinely into live training and vice versa. For example, in the British Army’s “Synthetic Wrap”, the positions of combatants in the real world are used to generate a simulated UAV camera feed for an HQ without requiring the expense of an actual UAV.

After SIMNET the military continued to embrace the idea of the real and virtual worlds coming together. In 1999 the UK MoD defined synthetic environments as linking “a combination of models, simulations, people and real equipments into a common representation of the world providing consistency and concurrency across previously discrete activities”. In 2007, the then UK Chief of Defence Staff Sir Jock Stirrup told an RAeS audience “the crew of a submarine when it is submerged are in many ways already operating in a virtual environment … and we can see emerging the fusing of synthetic and real world environments such that it would make increasingly difficult to distinguish between training carried out in simulation and the real thing.” And up to the present day, the leader of the US Army’s Synthetic Training Environment Cross-Functional Team Maj. Gen. Maria Gervais said this July that her team’s goal was to establish a common synthetic environment, where live, virtual and constructive training can be converged through “common standards, common data, common terrain and an open architecture.”

The British Army has also been investigating the use of XR technology at scale, with over 30 soldiers in the same virtual training scenario. Perhaps the most ambitious project to simulate the whole battlespace is the UK MoDs Single Synthetic Environment (SSE) which, inspired by Improbable’s SpatialOS technology, is aiming to harness technology advances in gaming and the Internet. It recognises that the world has become increasingly more interconnected and data driven and it is thus more challenging for decision-makers to understand, visualise and respond quickly. According to the MoD SSE “Opportunity Detail” document, a “digital twin” of the real world will be built with equal complexity “as a medium to harness the challenge, visualise the hidden and transform decision making support”. Providing access to models and data across the enterprise, the SSE perhaps has potential to be the foundation of a secure military part of the metaverse.

So, has the military been developing the metaverse? Many aspects of the metaverse have been created and are being used routinely. Through the use of interoperability standards, military simulations from different vendors can be linked to provide a larger richer, collective training experience and live training can be enriched with virtual worlds and vice versa. What has not been achieved thus far is a persistent network of live and virtual worlds that the warfighter can exploit 24/7, meeting and training with other warfighters, sharing ideas and innovating. They cannot easily carry their digital twin or avatar and records from one simulation to another. There is no ability to easily discover, measure and decide what is the most cost-effective training. There is no economy as such where different training systems compete with each other and respond quickly to demand. Further, is the military making significant efforts to understand what the next generation of recruits need for their training? A generation that is living in the early days of the metaverse.

Learning from Different World Views

Equally, is the wider world learning from the experience of the military? Maybe open standard protocols such as DIS and HLA might be the basis of the interoperability that the metaverse requires. And maybe a metaverse-like persistent military simulation infrastructure has not been developed as there has been no compelling reason or reasons to do so, or has it been because there are too many vendor stovepipes? Likewise, what are the compelling reasons for the whole of society to live their lives in a metaverse and can industry work together to make it happen? In Snow Crash, the metaverse’s design is “fixed by a protocol, hammered out by the Association for Computing Machinery's Global Multimedia Protocol Group”. Is the gaming industry willing and able to do the same? A fascinating future awaits.