What happened to the virtual reality gaming revolution?

Six years ago, consumer virtual reality seemed set to be the next major tech breakthrough.

With the demonstration of his impressive prototype Oculus Rift head-mounted display (HMD) in 2012, Palmer Luckey managed to instantly erase the poor image VR had garnered from ‘90s movies like The Lawnmower Man and woefully premature commercial curios like Nintendo’s Virtual Boy. This led the Kickstarter campaign for the first Oculus developer kit to balloon past its $250,000 funding goal on the way to a final haul of $2.4 million. Two years later, Oculus accepted a $2 billion buyout offer from Facebook.

The lead-up to the 2016 launch of the first consumer version of the Oculus Rift (the CV1) only raised consumer VR’s profile further. Analyst predictions were bullish, going so far as to say that the VR market would be worth $150 billion in just five years. Oculus’ co-founders were breathlessly profiled in glossy magazines, with Luckey landing on the cover of Time in August 2015. Google even partnered with Disney to give away its low-tech paper Cardboard sleeves, enticing fans of Star Wars and other mega properties with themed mobile experiences. Decades removed from the hangover of failed VR arcades and gimmicky consumer trinkets, things would be different this time.

Double Fine’s Tim Schafer put it best at DICE 2016. “We all wanted Snow Crash to happen, and then we put on the things, and it was just Pterodactyl Terror, and we all threw up,” he told Ars, possibly (jokingly) misnaming Virtuality’s less-than-stellar VR arcade experiment Dactyl Nightmare. “I think there’s been a huge leap [this time].”

Six years later, VR has yet to reach the stratospheric heights its cyberpunk fantasy promised. But the latest wave hasn’t been another high-profile failure, either. Meta’s Quest 2 headset has helped significantly revitalize consumer interest in the sector with its user-friendly experience and relatively low price (though it’s not as low as it once was), with its Oculus Store supporting a handful of bona fide VR-native hit games.

This all goes a long way toward explaining how, given the ups and downs of iteration and experimentation that followed Rift’s consumer release, VR developers and watchers told Ars they’re still excited about virtual reality—and they’re thrilled to see where the technology is heading next. And while the initial enthusiasm about its global impact has been tempered a bit since 2016, most in the space now say it doesn’t need to have a profound impact to be a success.

Hype meets reality

The Rift CV1.

Credit: Evan-Amos

When the Rift CV1 was released, evangelists proclaimed that VR wasn’t just going to revolutionize games—it would change the world. (Goldman Sachs said in 2016 that mass adoption of VR hardware alone would overpower the $99 billion TV market by 2025, and it was hardly the only company making such lofty claims.)

But an instant revolution was never in the cards, as Road to VR executive editor Ben Lang told Ars. “The expectation among the nascent industry was that it was going to be this crazy takeoff,” Lang said. “But as happens with very new technology, until you can go from pure hype—like, ‘this is going to change everything,’—to really finding specific useful cases, it never becomes this instant, overnight thing.”

Back in 2016, it seemed that every major tech company was eager to carve out its piece of the VR pie. Rift and HTC’s Vive were available for PC early that year, while Sony’s PSVR would be out in October for PS4. On the mobile side, Google improved on its Cardboard product with its mobile-powered Daydream to counter Samsung’s Gear VR.

PSVR.

Credit: Mark Walton

All of them had flaws. The Rift and Vive offered low-latency, room-scale VR (allowing users to optionally move around a to-scale physical space), but they needed $1,000-plus PC rigs for its high-res visuals and used external sensors that users had to install. PSVR’s cheaper price meant a processing ceiling, cruder motion tracking, and lower-fidelity games.

Mobile options might barely run at all, being at the mercy of a user’s under-spec smartphone that could often lead to choppy, nausea-inducing experiences—and any devices worth more than a cursory look were locked to Android phones. To top it off, the more capable first-gen HMDs weren’t exactly light and could be uncomfortable if worn too long.

The HTC Vive.

Credit: Lee Hutchinson

Controllers were another mixed bag. The Vive offered the best motion tracking available, while the Rift initially shipped with an Xbox One controller (until Oculus introduced its Touch controllers later that year). Though better than PSVR’s finicky PS3 Move tech, first-gen tracking on PC was nevertheless prone to calibration issues. Gear VR and Daydream were a step behind that, only supporting “three degrees of freedom” movement that didn’t let users move their heads or arms freely in a digital space. (And without front-facing cameras, strapping on most headsets amounted to being blindfolded.)

Despite improvements to hardware designs over the next few years, the initial appetite for VR consumption out of the gate was nowhere near what investors had been counting on. The future would have to wait.

A lot of hassle

Aside from hardware itself, adoption of the first generation of new VR devices remained in a niche largely because early HMDs weren’t known for their ease of use. The resulting “friction” in the user experience took any number of forms—motion sickness, platform accessibility, difficult physical hardware setup, PC compatibility, motion control calibration, and onboarding—all of which were major hurdles to enjoying first-gen setups.

Andrew Eiche, the head of Job Simulator developer Owlchemy Labs, recalls some of the painful specifics. “You had to get a big, beefy computer and put holes in your wall to hang [sensors] at the corners of your house to get the best tracking possible, with lasers shooting everywhere,” he said. “And then you had this setup that takes up to 30 minutes to get stable—and then it was finally time to go. That’s a lot of friction, right?”

Ironlights developer E. McNeill agreed, though he has found that hardware user-friendliness has improved over time. “I’m not that technically challenged, but I feel like every time I use PC VR, I have to do some sort of troubleshooting,” he said. “That’s less true now than it was, but for a long time, it was a pain in the ass more often than it should have been.”

Lang gave a simple example by way of comparison: sending a text. “If my headset was the only place I could send a text, I’d never ever go through all of those steps—put on the headset, turn it on, boot an application and type it there,” he said. “But because my smartphone is five seconds away, I do it all the time. So as friction decreases, practical use cases expand.”

In other words, whatever you’re doing in VR has to be worth a significant amount of trouble. “Right now, VR is great for a small number of things that warrant going through that friction,” he said. “So if that’s an amazing, super-immersive game you can play for an hour or two in one sitting, you’ll go through that five minutes to get it all set up for that big reward.”

A new Quest

The Oculus Quest, as enjoyed by an invisible model.

Credit: Oculus

When it comes to reducing that friction, the 2019 release of the standalone Oculus Quest was a true game-changer, according to developers we talked to. “It’s better to think about Quest 2 as a console,” Cloudhead Games CEO Denny Unger told us. “That’s really what it is. It’s an all-in-one VR console.”

Unger said that Cloudhead made rhythm shooter Pistol Whip specifically with the Quest in mind in 2019. “So when a lot [studios] saw that early on, we recognized it would be a significant mover of the technology for mainstream adoption. And that turned out totally to be the case.”

Chris Milk, CEO and cofounder of tech company Within, was similarly mindful of accessibility when developing its subscription-based fitness app Supernatural for Quest hardware. Though resembling other VR fitness game layouts at a glance, Supernatural uses what Milk calls distinct “modalities” for boxing, meditation, and “flow,” or full-body aerobics, all accompanied by scripted trainer instruction—and specifically designed to be as easy as possible.

But Milk believes the “bifurcated” divide between PC and mobile—and how they were marketed—didn’t convince anyone that VR was easy or worthwhile before the Quest.

“You had tethered headsets with controller tracking—basically the functionality of a Quest on a far more expensive, cumbersome, and complicated model—and you had 3 DoF [degrees of freedom], cheaper, way more accessible headsets with super limited processing,” he said. “The [split] way that evolved wasn’t advantageous for VR’s adoption, growth, and abundance.”

Milk said that in a pre-Quest world, he wouldn’t have made a VR app in a PC-only environment. “Supernatural doesn’t work on a headset plugged into a $1,500 gaming PC; it works exceptionally well for an all-in-one device. We built it with the knowledge that all-in-one devices were coming.”

Facebook released its last tethered Rift headset in 2019, discontinuing the line in favor of the console-like, all-in-one path exemplified by the Quest line. Lang sees it as a needed course correction after 2016’s false start.

“By moving to standalone, Meta acknowledged that PC wasn’t working [for VR],” he said. “That was a conscious effort to start eliminating the biggest friction issues [and] PC troubleshooting nightmares—who knows if their USB controller has enough throughput? So eliminating PC wasn’t just cheaper; it was easier to set up.”

Meta’s world

The Quest 2.

If there’s a single product defining the state of VR in 2022, it’s the Quest 2. Meta’s second all-in-one headset, released in 2020, reigns over the VR space, accounting for as much as 90 percent of HMD sales, according to a June report from the market research firm IDC.

All told, the Quest 2 has sold around 15 million units so far, according to IDC Research. To put that number into some context, that’s roughly on par with estimated lifetime sales of the Xbox Series X|S in late 2020 and only slightly behind 22 million sales for the PlayStation 5 in the same timeframe. Then again, the Nintendo Switch has sold about 52 million units since the Quest 2’s launch (and tens of millions more before it).

Still, Quest 2 sales are more than double the sales of its nearest VR competition, PlayStation VR, which sold approximately 6 million headsets since its 2016 launch. It’s also about 15 times the conservative estimate Oculus co-founder Brendan Iribe once predicted for sales of the first Rift.

The explosive success of the Quest 2 has completely changed the tenor of the VR software market, according to Survios CEO Seth Gerson. “[VR] went from a 500,000 unit installed base in 2017 to more than 14 million today,” Gerson said in an email. “That is real growth.”

McNeill took a more measured approach to describe the “Quest effect,” though he agrees Meta’s headset has been a breakthrough.

“I think Quest 2 really was a big inflection point for mainstream adoption. And… it’s a curve that’s still sloping upward,” said McNeill, who has made five VR games on various platforms since 2014. “And it’s really hard to say that VR has made it or is dying or really anything while it’s still on that upward trajectory. It’s frankly too early. Nobody knows where that curve will bend and plateau.”

Comparisons between the Quest 2 outselling more powerful Xbox Series hardware came up in multiple conversations as well. “Is Xbox mainstream? I have yet to find anyone who says it is not,” said Gerson, whose studio has made a name for itself with popular licensed VR titles like Creed: Rise of Glory and The Walking Dead: Onslaught. “Our larger [VR] games sell more than 2 million units of software alone. With new devices and more hardware coming online next year, you can start to add millions of additional units per title.”

Beyond the current success, Gerson is confident that VR is just now on the cusp of a much bigger moment. “As I look at our weekly sales numbers this year, I only see volumes increasing, to the point we have to adjust our yearly forecast upward,” he said. “Immersion is the future. It is the paradigm shift, and the inflection point was last year.”

Setting new standards

The success of the Quest 2 comes down to several factors. The lack of major rivals releasing comparable headsets around the same time certainly helped, as did a pandemic-led boom in demand for immersive entertainment at home.

But the Quest was also given a big leg up thanks to one major technical innovation: inside-out tracking. Unlike most earlier headsets, which required external cameras to keep track of head and hand positions, the Quest hardware uses algorithmic data and camera sensors embedded in a headset to detect the physical environment around a user.

Since the release of Quest, the market has shifted to the point where every single VR headset in development today is expected to use the feature. That wasn’t always fated to be the case, though. “It’s so funny to think of inside-out tracking as the gold standard when it was this experimental, weird thing that sometimes shifted your floor into your face,” Eiche said of the tech’s beginning as a Facebook research project.

While tethered PC headsets like the Rift S also used inside-out tracking, the Quest line improved things by going completely wireless. Looking ahead, the Quest Pro, ByteDance’s Pico 4 series, and Apple’s mixed-reality set have all opted to drop a mandatory tether, though, like Quest 2, they will likely include the option to wire into a computer for a bump in processing power. Headsets that require a tether, like next year’s PS5-tethered PSVR2 or the premium-priced Valve Index, are increasingly the outliers.

Unger said the Quest 2’s standalone design is a major factor in VR continuing to gain new users. “You have a stable of games without the need to buy an expensive PC, you can hook it up to a computer, you can throw it in a backpack and take it to a friend’s house—it does all of the things you need VR to do,” he said. “So you’re seeing a refocusing of the industry to get their heads around what ‘standalone’ means. Everybody’s going to be working on standalone devices from this point forward.”

Running on Qualcomm’s mobile-centric Snapdragon XR2 chip, the Quest 2 can’t compare to the sheer pixel-pushing power of a tethered VR headset. For most average users, though, that level of over-the-top performance is less important than price and ease of use.

“You don’t buy a console if you want the very best 4K graphics,” Lang said. “But you’re going to get that smooth, consumer-friendly experience. So that’s what Meta’s doing. And if we’re talking about adoption, I think they’re demonstrating 100 percent that this is the way to go—you’ve got to cut that friction down big time.”

Milk noted that his mom uses Supernatural more than he does, an anecdote he thinks serves as a strong reason to support standalone hardware as the standard. “If my 75-year-old mom and others like her are buying this headset from Best Buy, setting it up themselves and using it on a regular basis, I’d say you have a mainstream device,” he said. “It may not be so widespread that people have discovered it yet, but I think they will, and I do think that exercise will be the first killer use case for a general mainstream audience for the technology.”

Hit software

Beat Saber trailer.

The relative success of the Quest 2 has come alongside a number of hit games for the platform. Developers name-dropped rhythm-dance hybrid Beat Saber and survival-horror port Resident Evil 4 VR as examples of the most successful uses of the hardware, with each having earned its place on the Oculus Store’s top 50 games. (Despite its exclusivity to tethered headsets, Valve’s Half-Life Alyx was also brought up as a common benchmark for both player immersion and intuitive, motion-based interactions—and an influential blueprint of how to lean into VR’s strengths.)

Cloudhead’s Unger chalks up the studio’s viability since 2019 to the Quest 2’s success. “We built Pistol Whip specifically for the Quest and it has done insanely well for us,” he said. “We weren’t hitting numbers [as a studio] that were even relevant before 2018, but when the Quest took off, it really boosted that market substantially—to the point where we could finally grow as a company.”

While Pistol Whip initially gained steam after launching for the original Quest hardware, Unger tweeted in October 2020 that the game’s sales had increased by a factor of ten in the first week of the Quest 2’s release.

“We’ve been able to launch a lab dedicated to R&D, and we’ve got a big, big triple-A title coming up,” he continued. “All of those things were funded off of the success of the Quest. As much as no one wants to really focus on it, VR’s success is gaming-hits-driven. It is carrying that part of the market.”

Owlchemy had one of VR’s earliest hits with its 2016 workplace satire Job Simulator, and it continued its success with the breakout launch of silly sandbox puzzler Vacation Simulator in 2019. “We’re actually very happy with where the industry is going and where it looks like it’s going,” Eiche said. “Right now, from our perspective, VR does look like it’s on track for mainstream adoption. We are seeing massive growth year over year.”

Job Simulator.

Credit: Owlchemy Labs

“Owlchemy Labs is able to sustain multiple games in Oculus [Store’s] top 50,” he continued. “That’s a testament. The numbers that Meta has released will tell you how much revenue is going around in there. And it’s growing.”

Despite the sales success of individual games, some observers worry that VR users are mainly sticking with one or two killer apps without bothering to expand to a wider library of titles. For instance, Milk mentioned how a significant number of Supernatural players aren’t interested in the wider world of VR at all and don’t really touch their headsets except for workouts.

The narrowness of some players’ VR libraries complicates the question of defining what “mainstream” success for VR looks like. But raw sales figures also discount the cultural import of things like Beat Saber videos with tens of millions more views than every commercial headset on the planet combined.

As far as Unger is concerned, that’s indicative of VR’s first truly mainstream sensation. “If you had to classify a VR killer app, you would have to lean into Beat Saber being that one game or program or however you want to define it,” he said. “It’s sucked the most people in and had the greatest retention and draw and has become this huge viral thing on YouTube and social media.”

What you think you know about VR

Difficulty tracking VR’s accelerated progress may have heaped more complications onto wider adoption.

Gerson listed off a slate of advancements all coming to the fore at once. Alongside typical upgrades to internal components like higher-res screens, more powerful chipsets and heat displacement solutions, bleeding-edge features like eye tracking, and foveated rendering (which produces visuals based on where a user’s eye is focused, significantly cutting down on processing load) are being rapidly integrated into the latest HMDs.

“All of those features benefit consumers and make experiences magical,” Gerson said. “We are forever moving toward that Turing test moment, and at that point, we will see a ubiquity in spatial computing, like when we went from smartphones and Blackberries to iPhones and Android devices.”

But Milk explained that progress is coupled with an inherent stumbling block: the negative experience that many users had during their first brush with consumer VR years earlier.

“We still meet people to this day who say, ‘I tried VR and it wasn’t really for me,’” Milk said. “You ask them what they tried and it was something on Google Cardboard six years ago. So when that meaningful experience they’re looking for with this technology is some low-res entertainment vehicle that isn’t compelling to a person, they’re… potentially going to write VR off as a medium.”

The low-tech Google Cardboard.

Credit: Andrew Cunningham

Unexpectedly, pushback from major publishers and developers has compounded the issue as well. VR presented a new world of design challenges to solve, from player vection (moving or turning in a direction at a specific speed to prevent nausea) to making experiences that suited the medium in the first place.

In traditional corners of the industry, Unger thinks some studios weren’t ready for this shift. “If VR was going to be a success at the level that people were hoping it would be, it would have crushed a number of [development] processes,” he said. “Entire production departments, entire skill sets at traditional 2D gaming studios worth hundreds of billions of dollars—they would have to rethink their entire process to build these things.”

The trickle-down process meant company-wide retraining from engineering to marketing, he added.

“I know the traditional 2D gaming conglomerates were thinking, ‘Oh shit, this comes with a bunch of knock-ons and consequences to my bread and butter,’” Unger said. He used EA’s sports games as an example. “How do they recontextualize that for the VR market?”

Survios has approached this bottleneck by building engineering solutions for hardware, firmware, middleware, and software in-house. “To date, there is no Unreal Engine of Immersive Technology,” Gerson said, referring to Epic’s versatile, near-universally adopted game engine. “That technology layer does not exist commercially at the moment, so we built one. That is a heavy lift. But it’s an additional layer to the development pipeline, enabling a more cost-effective, efficient, and approachable processes for immersive worlds. And that is our future.”

Owlchemy’s Eiche brought up another trend: the abundance of VR first-person shooters. “I think some of that is just the earliness of VR,” he said. “You have a design language that’s been built for 30-plus years around a very specific style of making games. Then you have to unlearn a lot of that and relearn in a new way. Sometimes people unlearn it well, and sometimes it takes them a game or two or longer.”

Even today, Unger doesn’t think designers who “get” VR are all that common.

“That skill set is really hard to come by, even though we’ve been, as an industry, doing this for nine-ish years,” he said. “Most developers just don’t have the breadth of knowledge to do it properly, so you come away with a bad experience.”

Unger specifically highlighted three all-too-common design mistakes: taking camera control away from the user, dialing in an improper speed for vection, and porting of existing games with VR functionality merely slapped on. “We honestly have a really good guidebook now for why this stuff happens,” he said.

And as for ports? “Every time you’re converting something [designed for a 2D screen] to VR, it almost universally falls down,” he said. “It’s wrongheaded to start the journey that way; you have to understand [VR’s] strengths and design for them.”

The pitfalls of a decades-old dream

Ports aside, considering best practices in VR brings up an entirely separate but related problem: Its long history in pop culture and sci-fi is saddled with baggage. That means it’s almost impossible for an average VR user to approach this tech with zero preconceived notions. McNeill thinks this is a problem that’s almost exclusive to a medium with such tangible “presence.”

To explore this, McNeill brought up virtual swords—arguably one of the longest-held VR dreams before it was available. The first players to actually try VR sword duels, though, were in for a rude awakening. Primitive implementations would easily lead users to immediately assume that VR “sucks.”

“You might get a lot of people who said that swinging a sword in VR is going to be great,” he said. “But if you just play the naive version of that, people get disappointed.”

By letting go of assumptions, though, VR design might be able to provide users with a better experience than they could have imagined. McNeill sees Beat Saber as a prime example of this. “So along comes something that you didn’t know you wanted—that no one had really envisioned—with the potential to prove itself on its own terms,” he said of Beat Saber’s rhythm-based cube-slashing experience. “Maybe this isn’t what you were imagining, but that’s because it was in your imagination. Here’s what VR can do for real. Maybe that looks very different. And we’re still figuring out what that is.”

For Unger, it comes down to a mindset adjustment before strapping on the headset. “At the end of the day, VR is a physical medium. It’s a bit like putting on your gear to go rollerblading or anything that involves a physical process before you go do it,” he said. “VR is like exercise. You just have to flip your brain to accept that. Otherwise, you’re going to have a bad experience.”

Ironlights trailer.

Similarly, McNeill recalls introducing his 2020 fighting game Ironlights to players. At the time, he described it as “Soulcalibur in VR,” a description the fighting game community didn’t agree with.

“Fighting game fans did not like that, although it seems analogous. There are two combatants, and they’re trying to hit each other,” he said. “And the way that hits are determined in fighting games is about primarily timing and positioning, with pixels and hit frames—that’s in some ways what’s happening in Ironlights.”

The nature of VR meant the similarities ended there, however. “In 3D space, if you have two swords and they’re moving at different speeds, it’s like a much more fluid problem. It doesn’t feel like trying to hit timing,” he said. “It feels like trying to read the body language of your opponent with this element of Yomi, or getting in the other player’s head and trying to know what they’re going to do. But it plays out very differently. It feels like a different experience.”

Milk took his interpretation of the medium a step further, where even approaching VR with an eye toward creating new genres puts constraints on the canvas.

“What we’re actually crafting is human experience,” he said. “How many different use cases can you imagine for human experience? That’s the palette that you’re talking about for what’s possible with virtual reality. So I think it’s sort of false to look at it now and think it’s gaming or fitness. There’ll be so many more use cases that people will design and discover.”

Endless possibilities?

Mark Zuckerberg delivers the keynote address to kick off Facebook’s F8 developer conference in San Francisco on April 12, 2016.

Credit: Michael Macor/The San Francisco Chronicle via Getty Images

At Facebook’s F8 conference keynote in 2016, CEO Mark Zuckerberg described a future where lightweight, mixed-reality eyewear would become ubiquitous, declaring it a company goal over the following 10 years to make it happen. Four years out, there isn’t a great deal of evidence that Meta is anywhere near that target.

Maybe the closest public hint is the company’s Project Aria, a chunky, black-framed experimental AR device that Reality Labs says isn’t a prototype and won’t be for sale. Aria began internal testing in 2020 to find the hardware and software needs for an eventual, official AR glasses device. To date, Meta hasn’t given any indication of where its findings may lead it, if anywhere, with only scant info available.

For tech firms and investors, Meta’s mixed reality vision may be one they competitively share before a presumed evolution to contact lenses (and if you want to get really out there, a subsequent dive into murkier, full-on cyberpunk territory like transhumanism). From McNeill’s standpoint, that’s not realistic just yet.

“If you went back five or 10 years ago and said I want high-res, low-latency VR/AR headsets that look like pair of glasses—cheap enough to produce that everyone can have pair—I don’t think any serious VR hardware developer would tell you that’s in the cards for the next decade,” he said. “But I think a lot of them would say, ‘That sounds cool, let’s get to work and start making incremental progress.’ I suspect something like that is happening here.”

Unger mentioned several obstacles with hardware, beginning with providing a wide enough field of view, which in prototype stabs like Google Glass and Microsoft’s HoloLens amounted to looking at a postage stamp-sized window in the frame.

“Most people don’t understand that technology is still so far away. You have to move all of that compute and all of the cameras and sensors somewhere,” he said. “Dealing with light waves and how you bend light into a clear transparent display are super-hard problems. It’s still a massive problem in the industry in general.”