VR started with gaming—but it will take over every other industry

“We are looking at sensory immersion. The ideal scenario with virtual reality is to get it as near to real as possible for you to maximise that environment.

“From the autonomous driving point of view, it’s a big thing.”

Brian Waterfield is virtual reality and high-end visualisation technical lead at Jaguar Land Rover (JLR), and he is not talking about virtual reality’s potential in the near future; this is technology he is developing and working with right now.

This might seem surprising at a time when virtual reality (VR)—and its close cousin augmented reality (AR)—are mostly thought of as gaming and entertainment technologies. Yet UK developers, not least those emerging from London’s tech incubators, are successfully pushing themselves to the forefront of industrial and business applications for VR.

With so much disruptive development taking place in such a short period of time, thanks in most part to the emergence of affordable head-mounted displays (HMDs) such as Oculus Rift and HTC Vive, industry analysts are struggling to predict how the market is likely to grow over the next few years.

MarketsandMarkets, for example, initially suggested that VR would reach $1.06 billion (£820 million) by 2018, but this estimate has since been revised; the industry was already worth $1.37 (£1.06 million) billion last year—and could hit almost $34 billion (£26 billion) by 2020 for VR alone.

TrendForce is more optimistic still, predicting a total market in 2020 for both VR and AR of some $70 billion (£54 billion), while Digi-Capital goes all-out with a forecast of $150 billion (£116 billion).

High-end gaming will continue to be the public face of VR for a while yet. But much of that predicted market share will lie in everyday business, health, and manufacturing applications, with developers overjoyed at being able to take advantage of the new wave of inexpensive and increasingly mobile VR hardware spawned by the games industry.

“We’ve moving from old school to new school,” admits Michael Kaplan, who heads up VR development at Nvidia. “Gaming is low-hanging fruit. The professional side of VR will become more important going into the future.”

JLR’s virtual cave, showing how it tweaks both car design and the manufacturing process via VR/AR simulation.

JLR’s virtual cave, showing how it tweaks both car design and the manufacturing process via VR/AR simulation.

Take it up a gear

Back at JLR, Brian Waterfield is quick to point out that the company started using a four-wall VR cave within its vehicle packaging department, which does space-assessment for new vehicles, almost 10 years ago.

“We found VR gave us a lot more scope across the business,” he says. “The high resolution and functionality of the cave allowed us to trial it in different areas of the business such as manufacturing and service. That developed and now it touches on the whole of the development process.”

In JLR’s current Virtual Innovation Centre, Waterfield added HMDs to the mix—“we’re using Vive HMDs which are more in the gaming and entertainment sector but we’re trying to enhance them to industry”—and has more recently introduced motion capture techniques not too far removed from those used by Hollywood.

“You’ve got all these little balls on the joints of your body. Optical tracking picks up their infrared reflection and sends it to software that has a skeleton. This can then work out the movement of the body in relation to muscle stresses and strains on the operator or, from the customer point of view, how the experience of driving may be affected.

“Say you’re going down the track [the manufacturing production line] and there’s an operation such as fitting a wheel. We would set up the same thing in the virtual world and record the body movements while that activity was going on. Then we can look at how to maximise that process, but also look at the safety of the operators themselves.

“It’s a study of the body of movement.”

JLR has used these techniques to optimise the customer experience too. For example, the company made changes to its vehicles’ stowage design after studying in the virtual world how luggage areas of the boot were accessed.

The VR cave now covers the whole product line: “We have designers in there, engineers from development, and the manufacturing and service guys too. We even have some of the marketing and sales people.”

Of course, tweaking the low-level manufacturing environment and processes by way of VR modelling and simulation might not be practical in every situation. Another approach is using HMDs to simply boost the efficiency of day-to-day working. Vuzix, for example, has some smart glasses that, backed by some clever software, send real-time updates to warehouse pickers. Those glasses, with the right setup, could also overlay a map that shows the most efficient route for a picker to take.

VR, AR, or “mixed reality?”

With the buzz surrounding the Oculus and Vive HMDs, it’s possible to have missed the arrival of a new headset on the block—one that has the potential of actually living up to the “game-changer” tag, especially in business.

Microsoft’s HoloLens is a hybrid device that combines full stereoscopic 360-degree virtual reality with the real-world overlay benefits of augmented reality. Its lenses are transparent, allowing the wearer to experience and freely move around the real-world and virtual-3D environments simultaneously.

For the moment, HoloLens exists only as a developer edition with a software development kit and samples.

However, London-based innovation studio Kazendi jumped at the chance when the first units were made available three months ago, and has developed a number of prototype HoloLens business applications that have already grabbed the interest of certain FTSE100 firms.

One of these, developed for a major insurance company, makes use of HoloLens in the practice of corporate insurance assessment.

“I’m assuming the role of a surveyor doing an assessment of a client’s building,” says Kazendi’s Maximilian Doelle during a live demonstration. “I might say ‘this wastepaper bin is a fire risk,’ so I place a fire-risk marker down by the bin.”

A (virtual) red flag for insurance assessment.

“Or I might place a marker here, to say that the distance between the chair and the wall represents an obstruction risk. If I return to the building a year later, I can remind myself where the fire and obstruction risks were on my last visit. I can tell if something has changed.”

Doelle switches role to that of an assessor evaluating the retail risk.

“I have the detailed premises instructions but now I’ll go into my documents, view the electrical certificate, and check the data.”

The certificate hovers in front of you in high resolution. You can read through it like a PDF, perhaps pinning it to the wall for easy reference—the real wall, that is.

“Maybe we have a discussion about the bin as a fire risk. Let me call an expert via livestream for his opinion.”

Doelle conducts a quick chat with the remote expert. The client has the ability to join in via their own HoloLens, letting them see the risk markers, assuming the position of the surveyor, to better understand the insurance company’s view of their building.

“In addition, I can take photos and livestream documents of the building to assess what I’ve just done, so I have photos as well as the track.”

A gallery of images (roughly) showing the experience when using HoloLens with Kazendi’s software.

A gallery of images (roughly) showing the experience when using HoloLens with Kazendi’s software.

A gallery of images (roughly) showing the experience when using HoloLens with Kazendi’s software.

While AR tends to conjure visions of head-up displays, video feeds, and, yes, bloody Pokémon, Microsoft refers to HoloLens as a “mixed reality” device because of the way it combines VR objects and spaces with real-world objects and spaces.

Four 3D scanners, two on each side of the unit, create a spatial map and surface text reconstruction of the real world as you walk around and move your head. This means you don’t need to hire a 3D artist draw up a model of the real-world environment before adding virtual objects to it; you could walk into any room for the first time and let HoloLens generate the 3D model for you.

Doelle demonstrates this by walking around a room for a minute before adding 3D office furniture to it. He picks up a real chair and slides it under a virtual desk to make sure it fits.

At another point, he picks up a holographic object and puts it under the real-world boardroom table that sits in the middle of the room. Even to other HoloLens wearers—the platform is collaborative—the object is now no longer visible unless they physically bend down and peek under the table.

The full HoloLens Development Edition kit.

Credit: Microsoft

HoloLens is also unlike other HMDs; it’s non-tethered and, since it contains its own processor, it does not require an external computer, tablet, or smartphone to operate at all—just a Wi-Fi connection.

“HoloLens is nothing like Google Glass,” Doelle emphasises. “It’s a completely different thing. It’s like comparing a bicycle to a sweatshirt.”

VR in the theatre of war

Fully-immersive VR is how most people recognise the technology, typically as part of a first-person 3D game. While wearing an Oculus or Vive HMD, or even a simple smartphone-powered GearVR, the player enters a completely different environment conjured up entirely in software.

This concept has also been used for a host of practical, non-gaming applications over the last couple of years, from curing phobias to working with autistic children. It is also gaining traction among developers who work with top-end boutique retailers to provide beautiful and realistic virtual stores for customers to walk around remotely.

One untypical application, developed by Cambridge-based Plextek, puts collaborative participants into a very realistic, perilous, and thoroughly nasty theatre of war. Its simulation has been designed to train British soldiers in high-level procedures when they’re forced to deal with a serious injury on the battlefield.

Initially created for the UK Ministry of Defence over just a few months, the current build dumps a squad of up to eight trainees into a village in a war zone of roughly three square miles.

“There is an explosion in the distance,” explains Plextek’s head of medical, Collette Johnson. “The squadron must negotiate how to approach the site.

“On their arrival, they find a victim whose foot has been blown off. The team must determine how to triage the patient according to medical procedure—choose a medical kit, apply a tourniquet, deal with complications, and call a helicopter to evacuate.”

The scenario can be spiced up with adjustments such as unexpected gunfire interrupting treatment, smoke restricting visibility, or sudden complications suffered by the patient.

This is a significant improvement from conventional triage training, which might involve playacting in a meeting room with a dummy. With VR, participants do not even need to be in the same physical location to take part in the scenario, and it is possible to play a recording back afterwards to evaluate the team’s performance, and obtain accurate feedback concerning the patient outcome.

Plextek kept development time to a minimum by starting with off-the-shelf software—the Unity 3D gaming engine and a map of Afghanistan—before adding custom artwork to the template. The project has picked Vive HMDs over Oculus, due to the better freedom of movement it supports.

“We are hoping to develop it further with 3D sound,” says Johnson, “adding gunfire and other stress-inducing audio distractions.”

She also thinks the Vive will open doors to other real-world applications outside gaming, such as disaster relief.

And if the data can’t keep up?

The issue of mobility is likely to become increasingly important as the VR hardware market matures. “Mobility is key,” emphasises Nvidia’s Michael Kaplan. “People are screaming for portability.”

This demand has favoured entry-level VR products based on smartphones, such as Google Glass and Gear VR, and it will likely benefit untethered HMDs like Microsoft HoloLens once people get the chance to compare them with tethered Oculuses and Vives.