Making the virtual a reality: Qualcomm’s Brian Vogelsang on the tech behind VR headsets

Whether it’s the Oculus VR headset range or Microsoft’s HoloLens mixed reality devices, most people are aware of AR and VR technology being used in both consumer products and within the logistics and manufacturing sectors.

Known for its pioneering work in mobile technologies, Qualcomm has been at the heart of bringing these devices to market with its innovative chipsets. And as the technology rapidly advances, the company is finding more ways than ever to usher AR and VR into the mainstream.

PCR caught up with Brian Vogelsang, Qualcomm’s senior director of XR strategy, to find out more about the various use cases for VR and AR headsets, and how 5G, AI and smartphones are going to play vital roles in ensuring this sector continues to grow and meet the demands of consumers and enterprise users alike.

Tell us a bit about your work background and current role at Qualcomm.

I’ve been at the company for 25 years in a number of different roles. Most recently I was doing industrial IoT. About a year and a half ago I moved into Qualcomm’s business group. XR is extended reality – which is an overarching term for augmented, virtual and mixed reality.

Our primary business in this group is selling chipsets, so the processors that go inside XR devices – primarily virtual and augmented reality headsets.

I work on strategy and partnerships for the XR business. I do a lot of work in enterprise, but I also work a lot with network operators, which are becoming more and more interested in XR as a technology with the emergence of 5G networks.

With the capabilities that 5G networks bring, XR is going to be an important technology that plays a role in the evolution of 5G.

Would you say the technology has been waiting for 5G before it could really achieve its full potential?

The technology has been around for some time. We saw devices like HoloLens and Google Glass come into the market quite early. We also saw Oculus bring virtual reality into the market.

5G will definitely be a catalyst. There’s an intersection between what’s happened with 5G, AI and with XR that is accelerating the pace of innovation in XR in particular. 5G is very important in XR’s future. XR has been growing over the past five to seven years independent of that, but this will be an accelerant.

What are the benefits of XR in manufacturing and in the warehouse?

There are two areas of XR that are applicable to this. The augmented reality side and virtual reality side. If we take a look at AR, one of the use cases is guided work instructions. In the manufacturing environment you want to be hands free. You want to use your hands for moving goods or working with machines. Wearable technology allows you to wear a computer that can feed you information about the work you need to be doing. It can keep track of the quality levels of the work you’re doing and it can improve your safety. So we’re seeing the ability to use the augmented reality glasses as a way to augment the worker and improve the outcomes of the business and the safety of employees.

There are two different kinds of these augmented glasses. There is a smart glass and then there are full augmented reality glasses. The smart glass is a monocular display, so it typically puts a screen in front of your face, almost like a 7-inch tablet floating in front of you. You can have guided work instructions – a video showing you step by step instructions. You can use your voice to tell the device to play instructions, or the glasses could be used as a checklist giving you steps to complete. That allows users to monitor data coming from those wearables so you can see where workers are in the process, or if safety checklists are being followed correctly.

Another area for augmented reality is if you have instrumented machines. You will be able to see the data coming off those machines in your field of view. You can simply look at a machine and get the information syndicated to the glasses in real time about the operation of it, or you can monitor systems and real time.

How can this tech can be used in retail?

There’s the consumer and the B2B side. A good example of the B2B side is the work we did with Accenture and Kelloggs, where we looked at the placement and assortment of products in a retail environment.

We built a virtual store which we put people into, letting them walk through that store and look at the products on the shelves, as if they were in a real store. The thing that we looked to do, which was unique to VR, was that we put them in a store where we could change the product placements in real time. The other thing we could do was track the gaze of the eyes using Qualcomm’s eye-tracking technology. So we could see exactly where a person’s gaze is and use that information to better understand their intent. With this information, you can understand what packaging is working better and what’s going to drive the best outcome.

Another good use case of XR is inventory in a retail environment. There’s things like stocking the shelves in stores, which you can use augmented reality to assist with. There’s also order picking. A lot of retail environments these days are doing home delivery, so that requires someone to go through the store and put the order together. So if you have the checklist and something to tell you exactly where to go, you can pick that order much more efficiently. AR can show you where to grab a product so there’s less room for error, this is used in logistics and manufacturing settings as well. Smart glasses can give you a display, whereas AR glasses can overlay on physical things. Basically, AR is just a more sophisticated level of the smart glasses.

Qualcomm builds the chips that go into these devices and one recent product that has launched with our chipset is the HoloLens 2 from Microsoft. It is one of the most sophisticated wearable technologies in the enterprise market today. It’s being used in a number of different verticals. That one is really exciting.

What are some of the latest advancements in VR gaming?

Virtual reality has been really popular in the gaming space. We have headsets that tether to the PC as well as ones like the new Oculus Go, which is a standalone headset. It’s based upon our Snapdragon 821 processor. It’s an all-in- one, so you can immerse yourself in a game or app straight from the headset.

Oculus has also announced a new product called the Quest. It’s coming out in the Spring and it will have more sophisticated graphics capabilities. It’s using a higher-end Snapdragon processor so it’s got more compute. It also has what’s called six degrees of freedom headtracking, so it uses cameras to understand the position and orientation of your head. This makes it more immersive, letting you move around more inside the game. It also has controllers with six degrees of freedom movement, so now you can bring your hands into virtual reality. Oculus has announced 50 games that will be available with that headset when it launches, so we’re really excited about that.

So we’ve got the HoloLens on the augmented reality side and the Oculus Quest of the VR side, both using Qualcomm’s Snapdragon chipsets.

While VR is doing well in gaming, AR is also making its way to that sector with products like the Magic Leap. There’s also a device from a company called Nreal, which I think will be really important for gaming and entertainment. It’s really lightweight, about 85 grams. It has a wide field of view – 52 degrees, which makes it feel more immersive, and it plugs directly into the smartphone. The reason it’s so light weight is because it has no battery. It connects to a Snapdragon smartphone, which can power the AR experience. I think we will see more games and entertainment come from this kind of architecture.

This is also a tie in to 5G. We announced a compatibility programme between viewers and smartphones, so we’ll start to see more of those kinds of tethered devices come out in the near future. That will allow mobile operators
to bring AR and VR into their portfolio. So now you can have a 5G experience because your headset connects to a 5G smartphone.

How is XR being used in mobile devices and how advanced can we expect smartphones to become in the next five years?

One way to use AR with a smartphone is looking through the screen and augmenting with the camera. That tech has been used for the past five years or so. It’s been used in enterprises as well as consumers. There have been many games and apps that have launched that use AR looking through the smartphone screen. It’s somewhat limited in terms of the immersion. By taking all that and putting it into a device like Nreal’s glasses, the augmentation starts to feel more real.
In the next five years, we’re going to see the smartphone become a key part in using AR and VR headsets, especially those that tether to devices and use 5G connectivity.

One of the challenges with this is that you try to make the headsets as light as possible so you can’t put a lot of battery or technology inside the glasses that might enable a really rich gaming experience, so the graphics performance is good, but if you have a PC or server in the cloud, you could then offload some of that process. We call that split rendering. So some of the processing happens within the phone and some happens on the operator’s network, meaning you’re left with richer graphics. That’s an important element that 5G will bring to those kinds of glasses.

What other use cases are there for XR technology?

One application is field service. There are currently a few issues in field service, one is that the worker may not have the expertise to complete the task, so they either have to go to a manual or some instructions, or they have to make a video call to a remote expert. Now, this remote expert can use the camera in the worker’s smart glasses to see what they are seeing and guide them through a process. They could even annotate over the glass. Let’s say I’m 30 metres up on a wind turbine and I need some help. I can call a remote expert and have them see exactly what I’m looking at, and on the visual display they can draw on it and say “this is what you need to look at”. So now you can have a less skilled worker in the field, with experts working remotely when needed.

As we get to devices like the HoloLens, we will be able to implement more sophisticated applications. A remote expert could actually draw a circle around a physical object in front of someone they are helping. This offers a really easy way to guide someone through a process. That’s a really important use case.

We’re really excited to be at the centre of all of this. We provide the chipsets but also software that enables more sophisticated immersion.

So what is Qualcomm’s role in the XR ecosystem?

Our role is to supply the technology to the manufacturers but we’re also working with others in the ecosystem to enable these kinds of capabilities. We are a technology leader and we’re doing a lot of research and development in this space. We’re bringing new technologies to market, such as the split rendering I mentioned. We also work with other software and service partners.

There’s a Finnish company called Uros, a systems integrator, and they are working with a customer of ours called RealWear, which has a ruggedised computer that you wear on your head. It’s called the HMT-1 and it uses the Snapdragon XR1 processor. It has a camera and a display. Uros works a lot in the oil and gas industries, so workers need a product that is very ruggedised. The HMT-1 is intrinsically safe, so it can be used in volatile environments. RealWare worked with Uros on a digitisation project in Kazakhstan, where they will deploy 10,000 of these headworn products. It’s voice controlled and typically mounted to a hardhat. It’s basically a tablet computer that can float in your field of view. In an industry like oil and gas, safety is really important, so these glasses will help them accomplish their job safely.

What does the company have planned for the rest of the year and beyond?

We will see 5G becoming more involved in the XR experience, so we’ll be driving more 5G integration. We’re also doing a lot to enable AI in our chipsets. We have dedicated elements in the chips to allow you to run AI workloads much more efficiently.

AI is really important for augmented reality. If you want to track your hands so you can physically grab a virtual object, the hand tracking technology requires heavy AI, so Qualcomm will continue to advance these capabilities in our chipsets.

—

Like this content? Sign up for the free PCR Daily Digest email service to get the latest tech news straight to your inbox. You can also follow PCR on Twitter and Facebook.