[Interview] Andrew Bosworth talks Quest Pro, future devices, competition, and the challenges of innovating in XR

L'Usine Digitale : What has the feedback been on the launch of the Quest Pro? Has it matched your expectations?

Andrew Bosworth : We’re really happy with the device and the reception has been really positive on the hardware. It delivered on the promises that we wanted it to. I think it’s the first super accessible, programmable mixed reality device and that it's going to have a broad impact.

People rightly pointed out that we still have to deliver a lot of the pieces on the software side. We announced all these partnerships with Microsoft & Accenture at Connect, and those are coming during the first semester of 2023. We talked about the mixed reality capacity and the software is still not quite there yet because the device is the first of its kind as a development platform, so it’s going to take a little time. But we’re committed, as we have been with Quest and Quest 2, to continue to improve the software experience over time.

On the topic of mixed reality, can you talk about the absence of a depth sensor on the headset, which could have helped for things like automatic plane detection? Are you favoring a software solution over hardware?

We feel like the Quest Pro is at the level where we can deliver a really strong scene graph to people, and that’s going to be strengthened with software over time. Obviously we have labs full of various arrangements of houses and furniture and we’ve done as much testing as we can, but there’s just no substitute for seeing what developers are experiencing with consumers using the software in the real world. I expect our Presence platform APIs to continue to get a lot stronger, and so I think we have the hardware we need.

The hardware can get stronger, you can always add things like depth projectors, but those come with cost and weight among other tradeoffs. They’re better in lower light, but there are also conditions where they’re worse. We’re trying to always balance what hardware package we can deliver. The good news is we really hope that is abstracted from developers. They shouldn’t have to worry about the specific capacity of the hardware, they can just query the scene graph and feel confident in the results.

Some people have reported a microphone problem on the Quest Pro. Do you have information about this? Is it something that could be fixed with a software update?

That's an issue we actually are struggling to reproduce because it's not a widespread problem. We're not questioning it, but we’ve asked the people concerned if we could get access to their device and the environment they're using it in so we can understand what's happening. We're not getting a ton of user reports about it, and if somebody does report a problem to us, we're happy to work through a customer service channels to help fix the issue. Once we get access to a device that actually let us debug the problem, I do suspect we will be able to fix it in software.

I was surprised that the enterprise software package wasn’t available at launch for the Quest Pro. Is that still coming for the end of the year?

We’re making good progress on that and we have been for a while. Things like multi-device management, the types of things you need to deploy in enterprise environments and have the company that owns the devices or is managing the devices feel confident. It’s coming. And some of it is here now, like Workrooms for collaboration.

Then there’s software we’re going to deliver later in the first half of the year, like our partnership with Microsoft, where those integrations are underway but were kind of hard to do before Quest Pro was available. So it’s a bit of a chicken and egg problem there.

Could you give me some more details about what exact software solution will come from Microsoft with regards to Windows 365?

There are different layers to it. Desktop streaming is a major use case for these VR headsets, for example with Steam for gaming via Air Link or Virtual Desktop, so we can already do streaming through that, but right now that relies on you having a physical machine with you. Over time, the goal is to do desktop streaming remotely. That’s something Microsoft really believes in and that we do as well, because we think these devices could ultimately replace laptops.

Obviously, Windows streaming is something that Microsoft has already been working on, so we’re happy to intersect with their vision there. Separately, there’s progressive web apps that represent a tremendous opportunity, because for a lot of applications that you want to have access to if you’re in productivity, you don’t need to have a very powerful machine behind it.

You’ve just announced a colocation feature for Quest Pro, does that mean you might open up towards developers working on location-based entertainment?

I’m thrilled to even hear the question, because location-based entertainment was hit so hard by the pandemic in the last several years. It was a major conversation that we would have before 2020 and it’s just starting to come back into the scene now.

We’re still very focused on the end consumer, so we’re not targeting location-based entertainment, that’s not our goal, but we’re happy to help, to be part of a package these developers are building. Like we were with the Rift for The Void, back in the day. I think LBE is a great way for people who aren’t familiar with VR to discover it and explore it at a low cost.

The economic downturn has hit the whole world pretty hard this year, and Meta was not spared. Can you confirm that these adverse conditions won’t affect XR investments in 2023?

I don’t want to say it’s not going to affect us. We’ve been pretty open, Mark’s been talking about it for the last several quarterly reports: we are definitely across the entire business working to be more efficient, to figure out how we can do more with less. You’ve seen us make some changes, for example we had to say goodbye to Portal, a beloved product not just by us but by the consumers who bought it. So we’re making the changes we need to in order to ensure we can focus on what’s most important.

At the same time, we are committed to the long term vision of augmented reality and virtual reality being next generation platforms that will succeed mobile phones and laptops, and we are pushing very hard in that direction. So the economic downturn has a big effect, and we’re constantly thinking about how to be more efficient and about what the really most critical things are that we need to go and pursue, but I do think we’re going to be able to deliver great products in spite of that.

Quest 3 has been confirmed for next year. Will you be keeping a three year hardware refresh cycle going forward?

We have a goal of keeping a certain cadence, but when you're developing hardware, you have a lot of different programs going in parallel and some of them really materialize and you're enthusiastic about what that represents, and some of them don't. We will never ship hardware that we don't believe in, so there’s always going to be a possibility that some device we thought was going to deliver ends up not being good enough. And likewise, we've also had the inverse happen where a piece of hardware materialized faster than we expected, but that's obviously a bit rarer and harder to pull off.

We have a pipeline of programs and for each there is an idea of when they could launch, but we don't really commit to that. Not just externally but internally as well, because if you get committed in this manner, it could trick you into launching a product that you aren’t proud of, and we're just not going to do that. So each product line and each product prototype follows an arc, and if it continues to look promising we’ll continue to invest and if not, we'll fall back to the next product.

It feels like competition is going to ramp up significantly in 2023. What are you doing to make sure you remain on top?

Honestly, it's taken longer for the competition to show up than we expected, relative to how successful Quest and Quest 2 were almost immediately. We've expected this for a long time, and I think it's super healthy. It's great for consumers to have more choice and it's great for developers to have more targets for their software.

There are a couple of things that I believe are important in order for us to remain competitive. One is that we want to keep pushing the envelope and delivering more value to consumers than other people can. I think if you look at mixed reality and hand tracking, these are hard-to-deliver technologies. They take a lot of investment, a lot of polish. It's easy for someone to say that they offer hand tracking, but in your role you've probably experienced bad hand tracking and, you know, they may check the box, but it's not the same as what we're delivering on the Quest 2.

Indeed.

So we do have a technical moat that's beyond just content, that we've built-up through investment. Our displays also have great uniformity correction, they have really great visibility across the entire spectrum. Our teams have done a really nice job of making sure that we're maximizing the pixels that we have, as opposed to throwing a bunch of pixels in the corner of the display where no one's going to be able to see them or take advantage of them, or where the optics just distort those pixels.

Having said that, the other thing I want to mention is we have been diligent about maintaining and growing the Quest ecosystem, about making sure developers find success and consumers have access to a great content library. Developers should certainly go find every avenue available to them to recoup the investment they've made on the content, and so they should be in the other stores. But for consumers, they're going to find that content shows up for us first because we have the biggest audience. And they’ll see that content is going to run the best on our device, and I believe they will be able to tell the difference in quality between these experiences.

I feel like PCVR streaming hasn’t been perfected yet, where there’s still no "one click solution" that works right away and painlessly for all users. I had great hopes for the dongle that D-Link released, but it’s not available in Europe currently.

PC gaming continues to be pretty important for us. If you look at the Steam release charts of who's using which headsets to experience PCVR, I think we're still by far the most common headsets that people are using, and so we do pay a lot of attention to that as a major use case for our products that drives sales and consumer satisfaction for the device.

Really, what happens here is a division of two things. One of which is if you have quality Wi-Fi, and 60% to 70% of people do, then Air Link is clearly a great success, which is the main use case. If you don't have that and you can't get access to the dongle and you don't like the wire, those are challenges we're still working through, but we're not sitting on our heels.

I can’t give you details yet, but we're working with major vendors to integrate the same technology that's in the dongle into the native integrated wireless for PCs. We're working on some big projects here to continue to extend that capability to people whose wireless network is too congested or doesn’t have a good enough signal. So we're continuing to push and advance in this area.

To talk a bit about the Metaverse aspect of things, Tim Sweeney of Epic Games recently said that VR wasn’t a consideration for Fortnite because locomotion hasn’t been nailed yet. What would you answer to that?

I love Tim Sweeney and Epic’s work. I’m a huge fan and have been a fan of his games since the early days. Obviously he's a legend in the industry. However, I do think he's a little out of date on his information as it relates to VR content. I think with games like Population: One, we've got really strong proof that you can have a great and rich first-person-shooter experience in virtual reality without the challenges he's talking about. That’s not saying there are no challenges, and I do want to put respect on him for this.

For example, in Fortnite objects are on the ground, whereas in Population: One they’re more front and center in order to reduce head movement. There are design changes needed to bring the game to VR. And for a major developer, making some of those tradeoffs may not be worthwhile to grow your audience by however many millions VR represents relative to the hundreds of millions you already have playing. It is entirely up to the creators to decide where and when to deliver the experience that they believe in. However I believe those specific concerns that he voiced probably aren't very up-to-date.

There seems to be a widespread misundertanding about how much of Meta’s Reality Labs investments go into Horizon Worlds specifically. Could you tell me how big the Horizon team is, just to put that to rest?

About half of our investment is in augmented reality, which people haven't even seen yet. And obviously virtual reality as a hardware program is the majority of the remaining investment. We don’t share specific numbers, but Horizon is a software program, and a small one at that, relative to something like Facebook or Instagram.

So to be clear, it's not a huge expenditure. We've got servers we manage so the capital expenditure is not zero, but it's relatively low. And it's a big team, but not outlandishly big. For us it is one of those tragic misunderstandings that people equate Horizon with the entirety of the Reality Labs spend, when in fact it's the tiniest fraction of all the things that people have heard about.

Speaking of augmented reality, it’s obviously a challenge to bring a compelling product to market, could you tell me what the current timeline is looking like?

That’s a great question and the answer moves in real time as we have technology breakthroughs that brings us a year or two further, and then as we have technology setbacks where it goes back a year or two. It's really a very dynamic situation. There's an old saying in technology: we get less done in one year than we expect, but we get more done in 10 years than we expect. I think AR is going to have that property. On the backside of the decade, I think we will have experienced full AR in form factor appropriate glasses. When exactly that will land, between now and the back of the decade, is the open question.

There are a lot of intermediate steps, though. Can we deliver augmented reality experiences that don't depend on two high resolution displays and binocular vision? Yes, we can probably provide some great augmented reality experiences without those things. So there will be things that land beforehand from us and from competitors that look like that. Then you'll have augmented reality, but the form factor won't quite be right. It will still be a little clunky, a little buggy. It won't be all day wearable, it might not work outdoors because it's too bright there, it might be too heavy to wear for too long, the battery life might not be that great, but we'll have it. And I think in the back of the decade we'll get into form factors that are a little bit more all-day wearable, a little bit more functional.

Do you think we’ll see a Mirror Lake-level VR product before those first AR glasses hit the market?

These things are related. In virtual reality, we're in this very tight trade space where if I want to increase the resolution of a display, the increase in compute power required to power them is quadratic. However the increases in the thermal efficiency of processors is linear. So we've got this kind of mismatch there. And then if you want to expand the field of view, you increase your weight and cost while decreasing the resolution in the center of the field of view. You’re faced with these really tough tradeoffs between ergonomics, cost, functionality and resolution.

As long as you're using a conventional panel, whether it's MicroOLED or LCD, and conventional optics, whether it's pancake or Fresnel, you're in this trade space, it's unavoidable. To break free of it, you need eye tracking, which can solve the quadratic mismatch between processing thermal efficiency and display, though it doesn’t help with optics, weight or cost. To address that, you need to change the display type entirely, go to paneled waveguides, laser scanning displays… We're working on all those things in research, and they’re more on AR-like timelines because actually a lot of it is the same research.

It's a nice future expectation that these things will come together and AR technology will make VR better and VR technology will make AR better. That is definitely a thing that we see happening, but we're not quite there yet.


Lire cet article en français : Quest Pro, Quest 3, AR, concurrence, défis technologiques… Entretien avec Andrew Bosworth, CTO de Meta

Sélectionné pour vous

HTC Vive's China President: "We expect mass market adoption to take off in 2018"

Oculus will release fewer but bigger titles in 2018, says Jason Rubin, VP of Content