While Facebook and Microsoft believe future interactions, and transactions, will take place in the metaverse — an entirely digital world where people exist as avatars in virtual reality (VR) — Niantic founder John Hanke has been pursuing a “real-world metaverse”, using augmented reality (AR) to encourage people to explore their actual surroundings.

Spun out of Google in 2015, where Hanke had led the Google Earth mapping project, Niantic demonstrated the potential for AR technology by launching the worldwide hit game, Pokémon Go, in collaboration with The Pokémon Company and Nintendo.

Now, Niantic is creating an AR map of the world by collecting data and scans from millions of users. Here, Hanke tells FT technology reporter Cristina Criddle why he wants to enhance the real world, rather than create a synthetic one.

Cristina Criddle: You talk a lot about building the real-world metaverse — are you still focusing on that, and can you explain what that looks like?

John Hanke: Yes . . . I think that the transition to spatial computing is one of those big, fundamental changes. It’s like the internet, or mobile smartphones . . . we saw companies come in with products and fail a few times before these things became what they are today: fundamentally part of our lives. I think AR and spatial computing are on that same trajectory.

We’re focused on the same things we were before: building these games, these consumer experiences, and building this AR platform — which, at its core, is this AR map, which we launched in May.

We’ve now mapped over 100,000 locations. We call it the most dynamic map of the world, because it’s constantly being fed by users. So, even as we’re speaking, new scans of the world are coming in and new scans are being aggregated and processed into these maps that localise with your smartphone but, in the future, with AR glasses.

So that’s now an organic, living, breathing system. It is alive, and the data’s coming in and getting processed and people are using it, and we now have thousands of developers outside of Niantic building apps on our platform, which is very exciting.

CC: In terms of mapping, what ultimately are you hoping to do, once you’ve captured every bit of the world?

JH: We want to make it alive with information and services. For example, I’m guessing you’ve gone into a venue, or a store, or a transportation hub and used a QR code posted somewhere to do something with your phone: you’ve scanned a code, and got a train schedule or an app . . . It’s kind of magical because it feels effortless, like you just held up your phone and clicked and you’re getting something useful. The goal of the map is to enable that kind of experience everywhere without the need for QR codes.

When the map exists, when you hold up your phone or you look at something with your glasses, that allows the infrastructure to know where your focus is and, if there’s information or a service or even an app that would be relevant to that out in the real world, [you] can get it.

I think it would be really magical. It’s going to feel like the world is this living surface and there’s just stuff that will help you appreciate it more, or navigate it more easily, or do some transaction that you want to do, just more seamlessly.

That’s the promise. There’s a way to go but I think the technology is going to allow it to happen.

CC: I guess, as the technology becomes more advanced, these experiences become more immersive and more realistic. From a philosophical perspective, what does that do to our concept of what reality is, and what the world we live in is?

JH: I would say that there’s a fundamental difference between the goal of virtual reality (VR) and augmented reality (AR). There are companies out there that would blend it all together into an alphabet soup of acronyms and call it “XR” and say XR means VR and AR. I actually think they’re very different and they have very different objectives.

VR really wants to create that other world for you: a virtual reality, a synthetic creation that we can escape to: a beautiful beach; or another planet; or no earth; or wherever you might want to go.

That’s not the goal of AR. The goal of AR is to enhance reality. And that’s not necessarily replacing it with visual imagery. There might be very modest visual cues . . . so, if you want more information about when the next train’s going to arrive, there’s a button that you could virtually push to get your ticket for the train that’s going to leave in 15 minutes. But it’s not, in its heart, meant to overwhelm you with visuals.

We can selectively add things to the world, but it’s more like curating the furniture in the living room. It’s selectively adding things which maybe enrich the world a little bit. Like a Pokémon that you’re chasing — but it’s not meant to put you in the Pokémon universe so that you are no longer in reality. That would be VR, and that’s a different story — maybe good for escapism, but not as important in terms of the future of technology.

CC: How do you spend your spare time?

JH: I’m a parent, I have three kids, two of them are now on their way out of the house, but I still have one at home, so my spare time, for the past 20 years, has really revolved around doing stuff with the family and with the kids. We try to leave the house as much as possible and get out and enjoy . . . We’re fortunate to live here in northern California, there’s so much to do. Beaches, mountains, parks.

My goal — and my struggle — has been prying my kids away from their games, their iPad, their whatever, the thing of the month, to get them out and enjoy some of the stuff that’s out there. I’m constantly trying to get the kids out to do that — on bikes, or hiking, or surfing, or snowboarding, or anything, just taking a walk. And, usually, 95 per cent of the time, when you’ve broken them away and they get over the feeling that you dragged them out to do something that they didn’t really want to do, they’re really enjoying it.

That was part of the inspiration for starting Niantic: trying to turn the world into a game board; trying to take some of those qualities of games that are so really enthralling, maybe I won’t use the word addictive, but certainly captivating kids. [We] take some of that, and marry it up with an activity that’s about being out in the world and being active, to try and get the best of both.

CC: But, if you’re going to immerse yourself in the real world, and immerse yourself in nature, to then add a tech layer to it can feel a bit disjointed?

JH: It’s a challenge. You see us working with that with Pikmin Bloom, for example, and the way that game is designed. There are two characters, I don’t know if you’ve seen it . . . 

CC: I’ve been playing it.

JH: Oh, good. So you know there are characters and you can see them and collect new ones, but part of the game is that you go out walking and that is what incubates your new crop of Pikmin.

But, when you’re out walking, you don’t need to look at the screen at all. Once you set up the game, it just records those steps in the background — maybe you walk, and you stop in the park, and you sit down at a bench — until you get back home, [when] you can open up the app and see what you’ve accomplished through your steps.

So it’s a blend of having the incentive of the game but not having it intrude on the time when it would be more enjoyable to just look at the trees and flowers, or talk to the friends you’re on the walk with. We’re trying to find that balance between technology that’s motivating — that allure to get you off your couch and break the inertia — but not be too much in the way.

I think technology’s helpful to us. I can’t imagine the idea of taking a trip without my phone, to get maps and directions and all of that. But, at the same time when I travel with my kids, and they have a device, the last thing I want to do is go through [their] social media feed while we’re in a new place.

So these tools are powerful, but they can also get in our way. It’s something we have to deal with as individuals, but I think companies can also be more mindful and more thoughtful about how they design products — and have technology be helpful but not intrusive and not take over our lives in a negative way. It’s the challenge our generation has to come to terms with.

CC: We are now exploring new kinds of hardware, where you can experience AR but also VR, and they haven’t quite taken off. So how do you think our relationship with devices will change?

JH: I would challenge this idea that AR hasn’t taken off. We don’t have AR glasses yet . . . but smartwatches are now almost a requirement.

I remember when smartwatches launched in 2012, 2013: “Who needs a smartwatch? This is useless technology!” Over time, though, the technology’s improved, the battery life’s better, the applications are smarter [in] health and fitness and safety, and that’s a kind of augmented reality device that’s not a phone.

You can load maps on to it, and go out and explore the world and still have some of the benefits of tech.

It’s the same [with] audio. It’s almost unusual to see somebody on public transportation without pods in their ears, from Apple or Samsung or somebody else. That’s a kind of augmented reality, too. Often, it’s a podcast or music, but there are applications that can take you on virtual tours and tell you about places. And you can talk to your virtual assistant and make phone calls or take notes.

So AR is creeping up on us in the form of these wearables. And, eventually, I think it’ll get to our eyes as well. That’s the form [of AR] that we often think of first but, when it comes to AR, that’s the hardest. It’s the most technically challenging and it’s not quite there. But, if you look at what’s happening, you can see the groundswell that’s taking us in that direction.

CC: When you’re talking about some of these other companies — Apple, Google, Facebook/Meta — how do you disrupt the powers that currently control these technologies? Or do you even have to?

JH: I would say that you disrupt by building and shipping. And I think we, as a company, are very well-positioned to do something that’s unique here, and hard for other people to do. The benefit of us, as a midsized company, is we’re only doing one thing: we’re doing AR and we’re extremely focused on it. We’re all in. That is our life. We get up, we breath it all day, we dream it at night, we get up and do it all over again the next day.

That’s not as true in large companies. They’re doing a multitude, they’re almost in everything these days. In fact, almost a week doesn’t go by when they’re not entering some new market.

I worked inside one of those companies for a decade, and I know how distracting that can be. There’s a number-one initiative this quarter, and there’s a team that gets all the attention for a while. But, then, it’s another thing, and then there’s something else that catches attention, and then the big thing a year ago isn’t so shiny and new anymore and people kind of drift away and it loses its lustre.

That’s the challenge for big companies: to stay focused over time and not get distracted. So they’ve got their challenges; we’ve got ours. We’re the David in the battle, not the Goliath. But I like our chances.

I think the combination of building this community of users and games and building the underlying map really goes hand in glove.

CC: I guess this is all dependent on what type of phone people have, and also the connectivity. There are limitations at the moment on how far this technology can go, especially in terms of where you physically are.

JH: Yes and no. Three years ago, you kind of needed the latest phone to do AR. Now, the replacement cycle of phones is such that the vast majority of phones that exist in the world can do AR. So that’s not really blocking things. The way we do AR, we don’t require a lidar sensor. Some of the high-end iPhones have lidar. It’s helpful, you can do some things with it, but the technology I was mentioning earlier, that uses a monocular image, doesn’t need the lidar sensor. So that’s not an obstacle.

And those devices just continue to get better every six months. So I think, ultimately, there is an upper bound on AR on phones from the standpoint of the user experience of just needing to hold your phone up and look through the screen into the world. That’s not a super-ergonomic activity. There’s this phenomenon that got its name in early VR days: gorilla arm [fatigue from holding a device in the vertical plane]. You’re just not strong enough to do that. It gets tiring after a while.

So the future of AR will be when the glasses get here all the time — and it’s not something that you have to choose to do and stop doing.

CC: Your games rely quite heavily on IP. Why do you think that this IP is key to gaming and AR gaming?

JH: We do some original IP. Our first game, Ingress, was original. We have a game coming out now that’s in soft launch, called Peridot, which is a character-based game where you raise these creatures that exist in AR. That’s our own IP. Then, of course, we have a Marvel game, and an NBA game, and Pokémon.

There was a great book that came out a few years ago that nailed it: called Hit Makers. It was about popular music and it was talking about what makes a hit song. The writer’s fundamental thesis was it’s something familiar and something new blended together. His takeaway was that most pop songs that break out are building on something that you already know — the rhythm, the melody, might be lyrics — but then there’s obviously something that’s novel that catches our attention.

And the two things together make it easy for you to embrace the innovation, because it’s not totally foreign to you. It’s like, OK, I’ve heard something kind of like this before but there’s something new about it that I really like. So, with AR, a familiar character, something you already know and love, is a great way to introduce people to new experiences.

Like Pokémon Go . . . you’re going out in the world and running around and doing all this stuff that you never did with a video game before, but it’s to go chase Pokémon which is a fantasy that, in many cases, people are already in love with. So it helps.

CC: Do you have a favourite Pokémon?

JH: My buddy Pokémon has been Tentacool for a long time. And it’s a little controversial because I always thought Tentacool was a squid. I’m hugely fascinated by octopi and squid, because they’re so smart and they have this colour morphing thing and brains that scientists are totally fascinated by. But other people had said, no, no, no, you’re wrong, Tentacool is actually more of a jellyfish. So that would be a little disappointing, because I don’t think jellyfish are quite so interesting.

CC: One final question: we’ve talked about AR as a utility, while you’re out in the real world — using AR to complement your life, whether that’s looking up bus times through AR, or the menu at a restaurant. How far off are we from that?

JH: I think we’re already beginning to experience that in the form of QR codes today. QR codes have been around for a long time, and were popular in Asia, came to the west, but nobody really used them, then they became more of a thing during Covid when a lot of restaurants and places didn’t want to pass out menus, so QR codes sort of blossomed again.

Now, it’s something I think most consumers have learned how to do . . . I think we’ve learned that looking at things with a device can be the start of an interaction, whether that’s just learning about the thing or beginning to do a transaction, or whatever.

I think with the wearables and the watches we’ll be doing it a lot, and we won’t realise that we’ve transitioned to that. Certainly, an inflection point will be glasses, but there are many incarnations of that. You could have glasses that help initiate that experience but where the information’s coming to you through audio, for example. That’s something that we’ve experimented with and others probably will experiment, as well.

Watches, and earpods, and AR, computer vision, working together — I think we’re going to get to that future . . . the whole idea is it just becomes seamless, the friction melts away, and we can do more with less effort. If designed well, it’s less intrusive for us, it’s less distracting for us.

Maybe we get our left hand back, because we don’t have to hold the phone all the time. We can actually have two hands to carry our groceries or keep our kids from running out into the street, or whatever it is we’re trying to do. So that would be a huge win.

CC: Well, we’re not sure when those glasses are going to come. They’re still pending.

JH: It will happen.

CC: When?

JH: I remember doing mobile maps on the generation of smartphones before the iPhone — [when] they were just starting to add GPS to mobile phones. And it didn’t work very well. There were several that came to market and flopped. [Then] Apple did their thing and got it right, and then Android came fast on their heels. AR is going to be like that. There are going to be devices that come to market, [and] don’t sell very well.

Magic Leap was kind of v1 of that, and they’re back with another version of their glasses with a little more of a humble story around it. But there will be others that don’t work.

Then, inevitably, given the history of technology, it will be good enough; it will click; and it will happen. So three, five years, I think that kind of timeframe.