The Big Facebook Augmented Reality Play Is Starting

Facebook is no longer willing to hang around and wait for augmented reality glasses, it's pushing the technology into its smartphone apps today with a new platform. "I used to think that glasses would be the first mainstream AR platform, and we'd get them maybe five or ten years from now," Facebook CEO Mark Zuckerberg said on stage at the opening keynote for F8 2017, the company's annual developer conference. "But over the last couple of years we've seen primitive versions of these things on our phones."

Those "primitive versions" have spawned Facebook's latest platform, and it's building on the camera that the social network has been progressively adding to all its mobile apps. "We see the beginning of a new platform. We're not using primitive tools today because we prefer primitive tools," Zuckerberg argued. "When you look around at all the different cameras that are out there today, nobody has built a platform yet ... Today we're making the camera the first augmented reality platform."

Launching in closed beta today for all developers, it'll obviously include the basics you'll find in apps like Snapchat. Face masks, for instance, and art effects. However, behind all that will be some seriously complex AR toolkits.

They boil down to three key areas. First, there's simultaneous location and mapping, or SLAM. That allows for virtual objects to be located in space – viewed through your smartphone display – and fixed there. You'll be able to move around them and have the real objects coexist with the digital ones.

Then there's the ability to precisely locate virtual objects in real space and have the digital interact with them. That could be AR sharks that circle a cereal bowl – with the app understanding depth, and so the sharks are occluded when they're behind the bowl, or visible when they're in-front – or digital flowers that bloom on a real plant.

Finally, there's object recognition. It'll mean context-specific AR effects depending on what the camera sees. From a coffee cup, for instance, it could offer virtual steam gusting up from your drink.

However it could also be more practical than entertaining. Zuckerberg suggested that the AR system might be used to pull up reviews of products – the perennial AR favorite, a bottle of wine, was his example – and then shop for them. "Some of these effects are going to be fun," he explained, "and some of them are going to be useful."

Not everything will be coming immediately. A games platform that can allow AR gaming to take place atop a real-world environment is in the pipeline for later in 2017, for instance. Zuckerberg was blunt that it's only really the start of the journey.

"It's going to take a while for this to develop," he conceded. "There's a lot that we're going to roll out over time ... [but[ over time I think this is going to be a really important technology that changes how we use our phones, and eventually all kinds of technology."

Today, there'll be nine new components from the AR Studio, integrated into the Camera Effects API. That includes face tracking, scripting, front and back cameras, hashtag voting, touch gestures, device motion, triggered audio, data integration, and 3D rendering.

MORE Facebook AR Studio