Tech —

Facebook wants you to stare even more at the real world through your phone camera

"Augmented reality" platform launches in closed beta today.

Facebook founder Mark Zuckerberg kicked off his company's annual "F8" conference on Tuesday with a stark mission statement: People don't look at Facebook on their phones enough, and he has plans to change that.

The plan revolves around adding "augmented reality" (AR) features to Facebook's smartphone apps using our existing cameras. Starting today, basic features will be added to the Facebook app in a "closed beta" that makes more content appear when pointing a phone's camera at the real world.

"I used to think that glasses were going to be the first mainstream augmented-reality platform," Zuckerberg told the F8 crowd. "Maybe we'd get them five to 10 years from now, in the form factor we want." He continued by pointing out how Snapchat filters and Pokémon Go's catch-'em-all moments employ basic AR functionality already, and Facebook's augmented-reality embrace of cameras will follow similar suit.

Most of the examples were high-tech expansions of Snapchat-like live filters, bolstered with a real-time 3D mapping technique that Facebook calls "simultaneous localization and mapping" (SLAM). When working as advertised, Facebook's app will quickly translate flat planes and geometric objects captured by a smartphone's camera, then recognize object types (plants, coffee mugs, bowls, wine bottles) and offer playful, "relevant" 3D filters. 3D text can emerge from a flat surface; sharks can swim around a table; steam effects can appear above your coffee.

Another demo showed an existing 2D photo that Facebook's app supposedly translated into a 3D space, which it demonstrated by panning the photo to the left and right in 3D and by volumetrically filling the space with water and Skittles ("because the future is delicious," Zuckerberg said to mild laughter).

Zuckerberg also laid out a less clear plan for having visual elements appear based on a mix of GPS and image-recognition data. In Zuckerberg's vision, when you hold your smartphone up in the real world with Facebook's camera on, objects that your friends intentionally placed will appear. These include carvings on a wooden bar table, recommendations on a restaurant's menu, and notes on your home fridge. (In one case, Zuckerberg said these tags could eventually include links for content viewers to buy related stuff online.)

One creepy example of this in action was a group of Facebook staffers all holding their smartphones up to look at a blank wall at a Facebook office. These beta users could see a giant virtual art installation that covered the walls with splatters of digital paint and a massive, Facebook-blue waterfall pouring over everything. But as Zuckerberg pointed out himself, the group just looked like they were staring at a blank wall.

As a category, video games were brushed over, with a highly mocked up video showing a father holding his smartphone camera over a coffee table, where a bunch of robotic aliens and monsters appeared. The dad's kids flanked him and waved their hands over the table to swat aliens away on his screen. Zuckerberg indicated that work in this category was ongoing at Facebook and that we'd hear more about Facebook AR games at a later date.

People may not be as eager as Zuckerberg to let Facebook slap reminders and notes onto their actual kitchen walls.

Zuckerberg warned that the closed beta of these augmented-reality camera features was incredibly limited. "Your experience isn't going to change dramatically overnight," he said. "But over time, I believe this will be a technology that changes how we use our phones and eventually all of technology." He described the platform as "open" and invited developers to help Facebook create new filters and camera-driven, real-world image content for Facebook's app.

What Zuckerberg didn't talk about, however, was the fact that his new platform appears to require a ton of real-world mapping and capture data, especially to tag real-world locations with user-defined items like notes and graffiti. Facebook may be entering a hornet's nest of privacy concerns as a result. Other camera-powered apps allow people to take a lot of private photos, but they have not dipped their toes in this concept of precise, granular tagging in the real world, and people may not be as eager as Zuckerberg to let Facebook slap reminders and notes onto their actual kitchen walls.

Also revealed during the keynote was Facebook Spaces, the VR app demoed at last year's Oculus Connect conference. It is currently live and free for Oculus Touch users in beta form. We're testing that right now and will return with impressions later today.

Listing image by Facebook

Channel Ars Technica