The Secret To Snap’s Success

Snapchat’s AR features are known as Lenses, which first appeared in 2015 as Face Lenses. World Lenses launched in 2017 and allowed users to place 3D objects or characters into the world as seen through their Snapchat cameras. According to ARtillery Intelligence, Snap made $246 million in 2018 from branded Lenses, and the company’s stock price continues to soar as engagement and daily active user count grows. One of Snap’s latest Lens innovations is what the company calls “Landmarker technology,” which applies the same fantastical AR effects of World Lenses, but instead transforms real-world landmarks in real time.

Qi Pan, Computer Vision Engineer at Snap, explained the progression of Lenses to Landmarkers this way: “Our very first World Lenses launched in the spring of 2017 (face Lenses launched in September 2015). At first, these Lenses could simply put a character or object, which you could “place” and move around as if it was really there, into the world seen through the Snapchat camera. Since that launch, Lenses have come to transform more and more experiences. For example, our Lenses can recognize the sky in your Snapchat camera and transform it, or recognize your hands or your hair in the frame and trigger an animation. Landmarkers are a natural evolution for us, this time registered to large-scale buildings and specific locations.”

The first Landmarkers available are the Eiffel Tower, Buckingham Palace, New York’s iconic Flatiron building, LA’s TCL Chinese Theater, the US Capitol Building, and announced on the 4th of July, the Statue of Liberty. Snapchat’s AR capabilities can superimpose graphics, animations, and other effects atop these recognizable real-world locations using image recognition models built by a collection of user-generated data — GPS wouldn’t be accurate enough for overlaying augmented imagery atop geolocated landmarks. This unique approach effectively creates the digital twin of the real-world location, and the digital layer can be manipulated via augmented reality in the camera.

“To ensure our Lenses can transform the scene quickly, right within the live Snapchat camera view, we leverage the publicly submitted Our Story Snaps from our community to reconstruct the scene beforehand, from these many points of views and conditions. This allows us to understand the geometry of the building as well as leverage computer vision techniques that enable us to do fun yet complex effects such as making the Eiffel Tower dance or the Statue of Liberty wave. Applying such effects with GPS, which is only accurate to tens of meters, would just not be possible,” Pan said.

Snap connected us with NY based Lens and Landmarker creator Nico Shi(animatedreality.com). A recent graduate of New York’s Parsons School of Design (BA, Game Design, 2018), Shi created the famous Pizza Landmarker Lens for the Flatiron Building in New York, which was first unveiled at Snap’s Partner Summit earlier this year. After creating popular mobile games like Chef Umami, Shi turned her attention to Snapchat Lenses a year ago. “The bespoke, indie quality of my work is a good fit with the Snap audience.” Shi says she makes her living from her indie mobile games and Lens development for Snap marketers, but there are many people who focus exclusively on Lenses. With more and more brands working with Lens creators like Shi for paid AR content, that number is sure to increase.

Like Shi, anyone can create their own Lens or Landmarker by downloading Lens Studio, Snap’s free, public AR creation tool from Snap’s website. As of this writing, over 500,000 Lenses have been created by Snap’s Lens Studio community, and people have played with those Lenses more than 15 billion times.

A Snap spokesperson told us in an email that creator community Lenses can be found within Lens Explorer in Snapchat — just tap the smiley icon that appears underneath the Lens Carousel to access Lens Explorer. Snap also recently rolled out a new feature called Lens Creator Profiles, which offer a new way for Snapchatters to easily find all the Lens from a particular creator and subscribe to them on Snapchat, and in turn, helps creators grow their own followings on Snapchat

While the number of Landmarkers available today is but a handful for each city, Snap expects the new tools now becoming available to its Lens Creators will change that dramatically in the coming months.

“We’re primarily focused on the ways augmented reality can empower self-expression and communication. To support this on the product side, you can expect us to continue introducing new Landmarker locations around the world, but also overall continue innovating around the types of experiences that our camera can unlock. The Snapchat camera will keep getting smarter,” said Qi Pan.

“Second, we believe the success of AR is dependent on a vibrant, diverse creator community. Today, our Lens creator community is vibrant, global and growing — and we hope that in the next few years AR continues to become more and more accessible to any creative person,” Pan concluded.

Originally published at https://www.forbes.com.