Practical applications of Artificial Intelligence at a startup: How AI can improve your product and operations (Part 1 of 3)

It’s winter 2022, and the world is swept up in an artificial intelligence (AI) revolution. ChatGPT, powered by GPT-3.5, launches and reaches 100 million active users by the new year?—?becoming the fastest-growing consumer application in history, surpassing even TikTok and Instagram.

What makes this moment different? Users experience real, tangible value. The product delivers such high-quality results that it quickly becomes a daily necessity. Simply put: it’s useful.

But AI offers far more than just high-quality answers to our personal and professional questions. It empowers small, ambitious businesses to achieve what once required significant financial and technical resources. To illustrate this, I want to show how AI plays a key role in Strollhunt’s product and operations. For context, Strollhunt is a no-code platform for creating and publishing location-based Augmented Reality (AR) experiences. It also offers iPhone and Android apps that let users discover and play these experiences. Think of it as ‘Pokémon GO as a Service’ for destination marketing organizations, cultural institutions, and travel companies. Currently, Strollhunt is available in 80 cities worldwide and continues to expand.

This three-part article series explores how Strollhunt leverages AI. In Parts 1 and 2, we’ll examine two core applications: a “native AI” implementation (Part 1) and an “augmented AI” approach (Part 2). Finally, in Part 3, we’ll suggest two simple AI projects your company can build by the end of the day.

Native AI: Strollhunt’s Computer Vision-powered ‘Magic?Camera’

Since our launch in 2021, AI has been central to Strollhunt, enabling us to craft the experience we envisioned for our users.

How Strollhunt works

Users embark on a stroll through the city, searching for a hidden gem or a remarkable point of interest (POI), which is revealed by solving a clue. Upon arriving at the mystery POI, they tap the ‘ANSWER’ button, triggering one of three game mechanics: the Magic Camera, Magic Radar, or a Q&A challenge. Successfully completing the challenge rewards users with points, a Story about the POI, a shareable Digital Postcard, and Nearby Recommendations (such as parks, museums, or shops).

Bringing the ‘Magic Camera’ to?Life

Our goal was to develop a game mechanic that allowed users to photograph a POI to solve a challenge?—?effectively transforming their phone camera into a ‘Magic Camera.’ This idea stemmed from a simple observation: when people encounter something remarkable or quirky in the real world, they often take a picture to capture the moment. The next logical step was to share the story behind the POI they discovered.

We believed image recognition could make this possible?—?much like how Shazam identifies songs from audio samples or how plant identification apps recognize different species. The challenge was finding the right image recognition solution to power this feature/game mechanic.

Essentially, we needed a way to compare a user’s photo with the correct answer (i.e., images stored in our Content Management System) to determine if they matched. Computer Vision provided the solution.

The user flow diagram below represents the user experience we wanted to create.

What is Computer?Vision?

Computer Vision enables computers to ‘see’ and interpret images (and videos) much like humans, allowing them to analyze visual data and make decisions. This is achieved by training AI models to recognize patterns, objects, and scenes within images.

Key criteria for our?solution

We needed an image recognition system that was:

?Fast?—?Blazing-fast image analysis

? User-friendly?—?Accessible for non-technical staff to interact with or train

? Cost-effective?—?Scalable, ideally free or with minimal cost per request

? Consistent?—?Reliable across devices, ensuring similar results on both iPhones and Android devices, not just the latest models

How we built?it

1?? Explored different image similarity comparison methods, including:

• Pixel-based
• Feature-based
• Descriptor-based
• Hashing-based
• Semantic-based

2?? Developed Proofs of Concept (PoCs) to test these methods in real-world scenarios.

3?? Benchmarked and compared methods by testing image similarity results across five cities, using 10 landmarks per city. Two tests were conducted on-site, while two were simulated using online images from Google Maps, Google Search, Flickr, and Instagram.

4?? Chose the feature-based method because it:

• Performs well with clear objects and structures
• Is robust to changes in viewpoint
• Handles partial occlusions effectively

5?? Designed the UI/UX for the ‘Magic Camera’ feature/game mechanic.

6?? Integrated the selected solution into the app.

7?? Fine-tuned the matching accuracy by A/B testing different distance thresholds to determine how closely a user’s photo must match the POI images.

8?? Launched the app! ??

And here’s how it looks in the app.

Challenges and continuous improvement

While we’re happy with the results of the ‘Magic Camera,’ it’s not infallible. Occasionally, it produces false negatives (incorrect rejections) and false positives (incorrect acceptances), which can disrupt the user experience.

Since Computer Vision-powered image similarity comparison is not Strollhunt’s core product, we allocate only a limited amount of resources to refining this feature/game mechanic. However, our experience highlights the current state of image similarity comparison and its inherent challenges.

To minimize false positives and false negatives, we implemented two key improvements:

1?? Introducing a Buffer Zone for image?matching

Initially, the system operated in a binary way:

? If the image similarity was within the threshold, it was considered correct.

? If it was outside the threshold, it was considered incorrect.

Since finding a universal threshold is impossible?—?it varies slightly from landmark to landmark?—?we introduced a buffer zone. If a photo is very close but not quite right, users are prompted to retake the picture with guidance rather than losing points immediately.

2?? Geofencing to improve?accuracy

We discovered that multiple POIs could depict the same subject. For example, in London alone, there are at least three outdoor full-size statues of Winston Churchill?—?along with many commemorative sculptures and busts.

To solve this, we added a location check before performing the image similarity test. This ensures users are physically near the correct POI before their photo is analyzed, significantly reducing both false positives and false negatives.

I’ll leave you with a video of users engaging with the Magic Camera to identify landmarks around the world.

Final thoughts and looking?ahead

Our users love this game mechanic not only for its wow factor but also because it mimics a natural, everyday behavior?—?taking photos of interesting discoveries?—?while enriching the experience with useful and insightful information.

Transforming the “Magic Camera” from an idea into an integral part of the Strollhunt experience was a thrilling technical challenge. We continue to explore new algorithmic and non-algorithmic methods to further improve its accuracy.

?? Next in Part 2: We’ll explore how Strollhunt uses Generative AI to create some of the app’s text and image content. Stay tuned! ??

P.S.: I can neither confirm nor deny that a real human being was involved in the writing of this post.