Localizing API Landscapes with AI Models

In the news! WhippedUp now fully supports the Spanish language! Both on the Apple App Store and Google Play Store.

As a native Spanish speaker, when my friends and family, my early adopters, first brought up the lack of support for our native tongue, I said, “It will be challenging.” This was because WhippedUp relies heavily on information that I extracted from a recipe dataset in English, using a T5 model that was also trained in English and a classifier BERT model; also, you guessed it, in English. Additionally, it heavily uses GPT-4o, which is at least multi-language, but not without challenges that I will outline.

A second and essential implementation consideration was my deployment landscape. I must propagate the user’s language of choice throughout eight internal services within my app.

Finally, I built the App in React Native / Expo, with several localization and internationalization options. A detail is that React Native’s fetch library worked differently in iOS and Android when sending standard HTTP headers.?

In today’s post, I will give an overview of how I solved this monstrous problem.

Solving the Language Problem

To understand the language piece of the puzzle, I consulted with a close friend and localization expert, Ines Illarramendi. She explained that it’s not just about the language but also gender and regional considerations that make a pristine user experience. Every country calls cooking ingredients slightly different, and many languages have feminine and masculine ways of prompting for information or confirmations. Finally, countries have different standards for measuring units and date formats.

My initial decision was to support only the broader language category. Still, I decided to future-proof my app by passing enough information to every service to make a finer-grained choice.

Localizing the API Landscape

A simplified version of my API landscape looks like the one outlined here:

Here, the problem to solve is to propagate the context of the initial callers (mobile app and web interface) to the rest of the services so they can respond with the closest match available.

To solve this problem, I first had to understand how the clients are supposed to propagate this information. This mechanism works through the Accept-Language HTTP header, and React Native indeed honored it, at least on iOS. However, this header did not make sense for the rest of the landscape; this is because Accept-Language allows for many possible languages, and each could also have an attached weight, so I gave the Mobile API and the Web Interface layers the task of decoding it and digesting it into something I could quickly and reliably read and apply if necessary.

I also had the additional challenge of Android clients needing to send the Accept-Language header. Thankfully, I had built an abstraction layer over the standard ‘fetch’ javascript call, and hacking it so Android behaved the same as iOS wasn’t too much of a deal.

I stably propagated the language within the landscape through a custom HTTP header (x-wu-lang) with the best language choice. Still, I included enough information to get more granular responses in the future. (Peeps, help us get there!)

Localizing the Database

Another challenge associated with Localization was to detect the language in which the user created recipes so that later on, I could, either automatically or by user setting, filter out recipes that aren’t in the user’s preferred language (or offer an automatic translation).

My data store of choice is a document-based database (Firestore), so introducing changes and additional queries to support multiple languages wasn’t too complicated; the biggest challenge was handling caches of AI-generated content, but I will discuss this more in detail when I discuss how I localized AI.

Localizing the CDN

WhippedUp also uses many strings that I extracted from my recipe dataset using helper AI models. This information isn’t perfect and evolving, so it didn’t make sense to toast it in the app bundle. I also share it between the Web and Mobile apps.

Given that I already had a standardized language header within the landscape, I could use it to serve appropriate translations from the CDN.?

I built a script using Google Collab and Python to translate the files by calling the GPT-4o endpoint and saving it into the CDN.

Localizing the AI

This brand-new AI world introduced several challenges when it came to localization. In my case, I generated content using GPT-4o. I also ran classifications using an English-trained and optimized BERT that leveraged multi-tasking, distillation, and quantization techniques to run blazing-fast in my serverless infrastructure.

To solve these challenges, I had to do the following:

• Prompt Engineering: convert the GPT-4o prompts (that produced JSON objects) into prompts that supported contents in multiple languages but did not translate the field names of my resulting JSON (which would have made it pretty tricky to deal with)
• Convert Classification Tasks into Generative Tasks: As much as I loved my hand-trained BERT model, I didn’t have the dataset, the multi-language model, or the budget for the GPUs. So, I rephrased the traditional classification tasks into Generative tasks and moved everything to GPT-4o.

Since moving everything to GPT-4o introduced an unacceptable slowness in the UX, I had to resort to caching most of the classification results, even if they were potentially incorrect. Users can invalidate these caches by making changes to the recipes. Finally, these caches should store the relevant information in multiple languages, making the problem much more interesting to solve.

Final Thoughts

Building a consumer product that deploys in a global market is not simple. It is tempting to start with one target group that speaks one language, but quickly, the technical debt builds up, and translating the product becomes an increasingly unmanageable issue. Building software solutions with translations in mind is a great way to make the transition easier when the time comes. Here, I outline some tips for your consideration:

• Integrate the localization framework early - even if you have only one set of strings.
• Build your generative AI prompts with language in mind.
• Try and leverage, when possible multi-language ML models.
• Consider how important client metadata will have to be propagated throughout your landscape.
• Think bigger and consider different levels of granularity within the language.
• Think of your users and address them in their preferred way; your app design should consider this.