Evaluating Quality in User-Centered Localization

The localization and globalization industry is undergoing a significant shift in how we evaluate translation quality, particularly in marketing and media content. Rather than solely concentrating on the assessment of individual strings, industry professionals are now adopting a more comprehensive approach, focusing on the overall user experience (UX) of localized content. This is an important change in thinking that will result in the creation of more culturally relevant and user-centered content for global audiences, affecting the localization industry on many different levels.

The Traditional Way of Measuring Quality versus the User-Centered Way

Anyone who has been in the localization field long enough understands that the traditional method of quality measurement involves a meticulous examination of individual strings against predefined error categories within established QA frameworks (LISA, DQF-MQM, etc.). Linguistics experts assess each string logging edits in a QA evaluation form and a numerical score is generated at the end based on word count and recorded penalties. Each source string should correspond to a translation string in the target (one-to-one correspondence).

While language accuracy remains crucial, the industry’s evolving focus on UX demands an evaluation of content in the context of helping users achieve their goals while ensuring an optimal experience. A?broader range of factors play a central role, including:?

• Intent analysis: is the original intent maintained? Will the translation make the final user achieve the same goal as in the source?

• Engagement:?will the reader make it until the end or get bored sooner??

• Cultural appropriateness:?are the images,?symbols, formatting,?and language used culturally relevant?

• Local SEO:?does the translation use the right keywords and phrases to rank well in local search engines?

This shift introduces a new reviewing experience where content is assessed holistically, simulating the final reader’s perspective outside the translation management system (TMS)/computer-assisted translation (CAT) tool. Errors are identified not through source-target string comparisons but by detecting “language anomalies” directly in the target text.

In both traditional and user-centered quality evaluations, reviewers still scrutinize elements like accuracy, grammar, spelling, and fluency. However, in user-centered assessments, cultural appropriateness, adherence to original intent, engagement, and SEO relevance assume greater importance.

This paradigm change results in quality being evaluated through the lens of?user satisfaction.

Terminology and Consistency Relevance Shift

While terminology and consistency remain vital in technical domains like medicine or law, marketing and media translations are witnessing a redefinition of their significance. Increasingly, marketing and media businesses are foregoing the creation of their terminology glossaries, preferring translators to use natural language to express concepts in a way that resonates naturally with the audience. Consistency, once demanding uniformity in translations, is now redefined to prioritize variety and maintain reader engagement.

Evaluating Language Translation in Context of User-Centered Content

To assess language translation for UX, localization practitioners employ various combined methods, including user testing, satisfaction surveys, query and interaction analysis, engagement metrics, customer journey analysis, expert reviews, and linguistic analysis. This multifaceted approach allows evaluators to gauge the quality of language translation from a UX perspective.?This necessitates a more integrated collaboration between localization and globalization teams, moving away from siloed operations. Quality is no longer just a step in the production cycle; it’s a shared mindset and design principle across teams.

There are many other implications to this change in mindset. For instance, being able to directly craft impactful text in the target language, localizers can seamlessly transition into alternative roles, bypassing localization and assuming responsibilities such as transcreation or content writing.?

Why Is Quality Being Evaluated in Context of User-Centered Content?

Several factors contribute to the industry’s shift towards evaluating quality based on overall UX rather than a numerical string-by-string evaluation:

• The importance of user experience (UX):?UX is more important than ever, especially in light of the increased online presence during the Covid-19 pandemic. Users expect websites and apps to be easy to use and navigate, with engaging and informative content. If a localized website proves cumbersome or its content lacks relevance, users are likely to leave and go to a competitor’s site.

• The demand for instant engagement: As a direct result of the digital era, services are just a click away and information travels at the speed of light. Particularly among younger consumers, there is a preference for concise dynamic content such as short videos and reels over lengthy and monotonous articles. If text extends beyond a certain length or becomes repetitive, users swiftly disengage and move on.

• Good machine translation baseline quality: Many companies have invested in developing and training robust MT engines, capable of delivering a reasonably high baseline quality. When post-editing raw MT outputs, the occurrence of linguistic issues, such as typos, grammar mistakes, punctuation errors, capitalization issues, tends to be less frequent. This efficiency allows language experts to redirect their focus towards other facets of the localization and communication process.

• The AI revolution: New technologies such as large language models (LLMs) and generative AI are reshaping our industry. Companies can now expedite the localization of their content with unprecedented efficiency, optimizing some manual and time-consuming tasks. However, these new technologies are not flawless. It is imperative to implement a QA process in place with humans in the loop to avoid hallucinations and ensure the localized content aligns with expectations.

This shift reflects a combination of these factors, as businesses realize the critical role UX plays in global success and invest in technologies that deliver high-quality user experiences, including user-centered content.

Next Steps

Localization teams must reevaluate measurement processes, enhance collaboration between content and UX teams, and adapt to the changing landscape. Introducing new metrics requires an effective change management process, and Centific is poised to assist and can help you adapt.?