XLA washing

Now experience level agreements (XLAs) are gaining in popularity, the danger of "XLA washing" occurred to me. Much like "greenwashing" in environmental practices, XLA washing refers to the superficial modification of service agreements to create a facade of improved user experience without genuine value. Simply adding a few experience-related metrics to an existing service level agreement (SLA) does not constitute a true XLA. Instead, it masks underlying issues and fails to address the real needs and expectations of users. To achieve meaningful improvements, it is essential to go beyond surface-level changes and engage with users to better capture the desired experience.

Watermelon XLAs

Just as poorly-designed SLAs can cause the watermelon effect, where a service or system appears to be performing well on the surface (like the green exterior of a watermelon) but is actually failing or underperforming beneath that surface (like the red interior), the watermelon effect can also occur when XLAs miss the mark. Here are some thoughts on how this might happen.

User experience and context

Understanding user experience and context is essential for creating effective XLAs. Defining desired IT service experiences without investigating actual user experiences can lead to misaligned expectations. Emotional factors, the full user journey, and diverse touchpoints should be considered to capture a complete picture of user experience. A holistic view across multiple channels and devices, varying technical proficiency, accessibility needs, and workplace culture all contribute to how users perceive IT services. Additionally, aligning user experience with consumer technology expectations – or, at least, managing expectations – is important for meeting modern user standards.

Metrics and measurement

Metrics and measurement issues in XLAs can lead to the watermelon effect by presenting a misleading picture of user experience. If XLAs focus only on easily measurable metrics, they may not capture the full scope of user satisfaction. Basing conclusions on averages can obscure individual experiences and nuances, while metrics that can be gamed by IT teams may not reflect true performance. Additionally, failing to update metrics to reflect changing circumstances or inconsistencies in measurement across in-house and outsourced services can further distort the understanding of overall user experience.

Feedback collection and analysis

The effectiveness of feedback collection and analysis is crucial in shaping accurate XLAs. Asking poorly designed survey questions, missing context, and relying too heavily on automated tools can result in unhelpful or incomplete feedback. Sending surveys at inconvenient times, neglecting qualitative data, and not verifying inferred satisfaction with actual user feedback can further undermine the quality of insights gathered. Moreover, failing to inform users about how their input is addressed can diminish trust and the perceived value of the feedback process.

Other factors

Organizational and operational factors, combined with privacy and security concerns, play a significant role in shaping effective XLAs. Insufficient training for support staff can hinder their ability to enhance user experience, while XLAs that aren’t aligned with broader business goals may fail to address critical objectives. Segmentation of users and tailoring XLAs to different groups, as well as considering the impact of change management and IT policies, are crucial for relevance and effectiveness. Privacy and security concerns, such as breaching norms, can erode user trust and impact satisfaction, highlighting the need for stringent adherence to these standards.

Conclusion

While it might seem straightforward and tempting to simply append experience-related metrics to an SLA and label it an XLA, this approach falls short of truly enhancing the IT service experience. Genuine measurement and improvement of user experience require in-depth collaboration with users, not just for them. Without this collaborative effort, the result is just lip service – what could be called "XLA washing." True success in creating an effective XLA demands a thoughtful and comprehensive approach that genuinely reflects and addresses user needs and expectations.

How much do you actually care about the users, and does your system support or hinder your desire to care?

Related article: XLA goes to Genba