Language, Tone, Conversation is the new UX Sentences, Questions are new UI & Form elements

What does this mean to Product Mgrs., Digital-UX Experts & Quality Testers in age of Digital Products infused with more and more AI-Features??

??

In the age of AI-infused digital products, language, tone, and conversation are becoming the new user experience (UX), with sentences and questions acting as the new screen and form elements. This shift has profound implications for product managers, digital UX experts, and quality testers.?

??

Key AI Research Papers?

?At Theoremlabs.io, we stay ahead by keeping up with AI research papers that could impact our work in the next 6-8 months. These papers inform our advisory AI thought leadership and guide our experiments before applying them to client solutions. Here are two key papers that highlight how product management, UX design, and QA functions must adapt to support new AI features in digital products.?

If reading AI Papers is not your thing, watch this video for simplified deep dive generate from this article , papers and then prompting NotebookLM. Additional credit: Audacity, Descript.com (to help with audio and video editing)

1. VibesCheck Paper:

-Read the paper - https://arxiv.org/pdf/2410.12851?

-Discovered via a LinkedIn post by Ethan Mollick .?

?Key Idea: VibeCheck analyzes the unique qualitative characteristics, or "vibes," of large language models (LLMs). These vibes, which include aspects like tone, formatting, and writing style, significantly influence user preferences but are often overlooked by traditional evaluation metrics that focus primarily on correctness. VibeCheck provides insights that go beyond accuracy to reveal why users might prefer one LLM over another for a given task.?

2. RAGAS Framework: ?

- Read the paper https://arxiv.org/pdf/2309.15217? ?

-Explore Framework https://github.com/explodinggradients/ragas? ??

- Key Idea: RAGAS is an open-source toolkit that provides objective metrics, automated test data generation, and seamless integration with popular LLM frameworks like LangChain. It aims to streamline and enhance the evaluation process for LLM applications, particularly those utilizing Retrieval-Augmented Generation (RAG). RAGAS helps identify specific areas for improvement, such as context precision and faithfulness of generated responses, allowing for targeted optimization efforts.?

??

Implications for Product Managers and UX Experts?

?Product managers and UX experts must think beyond visual design and focus on user psychology within the context of the use case domain. While the best in the field already do this, others may still focus more on visual aspects like screens, colors, and button styles.?

?- Content as a Key Design Element: Sentences and questions, the building blocks of conversation, become primary design elements. Product managers must ensure the language used is clear, concise, and engaging, mimicking natural human conversation.?

? - Understanding "Vibes”: Product managers need to be aware of the subtle qualitative differences, or "vibes," that LLMs exhibit. Choosing LLMs that align with the desired brand personality and user expectations is critical. For example, a product targeting a younger audience might benefit from an LLM known for its humor and playful tone, while a financial application would require a more formal and professional vibe.?

?- Tone and Style Guide for AI: UX experts should develop guidelines for AI-generated content to ensure consistency and alignment with the overall brand voice. This involves defining acceptable levels of formality, humor, and emotional expression.?

??

Implications for QA Functions and Testing Roles?

QA’ing AI solutions and use cases is still an evolving topic. Determining the necessary QA tools and practices for a firm’s AI program is often an open and procrastinated topic, especially as many firms are still in the discovery and experimentation phase of their AI journey.?

-Skills Upgrade and Change Management: Traditional QA functions are used to a software development life cycle (SDLC) process entrenched in deterministic systems testing. AI systems, however, are predictive, meaning a conversational system interface may take the same input but provide multiple valid outputs. Traditional QA practices need to unlearn significant ideas and best practices from the deterministic testing world and adapt to new metrics such as faithfulness and accuracy.?

??

Conclusion?

?In this new paradigm, the traditional graphical user interface (GUI) takes a backseat, focusing instead on natural and conversational interactions. This shift means that product managers, UX experts, and QA testers must adapt their approaches & tooling to design, development, and testing to ensure the success of AI-infused digital products. VibesCheck & RAGAS are some of core methods & with associated tooling that Product Mgrs , UX and QA should experiment and practice. ?

?

At Theoremlabs.io--we are experimenting with RAGAS and VibesCheck –happy to share our experiences. Reach out: [email protected]?

?