Tackling the Untestable: Your Journey Beyond Conventional Testing Limits

Introduction

In the fast-paced world of software development, we often encounter code that, while functional, presents significant challenges when it comes to testing. This was precisely the situation my team and I found ourselves in. Tasked with ensuring the reliability of a feature-laden application, we faced the daunting prospect of dealing with code that was not designed with testability in mind. The classic solutions at our disposal required substantial refactoring that would take too much time—time we simply did not have. It was a pivotal moment that required a creative solution, and it led to the birth of repeatCheckUntilVerified, a Kotlin utility designed to navigate the murky waters of untestable code by providing a reliable way to perform asynchronous checks in UI tests.

The Challenge at Hand

Our journey began with a critical feature that was integral to the user experience of our application. This feature was built on a complex foundation of asynchronous operations that didn't always play nicely with our existing testing frameworks. The code, while effective in production, turned our testing efforts into a herculean task. The inherent unpredictability of asynchronous operations meant that our tests were flaky, at best, often failing for reasons unrelated to the quality of the code itself.

The Quest for a Solution

Determined to overcome these challenges, we embarked on a quest for a solution that would allow us to write reliable tests without the need for extensive refactoring. The conventional wisdom pointed us toward established practices and tools designed to enhance testability. However, these solutions demanded more time than we could afford, requiring us to rethink our approach from the ground up.

The Breakthrough

The breakthrough came when we realized that we could leverage Kotlin Coroutines, a feature we were already using extensively in our codebase, to create a utility that could systematically check the conditions we needed to test. The idea was simple yet powerful: instead of rewriting large portions of our application to make it more testable, we would develop a tool that could adapt to our code as it was. This tool would need to be flexible enough to handle a variety of testing scenarios, from checking UI elements to verifying the results of background processes.

The Utility Explained

The essence of repeatCheckUntilVerified lies in its ability to perform periodic checks on a specified condition until the condition is met or a maximum number of attempts is reached. This is particularly useful in scenarios where you're dealing with asynchronous operations that affect the UI, such as loading data from a network request or database.

Core Functionality

repeatCheckUntilVerified is built upon Kotlin Coroutines, making it ideal for use in environments that support asynchronous operations. It accepts several parameters:

• initialDelay: The time to wait before the first check is performed.
• maxAttempts: The maximum number of attempts to check the condition.
• onFailureBackup: An optional lambda to execute after a certain number of failed attempts, which can be useful for implementing fallback logic.
• checkConditionTask: The core of the utility, a lambda that defines the condition to check.

The utility method works by initially waiting for the specified initialDelay, then repeatedly checking the given condition after every 250 milliseconds until the condition is met or the maxAttempts limit is reached.

Technical Details

At the heart of repeatCheckUntilVerified is the logic to manage repeated checks with delays, implemented efficiently through Kotlin Coroutines. The utility method throws the last failure result if the condition isn't verified within the allowed attempts, making error handling straightforward in your tests.

How It Works

Upon each attempt, the checkConditionTask is invoked. If this task throws an exception, the attempt is considered failed, and the utility prepares to retry after a brief delay. The cycle continues until the condition passes or the maximum attempts are exhausted. The integration of an onFailureBackup function offers a flexible way to handle repeated failures, such as logging or adjusting test parameters dynamically.

Use Cases

repeatCheckUntilVerified shines in scenarios where UI elements might not immediately reflect changes due to asynchronous operations. For instance, when testing a login screen where user authentication involves network requests, or verifying that a list is populated with data fetched from a database. By employing this utility, developers can write more reliable UI tests that accommodate the inherently asynchronous nature of modern applications.

Practical Examples

Consider a test case where you need to verify that a welcome message appears after a successful login, which involves a network call:

kotlin

    @Test
    fun activityLaunches() = runBlocking<Unit> {
        ...
        repeatCheckUntilVerified { assertGreetingVisible() } 
        ...
    }        

This simple yet powerful usage ensures your test waits just enough for the asynchronous operation to complete, enhancing test reliability.

Advantages and Limitations

Advantages

• Asynchronous Support: Seamlessly integrates with Kotlin Coroutines for handling asynchronous operations.
• Flexibility: Customizable parameters allow for fine-tuning based on the specific needs of each test case.
• Simplicity: Offers a straightforward approach to a common testing challenge, making tests easier to read and maintain.

Limitations and Considerations

While repeatCheckUntilVerified is a powerful tool, it's important to use it judiciously. Overuse, especially with high maxAttempts or initialDelay, can lead to slower test executions. Developers should balance the need for thorough testing with the efficiency of their test suite.

Integration with Existing Projects

Integrating repeatCheckUntilVerified into your project is straightforward. You can include it as a utility function within your test source set, ensuring it's accessible to all your test classes. Customizing the utility to match the timing and behavior of your application's asynchronous operations can significantly enhance the reliability of your UI tests.

Conclusion

The repeatCheckUntilVerified utility represents a pragmatic approach to overcoming the challenges of testing in asynchronous environments. By leveraging Kotlin Coroutines, it provides a flexible, efficient, and easy-to-use solution for ensuring your UI tests are both reliable and resilient. As our applications grow in complexity, utilities like repeatCheckUntilVerified become invaluable allies in maintaining high-quality, user-friendly software.

We encourage developers to experiment with this utility in their projects and welcome feedback and contributions to evolve its capabilities further. Happy testing!