News from The Software Quality Intelligence Platform

Become faster, smarter and more efficient with every release.

In recent years we have noticed that in many software development environments often entire testsets are run over and over again. While the test sets continue to grow, the test process takes more and more time. In our opinion: a waste of time and a waste of energy. This can be done more efficiently!

Many test cases in an automated environment are often no longer maintained, prone to errors or flaky. This slows down the CI/CD pipeline and erodes confidence in testing processes. By automating (sometimes as much as possible), we have actually forgotten what was important in the past to go live within the time you had left: prioritizing which tests had to be run first and which were allowed to run later. But, because of test automation we don't need to do that anymore, right?

Indeed, within today's Agile environments, we have hardly seen this for years. In addition, we just run everything because it is - within the enormous amount of automated tests - almost impossible to find out which tests are most important to run first. Besides, it takes too much time as well. As a result, automated software testing can also be a delaying factor in accelerating the development and release processes. So, now we changed that for good ??. For some time at Orangebeard, we have been working on the feature called ‘Auto Test Pilot’.

In this news feed you learn all about this spectacular feature.

Connecting and collecting data

It all starts with data. That’s why Orangebeard aims to collect all relevant information on your test runs. Orangebeard provides interfaces to connect your test tools- and frameworks, as well as tool-specific integrations to send your test runs, -suites and -cases’ output to Orangebeard. You even can roll your own interface using one of our API clients. Regardless of where you store and run your tests from, Orangebeard handles data from any origin, such as your developer laptop, or a Cloud based CI service. And everything in between. Besides test run output, you can also provide Orangebeard with version information of your system under test, so Orangebeard knows what changes are being tested.

The feature: Auto Test Pilot

Based on what we know about a test and how it relates to your product (not just code!), the Auto Test Pilot will help you decide what to test in the limited time you have. Orangebeard links your tests to software components and code changes and proposes optimized subsets and prioritization for the test run at hand.?

At a later stage (based on the changes you provide at the start of a test run) Orangebeard uses an AI model to predict which tests have the greatest chance to fail. This results in optimized (sub)sets prioritized to provide lightning-fast feedback on your current change.

No more waiting for a full regression test when deploying a small change. Orangebeard gives you the opportunity to test the biggest risks first and because we report in real-time, you can act on the first sign of trouble.

At the moment the actual intelligence is based on these steps, while we are developing and training our model:

How we train our model

To train our model, Orangebeard needs meta-data on changes and test results. No static analysis, no instrumentation, so no need to change your software or to share your valuable source code with us.

Our model is trained using data from your test runs. When tests fail, or their characteristics change, Orangebeard learns that those tests have a stronger relationship with the files or components that were changed.

As the number of collected test results data grows, Orangebeard will learn what works best. Your automated testing will become faster, smarter, and more efficient with every test and every release!

Questions and feedback

Please we love to hear your feedback or questions if you need to!

Contact us at [email protected]