What to do if there are many tests, they fail, take a long time to execute, and everything Is unstable?

Ideally, everything should be in one place. So we could see:

• Which tests are long?
• What exactly are the errors in the tests?
• Which tests are failing?

It's great if test reports and CI are set up in a way that everything is visible and clear. The team knows what to do and how to fix the problems with the tests.

But what to do when there is chaos?

• When there are many different tests.
• Multiple repositories.
• A huge project with thousands of tests.

Imagine you have grown to the point where you have 100000+ different tests. Now they start failing randomly. Everything is red (unit, integration, contract, end2end). What exactly should we fix?

In such cases, I could advise you to collect metrics from the tests and set up some kind of monitoring.

For example, if you have DataDog. Oh, that's great! ?

You can easily visualize the test pyramid.

If, for some reason, you don't have DataDog, you can use regular Grafana. I want to show you how you can set up metrics.

Here's an example:

https://35.176.58.38:3000/public-dashboards/a9ec57ea6b934c8ab14f46fec32e00f9?orgId=1&refresh=5s

What kind of dashboard to create?

It depends on what you need. What information we need to see is what we should visualize. When we have a complete picture of the tests, finding places that need to be fixed or removed becomes much easier.

Here's a bit about how I set it up:

• Took a server.
• Installed an InfluxDB database and connected this database to Grafana.

Now I can see all my test reports from wherever I run them on my page.

This should be done only if there are many tests.

If you have a small product, few tests, they are stable, and you are happy, then you don't need metrics :)

Screenshots:

• Code
• CI Example
• Installation guide

Thank you for your attention! I wish you stable green tests!