Why to perform Synthetic Monitoring Tests with many virtual users?
As I was recently thinking about the architecture of our RealLoad product, I remembered that synthetic monitoring jobs in our architecture have the same capabilities as load testing jobs, meaning that monitoring jobs can run with hundreds or thousands of virtual users. But does that make sense? I have summarized the reasons for this here.
Many larger companies that provide web services that have hundreds or thousands of users logged in at the same time have started combining synthetic monitoring with load testing.
For example, if an average of 2000 real users are logged in at the same time, it doesn't matter if a monitoring test with 50 virtual users is running concurrently in parallel.
However, the data measured in this way are much more meaningful than a test that only runs with a single user, as rarely occurring errors can be identified better and faster.
In addition, percentile statistics are also available for each individual test run. This allows performance outliers to be better identified and evaluated.
Another reason to run monitoring tests with many users is when several servers are combined behind a load balancer. If one of these servers becomes unstable or slow, a monitoring test that only runs with one user will often not detect this - or detect it much too late.
Conclusion: No matter which product you use for synthetic monitoring, you should not compromise when it comes to detecting errors.