iOS 13 - Test Your App Performance and Memory Metrics

Introduction

Performance tests are not that common, but they are different in a lot of terms from standard unit tests because even if they fail, it doesn’t necessarily mean we have bugs.

Sometimes, a small change in one of the metrics can point to a bigger problem that can be identified only by performance tests, and not by regular QA process or even by your users.

In XCode 11, Apple added several new metrics to the plate and built the tools for performance testing based on a new protocol - XCTMetric Protocol.

Meet The XCTMetric Protocol

So, now we have a combination - a new protocol called XCTMeric, with a set of classes that represent new metrics to profile your functions, and a built-in UI tools in XCode that is really helpful in setting the baseline and show you the current status of your code.So what metrics do we have, that conforms to this protocol?

XCTClockMetric - to measure the time taken for your code to run.

XCTCPUMetric - to measure the number of CPU cycles, number of instructions retired, and CPU time that your code consumed.

XCTStorageMetric - to measure utilization of storage for your code.

XCTMemoryMetric - to measure the physical memory that your code consumed.

All of the metrics are relevant both for UI Tests and Unit Tests.

How do I use them?

Well, that’s easy - you just call the measure() method, and pass the metrics you want to examine, and that’s it!

func testPerforamnce() {
let obj = ProcessingImageService()
let image = UIImage(named: "myImage")
measure(metrics: [XCTClockMetric(), // to measure time
                          XCTCPUMetric(), // to measure cpu cycles
                          XCTStorageMetric(), // to measure storage consuming
                          XCTMemoryMetric()]) { // to measeure RAM consuming
                            
            obj.processImage(image: image) // this is the heavy process
        }
    }

After your first run, you will see a message saying “No baseline average for <name of metric>”

Tapping on that will open a “Performance Result” window. 

Let’s see what we have on that window:

Metric popup menu: If you used several metrics, you can select other metrics from here instead of exit and enter again.

Result: This is important - this is the actual result of your test. It could be the expected result, or better or worse.

Average: This is the average result of all your test runs in this session. This can help decide the expected result you want to get.

Baseline: This is the expected result.

MAX_XTDDEV: Since performance result can’t always be the same, you can deviate from your baseline in order not to fail in the test.

At the bottom, you have an edit button, when you tap it, you can change the baseline and your MAX_XTDDEV.

Best Practices 

1. You don’t have to run performance tests each time. But when you do, try to run them on a device, and the same device as always, especially when it related to clock time metrics.
2. Use Instruments to profile and analyze your code. It’s a waste of time not using this essential tool.
3. Don’t ignore failures of those tests, by just setting a new baseline. It’s very easy to do that because our code just works with no bugs. As I mentioned in the introduction, it can point on a serious issue that is cooking in your code.