A Beginners Guide to Logs vs Metrics

I meet with hundreds of customers every year and regularly encounter misinformation and misunderstandings when it comes to Logs and Metrics. There is often a sense of confusion regarding whether to use one or the other. 

Before I proceed, I must make a declaration. I work for New Relic (a digital intelligence platform, that at the time of writing this article, mostly gathers and processes event and metric data).

With this in mind, it is my experience that the choice facing businesses today is not whether Logs OR Metrics are needed. But rather when to use one or the other. Both are necessary.

What is a Log?

In its simplest form a Log is an historical record of events that occurred within a system. Logs are generally captured as a file and record detailed information about what occurred and what changes were made in the system.

Logs are like the details recorded in an aircraft flight recorder, or the log book and checklists that are maintained before and during a flight.

The detailed nature of logs can make them excellent audit trackers especially when there is a requirement to keep track of historical changes in an environment. Their detailed nature also makes them useful for post-mortem route cause analysis.

If we use an aviation industry analogy, logs are like the details recorded in an aircraft flight recorder, or the log book and checklists that are maintained before and during a flight. It’s essential that logs are captured so that analysis can be done post event or so the route cause of a problem can be found.

What is a Metric?

If logs are like the airplane flight recorder, then Metrics are like the instrumentation throughout an aircraft that feeds the cockpit dashboards with real-time information about the flight situation of that aircraft.

It would be ludicrous to suggest that the pilots of an aircraft would refer to the black box flight recorder to keep the plane safely flying or to deal with a mid-flight troubleshooting incident. The time it would take to analyse and process log data is unlikely to fit the need of a real-time rapidly evolving situation.

Metrics are like the instrumentation throughout an aircraft that feeds the cockpit dashboards with real-time information about the flight situation of that aircraft.

Instead, for this task airplane pilots rely upon the information from their cockpit dashboards which are fed by real-time metrics and event data regarding their velocity, altitude, fuel etc...

Comparably, instrumentation of a system provides real-time metrics on the health and performance of that system. Metrics capture detailed data on things like response time, load time, error rates etc…

When System Metrics are correlated alongside User Experience Metrics, businesses can also observe and react to system issues that are impacting their customers/users.

By their nature, Metrics are often a numeric value, so they require comparatively less storage, analysis and processing power than logs and they lend themselves to charting and tracking on dashboards over time.

Real-time tracking of Metrics also makes them useful for alerting when variances from norms occur, such as a spike in error rate or a degradation in response time.

When System Metrics are correlated alongside User Experience Metrics, businesses can also observe and react to system issues that are impacting their customers/users.

In summary, both Logs and Metrics are essential for any business that is serious about improving system performance and availability and making constant iterative improvement to user experience.

Businesses should not be asking whether to use Logs or Metrics, but rather when and where to use one or the other.

-------------

Chris Marshment is an Enterprise Account Manager at New Relic. The views and opinions expressed in this article are his own and do not necessarily reflect those of New Relic.