Day 2 Marketing: Metrics, dashboards, and all that jazz

“So what…?”?

Of course, as marketers our lives are a continuous splash of colorful creatives, deep copy, mind blowing campaigns, paradigm-shifting strokes of inspired brilliance… Once a decade. Every time else, and all the time between, there’s a thin line between predictably scalable marketing, and mindless fluff. And that line is green, inching up to the right? on a graph in your dashboard!

I don’t think I need to expound the importance of building, acting on, and presenting a data-backed narrative. So what gives, then? The question of “how”…?

So after deep-diving into Marketing Funnels (Day 0) and Building Strategy (Day 1), it only makes sense to talk about metrics and dashboards for the next one.

(Well technically it should be Day 64, considering the atrociously long break I’ve taken in between these essays, but humor me and let’s call it Day 2.)

I look to my marketing dashboard for three things:

• Gauge what’s going well (and what’s not): what investments are paying off, what are the gaps in my marketing strategy
• Fix leaky cauldrons: identify what’s broken, what could perform better
• Inspire external confidence: what narrative should I communicate up/down/to the side

[Nothing radically radical - chances are if you have a colored squiggly line on your console you look for something similar as well. But just had to leave it out there].

Down to the Ground: Zillion Metrics on the MIS (free template)

As we scaled, the level, complexity, and breadth of problems marketing owned (or at least influenced) grew with it. My last MIS – the central pool of all data trends I care about –tracked over 77 metrics and the interactions between them.?

Feel free to use this template of my actual MIS. You’ll notice my metrics around:?

• Awareness
• Funnel Impact: Top of Funnel/ Bottom Funnel trends, Downstream conversions
• Cost splits
• Basic Product Metrics
• Cohortized Conversions* (I’ll circle on this in a bit)
• Pipeline and Revenue behavior.

Going deeper, I use a bunch of “Cumulative Stats” for numbers I had to have on hand even if they weren’t necessarily “created” in the reporting period.?

Of course, there’s always a crisis in marketing world. And when that hits, you need to know where your plumbing breaks. So my report would have Conversion Trends ready – the rate of flow of my metrics through the funnel.?

The only other non-standard thing in my report is probably around “Pipeline Trends”. Given variations in sales cycles, win rates, and ACVs (average contract value or sell price), I think it’s important for marketing to closely follow how the pipeline they bring gets consumed.?

In previous iterations, my MIS included super-nuanced details like individual campaign performance and activities. Eventually though I had to draw some point of abstraction so I could still pull out some meaningful interpretations.?

Complete and deep, sure. But was that perhaps a bit much? With the shifts and sways of the world today, I don’t see how we could have played any other way.?

IF YOU MISSED IT: HERE’S MY TEMPLATE. Take it, it’s free. Think of me when your numbers turn green. Tell your children. Write songs. Or don’t. Just build on it, and make your marketing game stronger…

The Dimensions of building a Data-Backed Narrative

With all these metrics, the next big question is how do you make building an “insightful narrative” a habit…

I like to think of this in 4 dimensions: (1) The Scorecard; (2) The Timeline; (3) The Mechanics; and (4) The Analysis.?

The scorecard represents the fundamental question of “what”. What are we trying to measure or say here. The timeline describes the “when”, and mechanics the “how”.

The Scorecard:??

This is my highest level of abstraction – the narrative I might present to the board. I try to align metrics in here along 4 talk tracks:Growth, Efficiency, Effectiveness, and Coverage.?

1. Growth metrics are any data arc that show an expanding pie – starting from revenue, to top of the funnel. In most organizations (and scale) that I’ve been a part of, marketing directly owns all pipeline delivery – so this becomes my key growth metric.?

2. Efficiency metrics describe the input/output relationship. In our world, that basically comes down to cost. I like looking at cost returns across the funnel – mainly because marketing spend is immediate while closures take time. Measuring costs today to revenue today might not surface the real story. Also measuring spend efficiencies across the funnel let me to drive accountability at each layer while still allowing for inefficiencies at other conversion points (new AEs ramping up, process losses, etc.).

3. Effectiveness metrics surface the realized and estimated impact of marketing activities. Interestingly, I’ve gotten used to clubbing my funnel conversion trends under Effectiveness... Though I can’t seem to recall why. Perhaps because a ton of marketing activities we did on Stage Optimizations (Lead Quality, Conversion Rate, Nurturing..) were aimed at squeezing these intermediaries.

At Chargebee, we had an additional layer of complexity in the customer segments we focused on. TL;DR: while we wanted demand to grow all over, we particularly focused on building our fast-growth segment, so I had an additional effectiveness metric measuring the % of pipeline within our key (ICP) segment.

Another one of my favorite metrics here is “EoQ Estimated Pipeline Value”*. In one swell number it converts abstract MQLs into lovable $$ signs, pushes the burden of predictability to the marketer, and shows why conversion and ACV data is sacrosanct.?

4. Coverage metrics describe the net total pipeline available for sales to close, the amount of pipe estimated to close in the next quarter, and pipeline multiple against targeted revenue.

The way you look at coverage would likely be different based on your sales cycles (vs reporting frequencies). Assuming you present this quarterly, for extremely short sales cycles (say 48 hour PLG flows, or 30-45 day sales motions), starting coverage doesn’t even matter. As long as you can show effectiveness in your efforts to bring in new pipe that closes during the quarter, you’re good.

For extremely long sales cycles (>2 quarters), all that matters is coverage. If you didn’t have pipeline to close at the start of the quarter, it’s unlikely that you’ll be hitting numbers.

Most of us though are perennially stuck in Goldilocks World. At Chargebee, we had a wide variation in deal cycles based on customer segment, pain maturity, geo, etc. So we had to estimate coverage as a combined factor of starting pipeline that was expected to close in the quarter, fondly called Pre-Period Pipeline (PPP). And new pipeline that was brought in and closed within the same period, called In-Period Pipeline (IPP). Of course this gave rise to an entire family of things to decode in the “mechanism” phase.?

*EoQ Est Pipeline = [Current Qtr MQLs] X [% Last Qtr MQL to Pipe] X [$ Last Qtr Median New Deal Value]

The Timeline:

Obviously we want all our Growth metrics to trend up, and Cost metrics to trend down period after period. I’m a sucker for line graphs, so in a fewer metrics to consume world, that would be my view of choice.?

But beyond dashboards, what I care for are insights. Specifically trend and correlation anomalies that need to get surfaced and resolved.?

I wrestled with this for an embarrassingly long time – a colored mess of mini spline charts; a monstrous series of tables with daily, weekly, monthly, and quarterly aggregations…?

The view that I eventually landed on is elegantly simple – just compare the current metric with the period(s) that matter. Last quarter/ month to know if we’re ticking up or down (T-1 Data). Max value in the last 6 months/ quarters (6T Max). And 5Q% (compared to same metric 5 Quarters ago) to account for seasonality.?

The Mechanism

Obviously metrics are great. But what’s more critical in marketing is understanding the relationships and behavior as these metrics meet and socialize with each other.. Just to reiterate – the purpose of these mechanisms were mostly for me to draw out insights to guide my strategy, rather than numbers that could be universally presented.

To just make this easier, I’m going to try and describe our mechanisms with 3 buckets: Defined, Computed, and Cohortized. (I’m only bucketing it this way to explain here – on the MIS these are just the things with source “Computed”).

Defined Mechanisms: Some metrics, like “New ARR Won”? are universal in that they are undisputable. Some, though, are a bit harder. Like Win Rates for example. What’s a good way to define how your win rates should get calculated?

At a certain point in history, we measured this as the $ or # of Wins against Opportunities (or deals) created. This was easy, and made sense when (1) most of the opportunities created closed in the same period, and (2) our rate of new deal addition was linearly correlated with our wins.

Eventually both these rules broke. We were bringing in a mixed bag of opportunities with varying sales cycles. And we unlocked exponential growth in pipeline addition. So now comparing [Wins in Period] to [Opportunities in Period] wasn’t really apples to apples anymore…

The math we landed on was to look at this as the proportion of pipeline won against total consumed in period – how much pipe would we need to close, in order to win a dollar ARR. Of course, at this point our sales machinery had reached that maturity where we could have a decent level of trust on Closure numbers (meaning pipe that exited our system by getting Won, Lost, or marked as Dead).*

*In the early days of scaling a sales team, you should expect process inefficiencies. Reps are likely to push pipeline to the future (hope is a powerful thing) reducing the denominator and inflating this metric. Another reason why you need to revisit your Win Rate definitions every few quarters…?

Computed Mechanisms: I use a bunch of computations to analyze and predict behavior across my funnels. My favorite, of course, falls into understanding pipeline behavior. This is where the IPP metrics from the “Coverage” section get interesting.

When you have a team focused on adding pipeline at breakneck speeds, and another on winning as much of it as possible, some bad habits can go unobserved until too late. Are reps taking all that older pipeline to its logical closure, or are we constantly chasing new deals?

For this, I compare two things for IPP and Pre-Period Pipeline (so 4 data points):

(1) Consumption trends, describing the proportion of pipeline consumed against the pipeline available. This shows me the velocity at which pipeline gets eaten downstream, and

(2) Contribution trends, describing the proportion of wins from this bucket against the total wins.?

Another computed metric that I think is particularly critical now is Estimated Program Payback. This lets you surface when you are likely to recover the program cost spent on acquiring a dollar revenue. In our board presentations, we would typically discuss the Overall Payback as CAC/$Revenue (CAC = Fully Loaded GTM Cost including Marketing, Sales, and Channel Partner costs). But since over 70% of our marketing spend was on Programs, I found Estimated Program Payback a much faster computation to know when we had to go easy on the burn.?

Cohortized Mechanisms: And finally, this. These mechanisms have to be my favorite because they tell a story of such depth, it makes you tear up…

Most marketing activities take time to ripple through the funnel. MQLs take weeks to get through to pipeline, and months to close. Brand and ToFu activities could take multiple quarters before these leads become anything tangible. And with Intent data and ABM, just the time decay in attribution can be a pain. And yet, we need to run all these activities, and we need measurable response NOW.?

Adoption and growth experiments have a similar problem too. You can’t compare users signing up today against activations today. Lucky for us, we have a solution ready to go with Cohorts. In general, all activation, retention, and conversion metrics with any form of time delay deserve to be analyzed in cohorts. Here’s what your typical cohortized analysis might look like:?

I particularly love this because in one neat table you get to see static totals across rows (total events in month), conversion metric on the column(lifetime events attributable to source month), and progress (diagonal). The only trouble is each table gets so monstrously huge that comparing multiple cohortized data pieces becomes painful. If only there was a way to reduce all the intelligence from this entire cohort table into a couple of cute little metrics…

Early in our analyses journey, we tried wrestling with our cohorts for easier patterns. Of course, most conversions have a long tail – you could technically have MQLs moving into pipeline even decades later. But for the purpose of insights, we want to see when the vast majority convert. I work with an 80% threshold: how long do 80% pipe take to get realized from leads we bring in. When do 80% activations happen post signup…?

The fastest way to figure this is by converting your cohort table into time-decay (M+0 impact, M+1impact, etc.), and then plotting a cumulative distribution on this. Again this isn’t some brilliant new patented methodology - you probably analyze your cohorts this way already… Just something I ended up doing.?

For us we realized most things we cared about happened within two periods - most signups get to initial value within the first 2 weeks, or never at all; most MQLs become deals in the first 2 months or never at all… So all I had to track on my report was the M-0 (same month) and M-1 (previous month) impact on conversions.?

This view also let us zoom into squeeze efforts - nurture campaigns and focused sales processes for older leads.

Setting up Operating Cadences?

That was pretty much it. We viewed multiple abstractions of this report on a weekly, monthly, and quarterly level to keep us real, and ensure our strategy took us in the right direction.??

Of course, we didn’t break our heads on all these metrics all the time. But having them together helped me quickly eyeball where issues were and guide our diagnostics from there. Finally, tying the activities against the metrics they were driving let us focus on pieces that we were intentional about, and brought forward key areas we had forgotten to invest.

Acknowledgements and Love: A Zillion <3

I’ve oscillated through most of this post between an “I” and a “we”, but almost every line item here was the combined genius of my operations leaders. First, to Avinash Ravinder for patiently playing through a hundred iterations before we got here, and setting up processes so I could actually get all this data..

Second, to Mrigesh G. – the greatest extension of my operational brain and muscle, for actually co-building this version with me.?

And to the entire marketing team that showed the level of ownership to make this these kind of analyses possible. Sharan Suresh Shri Mithran Nikhil Muralitharan Siddharth Srinivasan Anirhudh Sridharan Prasanna Kumar Aslam B Divya Ganesan AKANKSHA DUTT and the rest of you guys...

Special mention to Krish Subramanian Rajaraman Santhanam Shelley Perry and the Chargebee ops crew for pushing our operational rigor.

Finally, after sputtering and struggling to put these words together for almost 2 months, I owe this piece to Nupura Ughade for helping me flesh out the first draft, review, and get this far with only 3 typos... I don't think I'd have published a word – let alone this novella – anytime this year without your killer Q&A and mad content writing skills!