Customer Events, or, The Airing of Grievances

In New York City, we are still deep into closure due to Covid-19.  With all the downtime afforded me by this and being between gigs, I’ve enjoyed sharing my thoughts and experiences on product management. In the first of this series, I shared lessons I learned in my three years at Lightbend.  Then I turned a little more theoretical by focusing on how to best enable innovation through interactions with engineering.  This week I’ve been thinking a lot about experiences at Oracle working on Coherence, where one of my favorite activities was running worldwide customer events.

There are many types of customer events.  At Oracle, we of course had the big pageants like OpenWorld and JavaOne, exhausting events that, for the most part, focus on big corporate strategy.  These events have their purpose and took a lot of my time to plan for and execute, but they were never my favorite event.  Smaller products can get drowned in the noise, and, as product manager, you don’t have the control of the event that you would want.  This is a corporate marketing event.

I’m also not talking about straight up marketing events, too, which are necessary for new product launches.  Product management works closely with marketing to shape these events, but this is product marketing’s event to own.  

No, what I am talking about are the smaller, product-focused events that you, as product manager, have complete control over.  In particular, I ran two types of events: Customer Advisory Boards (CABs), and User Group meetings, which we called Special Interest Groups (SIGs), and can be thought of as a meetup.  

Customer Advisory Boards

Many product teams, but not all, run CABs.  This is where you are going to have the deepest level of conversation with customers around product direction.  All product teams that meet the criteria to run a CAB should do so.  I ran CABs every 6 to 9 months, the cadence in part dictated by our global customer base; we would hold these events in Europe, North America, and Asia-Pacific on a rotating basis.  Although sometimes we would have a customer travel, say, from Tokyo to New York for one of these meetings, more frequently the time and money were too much for a customer to commit to.

Benefits

There are many benefits to running a CAB.

• Building trust with strategic customers.  This can be done in one-on-one meetings as well, but you won’t be able to meet directly with all strategic customers on a regular basis.
• Exposing more of your team to customers.  CABs are a great place to expose senior people to customers who normally only get to talk to customers sporadically.
• Deep discussion on product roadmap.
• Sharing customer stories so that both you and your customers can learn how the product is being used,
• Getting detailed feedback on new or proposed feature designs.

Pitfalls

Aside from the time and effort it takes to stage a CAB, there is one major caution to running a CAB.  It’s the same caution I am always aware of when taking roadmap input from customers. And that is, be mindful of the innovator’s dilemma - feature requests that meet niche requirements but make your product harder or more expensive to operate, while cheaper products catch up to meet the needs of less demanding customers.  CAB’s are the perfect trap for this.  You are listening to customers who have the most invested in your product, who are most knowledgeable, and who can make the most convincing case for a new feature.  But you always want to guard against feature requests that make your product more complex, harder to use, and pull you further from mass adoption.  

So, don’t ever think your CAB members are representative of your potential market.  On the other hand, you need to keep these strategic customers successful and happy: they are your best ambassadors, and they are funding your march to the broader market.

Is it time to hold a CAB?

Not all products are ready to hold CABs.  Your product should ideally meet the following criteria to run a CAB, as opposed to some other type of event.

• Enough customers using your product in anger, and preferably, in production.
• Enough commitment to get a quorum each CAB.  We wanted representation from at least 12 customers.  Generally, we capped each customer at two participants per meeting - two was an ideal meeting to encourage different points of view and color. 
• Customers willing to openly share their use of the product, sometimes in very candid conversation with their direct competitor under mutual NDA.

We kept total participation to around thirty people, making all aspects of planning easier, and making group conversations more manageable. I also participated in larger CABs, and these have very different dynamics.  I am partial to these smaller CABs.

Agenda

Customers put a lot of time and money into attending your CAB, and you want to make sure you show them return on their investment.  I like to spend time early on in the CAB to show how the product team has responded to the prior meeting’s feedback.  Sometimes high priority feedback has been acted upon and a new feature delivered, sometimes it is scheduled on the roadmap.  Since these are your “advisors,” even priorities that have not been acted upon should be discussed.  These customers should be held close - one of the big benefits, as said above, is to build trust.

A typical agenda would look like this:

• Evening before CAB: Informal get together, drinks, lite snacks.
• Day 1 morning: Product Update from Product Management.  This includes section on feedback from other regional CABs
• Day 1 morning: Customer talk.  Includes company background, use case, metrics, and feedback on challenges
• Day 1 afternoon: Engineering deep-dive on new feature
• Day 1 afternoon: one or two additional customer talks
• Day 1 evening: Dinner
• Day 2 morning: Day 1 recap, customer talk or two. Technical Deep Dive.
• Day 2 afternoon:  Heads will be on fire by this point, and some customers will have planes to catch.  Don’t plan a full afternoon.  I save this for roadmap updates, summary, and my favorite part of the meeting: voting on priorities.

Sometimes we held working sessions to ferret our requirements, or to offer participants a choice of conversation, but usually we found customers wanted to participate in all conversations.  And frequently our day 2 agenda was fluid based on feedback from day 1.

In feedback from meetings, we found that customer talks were the most popular among our customers attendees.  

Voting on Priorities

One of my favorite parts of the CAB, after two days of hearing customers’ challenges and wishlists, was to give them a chance to think more deeply about making tradeoffs.  While I started with a more complex or directed voting scheme, I settled on something very simple: on each feature we were voting on, collected from our own list and ideas gathered through the CAB, each company can vote on whether that feature was high priority, medium priority, or low priority.  A customer could vote “high” on as many features as they want, but in the end, voting “high” on every feature would be like not voting at all.  So customers had to really think about what was important to them.  This was a simple procedure.  (We did allow them to ask questions like how hard something was to do, to understand pre-requisites, etc, so they could prioritize smaller features over harder, longer to implement, features.  So they were asked to play product manager as well).  

Contact between CABs

Because we were only engaging at this level with some strategic customers every 18 months or so, we scheduled virtual updates between CABs.  These were shorter product updates, typically one hour or so, with no customer presentations.

Break Time!

One of the most common bits of feedback I received on both CABs and SIGs is that I didn’t schedule enough break time.  Break time gives everyone a chance to interact, and frequently that is where the real work gets done.  I would start with two half-hour breaks per day to encourage deeper conversation.  You’ll find that your agenda gets filled up very quickly and you may need to cut back on feature talks (not customer talks!) but it’ll be worth it.

Special Interest Groups (SIGs)

Well, I had to write about SIGs because these were the most rewarding, fun events I ran.  In addition to being amazing events, these brought me to so many awesome places:  In the US, I ran these in Chicago, LA, the SF Bay area, and across the border in Toronto. Munich, Madrid, Stockholm, Frankfurt, Paris, Copenhagen, London, Rome.  In Beijing, Seoul, Mumbai, Tokyo, Shenzhen. In Melbourne and Sydney.  

Special Interest Groups were our chance to grow local communities.  The idea for these meetings grew out of a singular London SIG.  Started by my colleague Brian Oliver, he ran these quarterly for our large financial services customer base.  These were such a hit that I, at the exhortation of my management, exported it first to New York, and then figured out how to create “traveling” SIGs.  In London and New York, these events, at their peak, drew on the order of 100 people - developers, architects, managers, and Coherence-expert wannabees.  In London, they spilled over to day-long events with two tracks.  But generally they were meetup-like events with at least 25 attendees, run with minimal budget: pizza snack, maybe some beer at a local pub afterward.  We saved on budget by leveraging an Oracle office when feasible.  

Where to Hold

In general, we had two criteria for where to hold a SIG: can we drum up attendance, and cost, including travel.  We had the good fortune to have a global sales and marketing team at Oracle, who can help with customer invites and logistics, and a small but global development team - we had developers in Australia and London, lowering travel costs to some destinations.

Coherence had hotbeds of activity, whether driven by a strong local field team, by word of mouth, or commonality of interest (like finserv in New York and London).  SIGs fed oxygen to the fire. Madrid was such a location, where a strong field spurred Coherence adoption, and SIGs reinforced the growth.

For London and New York, we held events regularly, first on a quarterly basis, and later, as we added more SIGs, every four to six months.  

For the traveling SIG, I liked hosting them twice a year at least.  These events would be held over a one or two week period in anywhere from three to five cities.

Agenda

A typical agenda - outside of our crazy London events where everyone in our user community wanted to speak - was as follows:

• Optional pre-meeting intro to the product for the newbies
• Update from product management
• One or two customer or partner talks, how technical features were leveraged, use cases, etc.
• One or two deeper dive talks from product development.

We aimed to keep these events to one afternoon, not including the pre-meeting. In some locations, we ran them in the evening; each city was different depending on culture and transportation logistics.    During this, we accommodated for breaks, and pizza.  In Madrid we had gambas al ajillo and sparkling wine.  After meeting beers were common too.  Not that I condone that sort of behavior.

Attendees

While CAB’s should be reserved for serious product users, SIGs should be open to all to attend.  (We kicked competitors out, though.)  In particular, it was a great time to show prospects the dedicated community behind the product, and led directly to several deals based on informal customer references.  The SIGs also were valuable to partners who were able to work the breaks and beer to drum up business.  

Summary

None of the above is rocket surgery (one of favorite malapropisms).  But with all the focus on the more technical aspects of the job, you may lose sight of the time-demanding but vital activity of running awesome customer events.  So much good comes from these events, from the cracks in the events, the little unplanned gatherings and conversations around these events, that the investment pays off in ways hard to quantify.