Research Leaders - What are we up to?

If there is one thing I know about us researchers, it’s that we love to geek out about all things research – how we are building our teams, the tools we use, ways to communicate our findings, and the challenges we face in our organizations.?At the end of 2021, I reached out to my network and found a handful of fantastic research leaders willing to talk all things research. I wanted to share back a bit about what I learned from leaders at these companies:

LinkedIn, Facebook, Hover, Asana, Lyft, Workday, Airtable, Google, Nerd Wallet, Capital One

Thank you to everyone who took time out of your busy schedules to talk with me!

?TLDR Takeaways

-?????????3:1 designers/researchers ratio is ideal

-?????????Research Operations is a critical function

-?????????B2B recruiting involves unique challenges

-?????????Creating quarterly/biannual research roadmaps is a best practice

-?????????Research democratization is a hot topic

-?????????New roles are emerging in some organizations

-?????????A magical research repository/knowledge management system does not exist (yet)

-?????????Think about personas and journey maps as synthesized insight repositories

-?????????Communicate often and broadly

-?????????Solicit impact testimonials from partners

The Nitty Gritty

?Resources and Building Teams

Team size varied from 2-40 researchers across the companies I talked to.?However, most agreed that an organizational 3:1 ratio of Designers to Researchers is ideal. This baseline metric was extremely valuable for me to confirm since, currently in my organization, our current ratio is 6:1 – no wonder folks feel overwhelmed and stretched!?

Practice building approaches varied widely depending on the needs, history, and research maturity levels of specific companies, but we all agreed that establishing a Research Operations practice was key to make research enablement as smooth and efficient as possible.?

A leader building a brand-new research practice at a startup shared she is establishing the Research Ops practice first before even hiring researchers, focusing on creating a user community panel for recruiting so when the researchers come aboard they will already have access to the B2B customers and the infrastructure to scale.?However, at her previous company, she hired managers first so they could build out their teams themselves instead of inheriting existing employees.

Another leader opted to staff up senior researchers first, as they would be adept at looking across multiple product pillars.?With those senior researchers in place, they are now looking to hire more junior researchers dedicated to one product area.?So, as with most things research, the answer to what should you do first when building out a new practice is “it depends” – but thinking about it strategically and going about it intentionally is crucial.

Research Operations

As noted above, the Research Operations function is critical for enabling quality and scaling velocity of the organization’s research function.?

These are the primary pillars of a research operations group:

-?????????Recruiting

-?????????Governance – NDAs, raw data storage, etc.

-?????????Tools

-?????????Knowledge Management

-?????????Communications

-?????????Research Craft Advancement / Training

?Recruiting Participants

The primary reason that Research Operations groups exist is to support participant recruitment, as they are fundamental to getting to feedback and insights.

In all research environments, access to the right participants is paramount.?In B2B companies, participant recruitment is magnitudes more difficult than in B2C arenas as, by and large, the pool of qualified participants is much smaller in the B2B context.

“The expectation is set in the consumer world that you can talk to a customer anytime you want and that isn’t necessarily true in the B2B context.”

Here are a couple of strategies being employed to mitigate recruiting challenges:

• Identify research scenarios where audience characteristics don’t really matter and recruiting parameters can be wider/looser, such as, testing micro interactions, some aspects of visual design, and content clarity
• Build a custom panel using multiple approaches – asking users to opt in, targeted recruitment into the panel, and use of external vendors who specialize in niche recruiting, such as GLG, Alphasites, etc.
• Inform, don’t ask gatekeepers.?In B2B environments, access to clients is often gated by salespeople.?At Wells Fargo, we have evolved to an inform model and away from asking for explicit permission to contact customers. In the inform model, we tell salespeople that we will be reaching out to XYZ clients unless we hear from them by date XX we will assume that is okay to do so.?This simple shift has increased our recruiting success rate by 50%.

Research Roadmaps

Another best practice is creating a Research Roadmap or Learning Agenda for the team, preferably covering one or two quarters and in conversation with business partners.?The roadmap outlines the major initiatives of the organization and identifies what research questions need to be answered for each.?Having a research roadmap also empowers the team to say “no” when hit with requests that do not hit the roadmap criteria (or at least open a dialogue for why an exception should be made).

Research roadmaps also give the leader a lens through which to reflect on the practice overall and ask questions such as, do we have the right mix between foundational and evaluative methods, do we need to build out new practice areas (i.e., Advisory Councils), and can the current team scale to the demand or do we need to make a case for growing the team or augmenting via outside vendors (and are those vendors vetted and approved)?

Democratization of Research

Research democratization was a hot topic among all the research leaders I talked to as a means to scale the volume of insights flowing into the organization.?Some were “all in” making customer research a part of every product manager and designer’s job.?One company is even including customer research training as part of the new employee onboarding process. Another leader had the explicit goal to get their researchers out of the business of conducting any evaluative research and to shift that completely to product and design, freeing researchers to focus on more strategic and foundational research. Others are taking a risk mitigation (“sensible democratization”) approach and only democratizing unmoderated research activities for simpler, evaluative studies.

But empowering Product and Design to run some of their own research does not mean that the Research team lets them run wild. In fact, most leaders stressed that a successful research democratization effort required a dedicated program manager to train, coach and consult to teams, giving support across intake, recruiting, prototyping, and template creation. Also critical is the creation of research artifacts and oversight of analysis. It’s crucial to identify, account for, and manage those invisible, or overlooked, tasks.?This could look like having a dedicated Researcher do a 30-minute intake with the team to flag issues and verify prototype creation as well as to set expectations and give tips for conducting research.?

I shared some of our challenges with democratizing research, like our Research Ops team recruiting customers only to find out that the prototype wasn’t ready.?It was fun to see the expressions of horror and empathy on my interviewee’s faces.?But, in all seriousness, it was so important to make the firm connection that democratizing research is not the same as handing over the practice of research. It’s equally critical to note that democratizing research may not be a way to save time for the research team, as overseeing and coaching others to conduct research may, especially at the beginning, take the same amount of time as leading the study. The primary benefits of democratization are around building empathy for the customer and deepening understanding of customer needs across a greater number of people in different roles across the organization. ?

Going forward, I think that in addition to dedicating a researcher to managing the democratization program, we will also ensure to identify the best types of studies for democratization, such as straight forward, unmoderated usability evaluation or very early concept conversations (although, I have to admit, I’m still a little nervous about these: “Don’t you LOVE this idea?!”). A good rule of thumb is if the risk is low, democratization may be a great way to reduce the burden on the research team, freeing up their time and skills for higher stakes investigations. But as noted above, there is an up-front investment of time in training/coaching of others before any efficiencies can be gained from the primary research team.?

New Roles

Some companies are adding new roles in their organizations, including Librarian, internal PR/Communications specialist, and a UX journalist focused on storytelling, creating documentaries, and headline pieces.?

One person I spoke with has the title Senior Equity Research Strategist.?This (very cool!) role sat at the Enterprise level and was responsible for influencing the whole company, not just a specific line of business.?Her team’s charter is to ensure that equity is designed into their company’s experiences and products and to increase the ability of designers, researchers, and product managers to create experiences and products with equity in mind.?An additional focus is making the company a more equitable place for employees by surfacing and addressing institutional inequities.?

Her team is also working on Machine Learning and Natural Language Processing initiatives to reduce algorithmic bias, by applying a culturally literate lens to coding qualitative data for things like sentiment. Even further, they are training employees who were formerly non-exempt customer experience representatives, many of whom are people of color, to do this work and thereby get exposure to and training in exempt status technology positions.??

Knowledge Management and Insights Proliferation

“The main currency of research is synthesis,” a statement made by one of the research leaders and one which I absolutely love.?Synthesis is all about making connections, storytelling, and sensemaking – in short, these are very much human activities. Synthesis, sensemaking, and storytelling are not time-wasting activities – these are the most valuable activities that a researcher can engage in.?

And then there is “knowledge management” that has much more mechanistic connotations, such as, a database where all insights will be catalogued, tagged, searchable and, with the click of a button, delivered to the recipient in a tidy, digestible format.?

“A knowledge management system that really works doesn’t exist.”

?“Our repository is not at the level we would want.”

All the professionals I talked with laughed at the dream of the magical research repository that could serve up synthesized insights with the click of a button. In part, this is because we all know that insights are inextricably linked to context and context must be interpreted by human sensemaking.?

The other challenge is the amount of effort required of researchers and research teams to feed inputs and maintain the solutions that are currently out there.?

Repositories are a means to an end for harnessing and leveraging the knowledge that you already have.

One leader shared the motto “scrappy not crappy” and elaborated, “Our goal is to quickly figure out, what do we already know and what is the fastest way to do more?”?And did her team have a fancy knowledge management solution to get these answers??No, they have Confluence research archive that lists reports in reverse chronological order.?Each report has the researcher’s name on it (such a simple best practice) and partners or other researchers can reach out to that author to learn more.

A key question when thinking about insight repositories is who will be using those insights and for what purpose??It’s safe to assume that, at a high level, the purpose is to ascertain what we already know about a topic.?But, as one insights leader and I discussed, the who matters in terms of how a person wants to experience that information.?For instance, a researcher may set out to replicate research that has already been done because they can’t access and verify the quality of the raw data.?While a senior executive, may just want to know the return on insights – what decisions did the research inform??In between, we might curate information across sources for a workshop or to ground a new team or leader in the work that’s been done to-date.?

Data density, audience, and use are all questions to be considered when thinking about designing a knowledge management strategy and approach.?

Another key question is how much effort will be required to upkeep the knowledge management system and who will bear that effort??Tools like Dovetail and Handrail seem very promising but require researchers to change and adapt their own data collection and analysis processes (typically honed over years) to feed the system.?One Research Operations leader shared this challenge and noted that low adoption has negatively impacted the usefulness of these technology solutions making them “brittle” and “a huge mess” instead of robust, expansive, or complete.

We need to get creative about rightly combining the human and technology into our knowledge management solutions by balancing the need for self-service learning with the richness of human context and storytelling.?

Creating User-centered Design Tools

I think of user-centered design tools – personas, journey maps, experience models – as products that can be made from research insights. ?In some ways, they can fulfill some of the promise of knowledge management repositories by taking insights out of reports and baking them into reusable artifacts.

User-centered design tools like Personas are created and used inconsistently across the organizations I spoke with.?For some, this is just a symptom of the current level of the organization’s maturity while, for others, they feel that the user space is so broad as to be difficult to craft meaningful personas.?Same can be said for Journey Maps – most often these are created for particular initiatives and less often to depict the end-to-end journey.?As one leader commented, “Journey maps are trickly to create and use effectively since no one person is responsible for the end-to-end journey.”?

That said, Personas and Journey Maps can be quite powerful insight synthesis repositories across studies as well as tools for encouraging a common language and understanding of the customer across the organization.?The challenge is how to approach the work – top down, trying to catalogue all persona and journey types, or bottom up, working on project-specific chunks at a time and adding to the persona and journey repository over time??

I’ve done it both ways and would say that there are advantages and barriers to both approaches.?The big-bang, top-down approach requires a lot of resources to conduct primary research and assemble these insights into the user-centered artifacts.?If done internally, these researchers then won’t be on software-release focused projects while they are dedicated to this effort.?The bottom-up approach - where tools are outcomes of project-focused research - has the advantage of informing shippable experiences. The disadvantages are it’s less likely that the organization will ever complete a “full stack” of personas or journey maps and they will always be “just-in-time,” as opposed to foundational for customer understanding. ??

Research Impact

?Most of the leaders I spoke with felt that researchers did a good job of communicating to product teams but fell short when it came to communicating more broadly.?Techniques discussed for proliferating insights more broadly included creating communications, like newsletters, or getting a spot in a broader company newsletter and posting insights or reports to Slack channels to raise awareness.?Extending the invitation for research read outs to employees in the company beyond the project team is one easy action to foster wider proliferation of insights. The most common communication strategy is to go where the eyeballs are in the organization – Slack, Confluence, email, presentations, newsletters were all mentioned.?

I was impressed by three unique approaches of insights proliferation I heard:

• One organization creates a research primer as input into the annual product strategy and planning process which is a 6–12-page syntheses for the Executive team consisting of one-page snippets
• Another has aspirations to immerse employees in customer empathy when they return to the office by creating visual exhibits and even to have teams of customers working in their spaces.
• One organization holds a bi-annual virtual roadshow to immerse folks in the insights collected in the last six months.?The roadshow is a weeklong for 1.5 hours per day and consist of case studies, interviews on research impact with stakeholders, panel discussions, etc.?They had 1K unique visitors to their last roadshow event!

And to sum up insights impact:?“There are no points for producing great reports.” – make sure you solicit feedback from partners by asking them to articulate research’s impact to the product.?

Thank you once again to the AWESOME researchers who shared their time and insights with me.

Thank you, also, to Haila Fine , Pamela Walshe , and Marise Philips for your thoughtful edits and comments on an earlier draft of this report. You rock, ladies!

I’d love to hear what resonated with you, what you learned, and any “a-ha’s” came up for you. [email protected]

Tools and Training mentioned in the interviews:

Ethnio - participant and screener repository and recruiting workflow

User Interviews – niche recruiting

Zoom – conducting remote interviews

UserTesting - unmoderated research, staff augmentation, and research democratization

UserZoom – unmoderated research, staff augmentation, and research democratization

Wevo

dScout – diary studies

Optimal Sort – tree testing, card sort

Qualtrics – surveys

Chaordix – community platform

Open Market – only B2B platform

Rewards Genius – Incentive Fulfillment

Mural – interactive research exercises

FigJam – interactive research exercises

Aurelius – note taking and synthesis – allows tagging (but researchers need to use consistently)

Confluence – research planning and report archive

Slab – database?

Air Table – reports repository

Google Sheets – reports repository

Slack – communicate findings

Sonar – analysis and knowledge management

Usability Hub – ability to access 1000s of people in a panel for 1K/year – give designers/content/product people access for simple research

Training: Turning business questions into human questions – Trisha Weng – Sudden Compass

Jump Associates – Insights Framework – Training

The 2021 UX Research Tools Map