How To Address the Five Things Driving the "Us vs Them" Debate About Data Mesh
Dan Everett
The Techno Optimist - Let’s Create A Better World Using Technology The DataIQ 100 USA 2024
It seems to me the debate about #DataMesh has reached full on polarization. On one side you have people outright dismissing it as hype. And on the other side you have people claiming it’s the only way to deal with data.
Zhamak Dehghani posted recently with a comment she was “fearful of a religion forming”, and (on sabbatical) Scott Hirleman (back mid next year maybe but prob not) said the "Church of Data Mesh" is something we must avoid. Well I’m sorry to be the bearer of bad news, but its too late. The debate has taken on the fervor of a religious war.?
So how did this happen? And more importantly, what can we do about it?
We need to understand that cognitive biases create neuro-response triggers. As I have written before cognitive bias is an outcome of evolutionary survival mechanisms, and all humans have them. What’s important is to be aware of how they can/do play out in the data mesh debate and frame our conversations with human neuropsychology in mind.
I’m going to use the SCARF (Status, Certainty, Autonomy, Relatedness, Fairness) model of neuro-response triggers to demonstrate my point. I’m also going to use some examples of things I have heard in conversations about data mesh to provide context for the SCARF examples. These are not meant to call-out anyone, only to illustrate how the neuro-response triggers play out in the context of the data mesh debate.?
Status:?Humans have a deep-seated emotional need to feel valued. A real or “perceived”?reduction in status will trigger a threat response.
In a discussion about data mesh, I heard a comment that “Data engineering as a practice has done more damage than good.” Later in the conversation there was a comment that, “data engineering skills sets need to be reframed as data product developers.”
Work is a fundamental source of status for people.?The first comment can trigger a threat response to status. If I were a data engineer, I imagine I would take offense to the first statement and start thinking about all the ways I have delivered value, and probably ignore the second comment.
How might we have less threating and more productive conversations about changing skills and roles? Perhaps something like.
Validating people’s expertise and value.?The skills people have in data engineering, and the work they have done in curating data have been critical for the foundation of data management and analytics in organizations.
Elevating people’s status.?We think taking those skill sets and reframing them in the context of data product development will elevate the importance of the skills and people who have them. And help organizations better use data to improve outcomes.
If we start with validation, people are less likely to be triggered and more likely to listen to the rest of the conversation.?
Certainty:?Humans are biologically wired to fear situations where we can’t predict and prepare for different outcomes. Consequently, change initiatives by their very nature create uncertainty, fear, and anxiety.
With uncertainty, expediency and experience bias kick in and we tend to default to what know. One approach to mitigate the neuro-response trigger around the desire for certainty, is to try to help people live with the uncertainty for a little longer.
Acknowledging uncertainty. The mere act of acknowledging uncertainty helps us reduce stress and anxiety and makes people more open to new ideas and change. It opens the door for dialogue about concerns and challenges, and how to address them.
A great example of this was a comment I heard that, “With data mesh we are early in the process, and we haven’t figured out all the details to do this at scale.”
Focusing on the present and what we can control. While we might not have all the details on how to scale data mesh right now, we do know that sharing and using data more broadly and consistently results in better organizational outcomes.
I have also heard some great comments about focusing on the business problems and desired outcomes. Start small with a specific use case, where you have the organizational readiness, and can demonstrate value.
Autonomy:?Humans also have a fundamental need to feel like we have choice in what is happening. Don't underestimate the importance of allowing people the time to come to their own insights about why change is needed, what needs to change, and how it should be done.
One comment I heard was that “Pipeline and data mesh should not go in the same sentence together.” My experience bias immediately kicked in, and I started thinking how can you have a distributed architecture without connectivity and pipelines? That’s just stupid!
Full disclosure, that was the exact statement that popped into my head, complete with emphasis. I’m human, I have a brain, hence I have cognitive bias. Fortunately, I’ve practiced hard at listening with the intent to understand.
As the conversation continued, my interpretation was that the idea of data pipelines needs to evolve from the linear collect and consolidate approach, to a connect and share mindset. I acknowledge that my "interpretation" may not accurately or completely reflect the intent of the speaker.
How might we facilitate productive dialogue about changing our definition and understanding of terms and concepts? Perhaps something like.
Asking about the need for change. Have you thought about how we need to evolve to support the business requirements for speed and agility in data and analytics? One of the things we’ve been thinking about is how we evolve the concept of data pipelines from the current linear data collection and consolidation approach to a connect and share mindset. What do you think?
Asking people for input. This is our current thinking on how the concept of data pipelines could change to better support speed and agility. What challenges do you see? Are there things we aren’t considering? What would you do differently and why?
Two thoughts here. “Should” and “Shouldn’t” are trigger words that make people defensive. When asking people to change its easier to modify an existing concept than to get them to forget one, or adopt a completely new one.
Relatedness:?People instinctively form in-groups and out-groups, and the way we perceive, think, and interact with other people changes based on which group we associate them with. In-group people are trustworthy and out-group people are treated with skepticism.
From what I’m seeing in the data mesh debate right now the in-group and out-group polarization is active and strong. One approach to mitigate the neuro-response trigger around relatedness, is to try to help people focus on the similarities, and why they matter for a larger purpose.
Focusing on similarities.?We are all wanting to use data to improve organizational outcomes. And we are all trying figure out how to best use data management capabilities to simplify and accelerate the discovery, curation, and sharing of data.
Creating a superordinate purpose.? Let’s focus on enabling people to be more productive as individuals, collaborative as groups, innovative as organizations, and equitable as societies. A superordinate purpose provides common objectives and shared goals making people more open to new perspectives and change. ?
Humans are social creatures with a strong desire for connection and sense of belonging. The more we focus on similarities and superordinate purpose the better will we be at creating relationships between people with diverse views and opinions.
Fairness:?Humans have biological wiring that creates powerful emotional responses to fair and unfair treatment. A real or “perceived” action of unfairness will trigger an aversive response, feelings of victimization, and a desire to discredit the concept, process, or person.
I’ve seen examples of what I consider disrespectful posts/comments regarding the challenges of data mesh. Even though they weren’t direct at me personally my triggered response was retaliatory in nature.
How might we facilitate non-inflammatory dialogue about the challenges of data mesh? Perhaps something like.
Asking about the challenges. Have you thought about the challenges of doing federated governance? One of the things I’ve been thinking about is how organizations struggle with semantic consistency within a centralized model. How do you think we should be addressing semantic consistency in a federated model?
If we can change the tone of our words from “I’m right” or “Your wrong” to “How do we” it can change the nature of our interactions with each other.
Although not directly related to the data mesh debate, I also see Fairness being an issue as organizational structures are changed to support data mesh. When teams are disbanded and new teams are formed, the disruption of relationships and networks people have worked to establish and nurture, can create feelings of isolation, alienation, and marginalization.
To help mitigate fairness related neuro-response triggers caused by organizational change we can create transparency into the process, and recruit allies.
Creating process transparency. We need to be more agile in using data to drive business performance. We believe organizing around business domains, and distributing data governance and stewardship will help us to improve business responsiveness and flexibility.?
When people understand why change is made, and the process used to determine the new order of things they may not like the decision, but they are more likely to accept it. And don’t forget to give people the opportunity to ask questions, and time to come to their own conclusions about the process, and the need for change.
Recruiting allies. People are also strongly influenced by who communicates information, and what others in their peer group say and do. Find the group influencers and recruit them as allies. These should be the people seen as a trusted adviser/expert among the co-workers of a particular group, not people in a formal authority role over the group.
If you can get them to acknowledged the change is required and the process is fair, they can help you influence their peers. Ask them to share their analysis and conclusions. In addition to influencing peers, getting people to formally express their views in front of others or by putting them in writing or in video, strengthens their commitment to the view.
How To Address the Five Things Driving the "Us vs Them" Debate About Data Mesh
These are my thoughts on why data mesh conversations have an “Us” versus “Them” nature, and some things we can do to address the polarization.
Now while I'm an optimist, I’m not na?ve. I don’t believe just highlighting how cognitive biases create neuro-response triggers will end the polarization. The ubiquitous and unconscious nature of bias means it will show up, we have to take responsibility for our behavior, and we must work to change it.
But if we understand the nature of bias and are mindful of our triggers, I'm optimistic we can make our conversations more respectful and productive.
I welcome your thoughts and feedback.
Technology Management | AIML | Digital & Data Transformations | Systems & Software | Consultant | Mentor
2 年well articulated Dan Everett - The Techno Optimist. I always believe healthy disagreements do lead to a better future, but yes these dug in reactions are hard not to notice. The SCARF approach you have outlined, if practised right, could help to keep bias at bay during such discussions. Well summarized. ??
The Techno Optimist - Let’s Create A Better World Using Technology The DataIQ 100 USA 2024
2 年It's been a busy week. I'm finally getting the opportunity to think about some of the comments here, and others of received outside this thread. 1) Jon Cooke Would love to explore further the 5 things you highlighted. Like how we create clearer language and reconcile conflicting definitions of terms. How do we create a process for integrating and sharing the lessons learned from practitioners implementing data mesh architectures. 2) Scott Hirleman I acknowledge that challenging the status quo is difficult and is being met with resistance. And I believe the language used by data mesh proponents does contribute to the problem, whether they engage in discussion or not. To me the statement, "Data engineering as a practice has done more harm than good" is not constructive, does not create a safe space for dialog, and creates a threat response in data engineers. I don't think the intention of the statement was to create a threat response, which is why I'm trying to highlight the need for mindfulness regarding the language used. Especially if you want to reframe data engineering roles as data product developers to be successful with data mesh implementations. And I could be wrong. Kind regards, Dan
Vice President of Research at BARC
2 年Dan, this is a refreshing perspective. While as professionals we're willing to examine the psychological factors at work within a company, I haven’t seen someone examine them for a tech community before. And it’s necessary if we’re going to help one another make better decisions. Wouldn’t hurt for political discussions, either ??
AI Digital Twins | Simulate business ideas in minutes with AI, real data and Data Object Graphs (DOGs) | Agent DOG Handler | Composable Enterprises with Data Product Pyramid | Data Product Workshop podcast co-host
2 年interesting piece Dan Everett - The Techno Optimist. My 2c on observing, creating thought leadership & trying to provide solutions to mesh movement for 2+ years. I, btw, in a 3rd camp ie love objective (bus facing teams building bus orientated data products) but think the are a lot of gaps in the actual mesh approach my opinion 4 what’s it’s worth: 1) the mesh is a bunch of concepts, not a tangible implementation guide so people project their view with many interpretations ensuing 2) mesh uses confusing & obfuscating language eg architectural/data quantum, sociotechincal (my anti-favourite) and tries to redefine industry terms like product, resulting in lots of friction & debates 3) many solutions presented to fix the gaps but not initially being accepted if it not come from ZD (religion imho). Tbh all the detail coming from industry practitioners interpreting the concepts & implementing 4) if disrupting dw & lake approaches, little detail on how to address data specific challenges (mesh concepts don’t come from the data world), massive push back (understandably) from practitioners in these disciplines 5) v confusing business case. Esp for bus reorg. Largely dev efficiency but not clear on bus strategy alignment
Data Mesh Radio Host - Helping People Understand and Implement Data Mesh Since 2020 ??
2 年One pushback I would have is that very few data mesh proponets actually engage in these discussions. Maybe you are seeing others but my role is not to convert people but help those on the journey. That REALLY angers people because they are used to be pandered to. It often leads to a visceral reaction. That might sound like combative language but I don't think it is - we need to address these issues and trying to say any solution has to be perfect before we start is silly enough, you don't want to engage with those people IMO.