Boost your digitalization: data-driven ways of working
Photo by Franki Chamaki on Unsplash

Boost your digitalization: data-driven ways of working

There’s more data in the world than ever before. According to some sources, the world stores over 50 zettabytes of data. A zettabyte is 2 to the power of 70, or one billion terabytes. So, one would assume that the entire world has moved to data-driven ways of working.

As we all know, this is far from the truth. The vast majority of decisions in the world are based on opinions, potentially outdated beliefs, expired experiences and politics. In many cases, data is used as a mechanism to reinforce the points one is looking to make to convince others and there’s little interest in starting from the beginner’s mind as advocated in Zen.

Although it’s easy to blame ill-intended leaders, the fact is that humans are simply wired to take mental shortcuts as a resource efficiency strategy. Much of our survival over the millennia depended on taking immediate, almost instinctive action in response to threats. Those deeply ingrained behaviors are as much present in modern humans as they were on the African savanna. Anyone who has been in a contentious meeting where things turned from factual to personal has seen evidence of that behavior and experienced it first-hand.

As humans, we’re wired to ignore 99 percent or more of everything that reaches our senses. This causes a strong information bias as we typically note that which we agree with and ignore, even subconsciously, that which doesn’t match our beliefs. As an example, in some of the company boards I’ve been part of, there’s a very strong tendency towards groupthink. Humans want to be liked and respected by each other and therefore disagreement is often suppressed. I then take it upon myself to be the dissenting voice to avoid groupthink, but I’ve realized that this isn’t to my advantage. Although others will treat me with respect, it doesn’t help me build rapport with my fellow board members.??

Because we’re ill-designed to employ data-driven practices, we need to put mechanisms in place to ensure that we operate based on data instead of opinions, beliefs and politics. In my view, there are at least four mechanisms to consider. First, we need to ensure the availability of data. There are numerous cases where companies were drowning in data except for the data needed for a business-critical decision. This means that we need to be very careful and considerate in the type of data we collect and start collecting it long before we see its use become prevalent. As discussed in an earlier post, we need to architect our offering portfolio to make it as easy as possible to add instrumentation to systems in order to start collecting data that we didn’t predict we needed.

Second, many companies operate in the realm of requirements. This is prevalent in software development organizations, but also contracts between companies and agreements between departments in companies are typically specified in terms of requirements. In my experience, every requirement is described as a means to achieve a specific set of outcomes. Earlier, we didn’t have the data to establish that the outcome was achieved and consequently, we used requirements. However, focusing on desired outcomes leads to a much healthier and much more data-driven way of operating and avoids the tendency of many to dictate solutions in areas where they have little to no domain knowledge.

Third, planning is a big topic in most companies. However, with fast feedback cycles through, for instance, DevOps, we can let go of planning as the coordination mechanism and adopt experimentation instead. The problem with planning is that it makes assumptions, many of which are simply incorrect. We tend to operate in a complex belief structure that’s largely unfounded and disconnected from reality. By defining hypotheses and conducting experiments to confirm or disprove these hypotheses, we can develop an open-minded, learning-oriented organizational culture that’s much better connected to reality.

Finally, even when experimenting, we tend to rely on humans to define the topics for experimentation. Where feasible, this should be complemented or even replaced with automated experimentation and optimization. Approaches such as reinforcement learning and multi-armed bandits allow for fully automated exploration of solution spaces and may result in solutions that no human would have ever conceived.

Data-driven ways of working are often agreed to in theory but seldomly followed in practice. This isn’t ill-intended but rather a natural consequence of the way we’re wired as humans. To address this, we need to put mechanisms in place to ensure that we use data-driven ways of working instead of opinions, beliefs and politics. These mechanisms include ensuring data availability, focusing on outcomes, experimentation and automated exploration of solution spaces. Einstein famously said that not everything that can be counted counts and not everything that counts can be counted. However, when it counts and it can be counted, we better do so!

Like what you read? Sign up for my newsletter at?[email protected] or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch), Medium or Twitter (@JanBosch).

Jilles van Gurp

FORMATION CPTO | Experienced tech entrepreneur, (interim) CTO/CPO, strategy, and product consultant, hands-on troubleshooter and mentor.

2 年

Data driven decision making is hard and requires some skills that are often simply missing in companies; and also not distributed evenly between the people that are in charge vs. the people that would actually be able to work in a data driven way. Especially senior management tends to be lacking on this front. And without their backing, it's impossible for companies to become data driven. I've been involved in multiple teams where there was talk of doing AB tests and a clear need for analytics data. But these teams lacked expertise on this front and it never happened. These things have a tendency of turning into a "somebody should do something!", usually with lots of finger pointing. Which gets delegated to some poor software engineer because it all sounds rather technical. Alternatively, this turns into a tools discussion where somebody insists we should use tool x because "they heard it was good" (which is not a data driven decision).

回复
André Ollesson

Din AI-partner | Nordic 100 in AI & Data | Nordics Best AI Startup | COO & AI Implementation @Artificial Collective,

2 年

Thanks for sharing an interesting read Jan Bosch! Let's make the world more data-driven together ??

回复
Christer Lundh

Interim Leadership | Change Agent | Business Transformation | Lean Product Development | Leadership Consultant

2 年

In Lean Product Development aka Knowledge Based Development, the data-driven approach is called Set-Based Development. While in traditional development the opinion-driven approach Point-Based Development is usual practice. Using Set-Based Development, developers collect outer limit and trade-off data. With some training developers can make patterns of feasible (safe) and unfeasible (risk) visible. With outer limit experimentation, exploring bugs, developers learn what doesn’t work. That’s valuable data points to collect, and these are reusable knowledge. What didn’t work this time won’t work next time. At least not without taking specific countermeasures. Decision making gets improved, when everyone ca see the same data. And teams make in real-time priorities, what to do next, based on latest learnings. One mechanism to put in place to ensure data-driven and more efficient development, is to collect, save and make outer limits, trade-offs and ‘what doesn’t work’ data points visible.

回复
Wouter Neef

Founder & CEO @ Data Booster | DataIQ Awards shortlist 2023 & 2024 | Data & AI Literacy Upskilling

2 年

Promising intro, will definitely give it a read Jan Bosch!

回复
Vsevolod Gavrilov

Business Development Manager, Industrial BU Volvo Penta AB

2 年

Thank you for article. Just a couple of comments based on my experience - we need digitalization culture. Often employees are not willing to do a records because: - they are not used to (major one) - they have no right tools/services to do that - they are not paid (or motivated) to do that, especially when recording is time consuming. In other words - it is not in their job description - they have no competence to do that The typical reply from companies: - automate everything you can collect data without involving people - punish those ones who escapes proper data recording (( Re. planning. Many companies does the planning. But many tries different ways to reach the plans i.e. they experimenting. Although in reports they still show figures according to plans. In my eyes planning is not connected to experimenting. You can perform as you use to do or you can try something new. This is managerial risk. The culture of risk/fail acceptance - that is another story. Many companies does not tolerate the risk. And there are many reasons for this. Which lead us to another understanding - data-driven management will require changes in many aspects of the way we do business today.

回复

要查看或添加评论,请登录

Jan Bosch的更多文章

社区洞察

其他会员也浏览了