How to measure collaboration?

Software development is a complex process that requires the collaboration of multiple team members. Measuring the efficiency of this collaboration, however, is not straightforward. It's a multifaceted task that can't be reduced to a single metric.

There are tools available that, from 'inside the black box', measure the efficiency of practices— such as Continuous Integration/Continuous Deployment (CI/CD), Pair Programming, and Code Reviews— that are designed to facilitate and enhance collaboration and shorten feedback loops in software development. However, it's also crucial to remember that it's both possible and valuable to gauge the effectiveness of collaboration from outside this 'black box'.

One way to do this is by examining the efficiency of knowledge discovery in relation to the number of active contributing software developers.

Collaboration, impersonal sources of information, and prior knowledge are integral to the process of knowledge creation, decision-making, learning, and innovation. They each bring unique benefits and, when combined, can enhance the overall effectiveness of an organization. Of those only collaboration can be affected by the number of people who work together.

• Prior knowledge doesn't depend on the number of people in the organization.
• Impersonal sources of information don't depend on the number of people in the organization. If people with access to better impersonal sources of information join, their personal capability will affect the organization only if there is collaboration in place.
• Collaboration depends on the number of people in an organization. Collaboration can be negatively affected by the coordination overhead. Teams of non-collaborating developers become less productive ,as they grow in size.

Thus, we can conclude that if we consider the efficiency of a knowledge discovery relative to the number of contributing software developers then we can infer if the collaboration is increasing or decreasing.

Collaboration can be measured, with the efficiency of a knowledge discovery process, relative to the number of contributing software developers.

The number of contributing developers can be determined by counting those actively participating in a project over a certain period of time.

The efficiency of knowledge discovery, on the other hand, can be understood in terms of how rapidly and effectively missing information is acquired and integrated into the team's work. This can be reflected in the time taken to incorporate missing information, the completeness or accuracy of the understanding of new information, and the impact of the new information on the team's productivity or the quality of their output.

Knowledge Discovery Efficiency (KEDE) is a metric that can be used to measure the efficiency of knowledge discovery in software development. We will refer to an organization's efficiency of knowledge discovery as its 'capability.'

Consider the time-series diagram below, which illustrates the interplay between an organization's size and its capability over time.

The x-axis represents the quarters, while the y-axis on the left displays the 'capability' in terms of Weekly KEDE values. The dark blue line in the diagram represents the average Weekly KEDE for all developers who contributed to the company's projects in a given week, calculated using Exponential Weighted Moving Average (EWMA). This line offers a visual representation of how the organization's capability fluctuates over time.

The right y-axis showcases the size of the company, depicted by the number of developers who contributed to the company's projects in a given week. The green line represents the company's size over time, with each point marking the count of contributing developers for that week. A detailed construction of the diagram can be found here.

The diagram displays a notable period where there is an inverse correlation between the company's size and its capability. Over the span of more than two years, the company size surged from 20 to 44, a substantial 120% increase. Conversely, during the same period, the capability steadily declined from 3.6 to 1.3, indicating a 64% decrease. The diagram illustrates a trend wherein the efficiency of knowledge discovery decreases as the number of contributing software developers increases.

interpreting the diagram, we might suggest potential issues with the level of collaboration in the organization. Challenges could arise in terms of communication, coordination, knowledge silos, individual contribution, or even information overload. While more contributors are generally beneficial, managing the complexity requires effective strategies, clear communication, and robust systems. However, it's crucial to remember that we are observing these trends from 'outside the black box'. The actual cause could be a combination of these factors, or even something entirely different. A more accurate understanding would require 'looking inside the box' to ascertain the underlying reasons.