How to REALLY Handle Technical Debt

First let’s ensure that we understand the subject at hand, although the definition below is only a little dry.

"In a Dagstuhl [German town] seminar held in 2016, technical debt was defined by academic and industrial experts of the topic as follows: "In software-intensive systems, technical debt is a collection of design or implementation constructs that are expedient in the short term, but set up a technical context that can make future changes more costly or impossible. Technical debt presents an actual or contingent liability whose impact is limited to internal system qualities, primarily maintainability and evolvability."" https://en.wikipedia.org/wiki/Technical_debt

When referring to technical debt there are numerous costs involved:

1.????? The current piece of work that needs to be implemented. If a better solution is implemented this piece of work might be unnecessary and it is often more expensive due to the fact that it might need to be implemented in an environment which is not optimal for this solution.

2.????? The future work that will need to be done to either maintain this solution in the system which is problematic or to replace it or move it to another system.

3.????? There is also the cost of slow and bad processes, and the amount of time users have to spend trying to perform their roles with processes that require intensive manual intervention or are very convoluted. If a user is spending 22 hours a week capturing certain information that should only require 4 hours week (don’t assume these numbers are exaggerated), the 18-hour difference is a real cost, although hidden from the business.

The problem with resolving the issue of incurring technical debt is not technical, it is financial. Technical debt is a hidden cost and as long as it remains hidden there will be no incentive by business to avoid or eliminate it.

The focus of this article are changes and new developments which either perpetuate existing technical debt or create new technical debt.

Most companies have existing technical debt, a debt either created by a specific piece of work or by time, where the system has aged. (Is the system now a debtor?) This category is not included here.

Developers are an interesting subspecies (superspecies?) of Homo Sapiens which I think I will call Homo Minutiae. They are detail focused. Developers swim in a sea of details and can free dive into deep ocean syntax sinkholes most people never visit. As a consequence, Developers will often talk about the details of an issue, often at a level that people they are interacting with are not comfortable. This creates a communication problem between the Developer and other people within the business, often leaving the Dev with the feeling that other people just don’t get the importance or urgency of the issue they are concerned about.

So, the issue of technical debt often becomes an issue of communication.

If we could calculate or estimate the cost of the work that will incur technical debt as well as how much the technical debt will cost to remedy later and add that cost against the cost centre of the department requesting the changes, this would have a significant impact on the decision-making process within a business. There are lots of smart people like engineers and actuaries that can work out these costs or design procedures for doing so, it is not an unassailable problem.

If a piece of work will add technical debt, then the amount or the cost, has to reflect against the cost centre of the requestor. This money needs to be moved into an account for future planned and unplanned maintenance and the cost incurred by technical debt during upgrades. Most SAP upgrades probably cost less than half of the final project cost. The rest of the costs are soaked up by technical debt (bad code, old implementations, integration with legacy systems and broken/outdated business processes that must be redesigned in the middle of what has become a challenging upgrade). Somehow IT, especially technology partners, have convinced CIOs that technical debt is a normal part of an upgrade. It is not. It has been normalized but it is not normal. (I would agree that a small percentage of technical debt is unavoidable.)

CEOs don’t even know technical debt exists. For that matter, most CIOs probably don’t really know much about technical debt either. No CEO will ever have a crack at a CIO for technical debt, and until technical debt lands as a line item on a financial statement that has to be justified by the CIO nothing will ever change.

Once this value appears on the financials and the CEO watches the figure slowly grow, there will be a suddenly urgent necessity to bring it down. Suddenly there will be technical debt discussions and a concept like Kaizen, in the context of software development, will have some meaning. Scrum meetings will include agenda points to keep technical debt to a minimum.

When comparing the cost of various approaches to a piece of work, suddenly, the approaches that incur technical debt will be financially much less attractive. Often these approaches win out as they are more convenient in the short time (like integrating with a really lousy/old legacy system vs implementing a new process that eliminates the legacy system), while hiding the real cost due to the hidden nature of technical debt, but they usually result in long term pain (read: expensive).

In this respect all discussions regarding technical debt are kind of pointless. There is NO necessity or incentive for functional consultants, product owners, project managers, scrum masters, program managers or application managers to avoid technical debt. In this context the decisions will most likely be based on convenience and the shortest route to a solution and of course how much it will cost.

As long as that is the case developers will forever more be pushing that boulder up the hill like the legendary Sisyphus. It is a pointless exercise and often creates friction where the developer is strongly evangelizing a solution that the business regards as unnecessary or inferior.

It is often hard to accept that the primary blocker of progress is the budget, but it is worth coming to terms with this issue which can be a hard stop against your plans. The bottom line (see what I did there) is that until something appears in the budget it is seriously problematic to get action on that issue.

As a developer it might be worth your while to start discussing the costs of technical debt with those in charge. Build an understanding of what technical debt is and discuss the hidden cost aspect of technical debt. Be sure to communicate that it is not just the cost of this change or development but also the cost later to fix what is being done now. Start estimating those costs, break it down in a similar fashion that a project cost will be calculated. See if a standardised method could be developed to calculate technical debt estimates with regards to cost.

This will, over time, change the conversation away from convenience and focus on the actual cost that technical debt incurs within an organization and after all that, it will, hopefully, result in better decision making and better technology outcomes.

It would be great to hear from anyone who has more to say about technical debt and their own experience in this area.