Technical debt in forest analytics

Have you ever had a strange feeling about a process or workflow you were engaged with, and thought to yourself “There must be other people that have gone through this?” That’s exactly how I felt up until a few years ago when I learned a term that captured what I was going through perfectly: technical debt.

In a nutshell, technical debt is the cost of future reworking that is needed when choosing an easy but limited solution instead of a better approach that takes more time. It’s a term from software development and encapsulates a process that many of us have been through: we need to get results out soon, so we execute a minimum solution to solve the local problem. Then, we realize that we need to revisit the problem and work a lot more to create a general solution.

I experienced this recently with remeasured forest inventory data. The data were collected from permanent sample plots that were revisited by field crews every few years where they would measure the size and status of trees. But the remeasured data were “messy” (as forestry data often are). One characteristic that made the data imperfect were the negative growths of tree diameters. For example, a technician measured the tree’s diameter to be 4.8 inches last year but this year it’s 4.2 inches. Normal, healthy trees don’t decline in size like this, and it’s a common problem that forest analysts deal with. (The code reconciles the negative growth trends by modifying the negative growth measurement based on subsequent measurements.)??

I had some code written a few years ago that I knew could tackle my negative tree growth problem. I thought a few hours of work to rework the code to my new problem, and voilà, I can ship the new data set off.?

I quickly realized that the code I had was written to apply to a much simpler data set, one that contained only live trees. My new dataset had a mix of live trees, standing dead trees, ingrowth trees, and trees that were harvested. Complicating the analysis was that the diameter variable in my new dataset was sometimes set to zero depending on its status, making the code I had written previously next to useless.?

So, rather than being done in an afternoon, a few days later (and with some help from Github Copilot), I was able to write a more flexible set of code that could apply to my new dataset, and (hopefully) future ones.?

The idea with technical debt is that “interest” will accrue over time the longer you let that code sit. That interest is the amount of time you or your team will need to spend in dollars and hours making the code usable in the future. The longer the amount of time you spend away from that code, the more time it will take to get it working again.

On the one hand, there is nothing inherently wrong with technical debt. A few years ago, I hadn’t envisioned a future application of the code set, a kind of unknown unknown issue. And I am a proponent of shipping minimum viable products out for application in the world. It’s better to have an imperfect solution than none at all.?????

In an ideal world, more time and resources would be available up front to assure that code is of high quality and can be fitting to future applications. Maybe if the analytical community migrates to a slow productivity world, then we could reduce the technical debt interest payments that accrue over time. In the meantime, there are strategies for reducing technical debt, including conducting regular code reviews within your team and testing the code in new applications.?

What is one example of technical debt that you and your organization recently faced?