The Technical Debt problem and how to handle it

What exactly is technical debt? Where does the analogy with real debt come into play? Where does it fall apart? And more significantly, how should we deal with technical debts when they arise?

Assume you need a costly automobile but cannot afford it. One alternative is to take out a loan and pay interest over time. The downside is that you will spend more on automobiles. It’s a tradeoff, but it makes sense to avoid paying too much compound interest if the loan is paid on time.

There is, however, a distinction to be made between taking out a $45k loan for a car required to do your business and purchasing an $850k Ferrari. You should only incur debt you can afford to achieve your goals and avoid declaring bankruptcy.

Technical debt is a loan

Technical debt is frequently associated with a bad connotation, although this is not always the case. It’s the same as taking out a loan. It will help you to provide software faster in difficult conditions without devoting enough time to finding the optimal solution.

Technical debt is one of those concepts that, when you hear it for the first time, you immediately understand means. Ward Cunningham coined the term “technical debt.” Ward also founded the wiki, so in a great example of resonance, here’s what Wikipedia, the internet’s wiki-based encyclopedia, has to say about technical debt.

Technical debt is the implied cost of additional rework incurred by selecting a simple or restricted solution over a superior, more time-consuming option. Like monetary debt, it may accumulate interest, making it more difficult to execute improvements over time.

Technical debt is a very common concept these days. It was arguably more contentious when it was first launched in the early 1990s. So here’s an analogy that compares financial concepts to software concepts.

“Let’s develop awful code that will force us to revisit it later due to its poor quality or the solution’s inapplicability.” That is a debt we will either repay or increase in size over time. Until we go bankrupt and are unable to repay our mounting debt.

Technical debt analogy with the financial market

In software terms, the equivalent of bankruptcy is a codebase. One that is so tough to modify that it grinds to a halt or slows to the point where you spend more time and money mending items than developing new ones. Many businesses find it difficult to introduce new features even once a year. The comparison to financial debt is useful in several ways.

To begin with, not all debt is terrible. We could take out a loan of some type to purchase a house or a car, which is OK as long as we can pay off our obligations. The problem is that software and software development are, as usual, a little slicker. If we take out a loan to buy a new automobile, someone lent us the money, and it’s clearly in their best interests to remind us when our payments are due. They’ll also keep a careful check on us to make sure we don’t fall behind on our payments and waste money that should be used to pay off the loan on, I don’t know, booze or partying. The debt parallel between software and finance fails down here.

Regarding software, the difficulty is that the debt depends on our accepting a loan from ourselves in the future. And, let’s be honest, if we don’t repay on time, we’re less likely to call the debt police or break our legs. This indicates that the path to technical debt is steeper and more treacherous than the path to real-world debt. There are a few other areas where the comparison fails. The first is in the repayment process.

How do we pay a Technical Debt?

When you borrow money, your repayments are generally time-based. You will pay off your debt at a set period or, more typically, in a series of payments at regular intervals. This makes it simple to keep track of the payments. If you fail to pay on time, the bank may repossess your vehicle, or Knuckles may pay you a friendly visit to remind you of the significance of prompt debt payback. However, how we repay technical debt differs. We seldom set a payback date, and even when we do, we rarely pay on that day. However, some debts are more vital than others to repay.

Technical debt = bad code?

I could be embarrassed by myself if I cut shortcuts and produce some dreadful code that’s ugly to look at and difficult to alter. However, the extent to which this matters depends on the circumstances. If my terrible code works and is part of a system that isn’t evolving right now, paying off that debt is far easier than if my change is flaky due to the horridness of my solution and is constantly being developed and used by other people. In the first situation, the debt is kind of tucked away and has a low real cost. In the second, the costs will climb quickly as it impacts other people’s jobs, and as bug cancer rates rise, so will the interest on our debt. I recall being in a similar situation a few years ago.

We had a pair of normally excellent developers working on a critical new service. I’m not sure what hit them on this piece of work, but they ended up with an unnecessarily convoluted, very nasty solution to the problem that everyone, including them, detested. But it worked, passed all of the tests, and continued to work after we put it into production. We had it on our to-do list for months, but it never rose to the top of the list enough for us to fix it when I was there. This was not a poor choice. I still believe that was the correct decision. But it still bothers me that this aspect of an otherwise amazing system we were all proud of was tucked away in the corner of an otherwise excellent system. Facing reality

I believe this is practically always true of any system, even if it is generally excellent. There will always be portions of it you want to go back and clean up that you’re not happy with. As a result, our selection of where to make Tech debt payments is influenced by time, urgency, and utility. If you’ll excuse me for stretching the tech debt metaphor any further, we may envisage three forms of debt.

Kinds of technical debt

We might borrow money from a helpful friend or family member. They are unlikely to harm us if we do not repay on time. They might not even charge us interest on the payments or lend us extra if needed.

We may get a loan from a bank. They’re going to tighten up on the repayments. It will surely be interesting to see how the debt grows over time and how they penalize us if we do not pay it back on time. The punishment will typically be harsh but not lethal.

Finally, we may consider borrowing from a loan shark. The interest on this will be quite high, and our debt will rise quickly. If we do not pay, the penalties will be harsh, and we will almost certainly be visited by Knuckles again. Depending on the nature of the debt, we may or may not survive.

So the loan from a friend or family member is an example of a service we weren’t proud of, but it functioned and wasn’t a maintenance burden. I believe deferring the adoption of automated testing in favor of manual testing is a bank loan. It’s a slick slope. It’s terrible because of the attention. The loan’s charges continue to rise until it is repaid. As the system expands, manual testing will become increasingly slow. You may accumulate so many debts that all upward growth is halted. You’re working so hard to keep the software up to date and test your releases that you’re at a standstill and struggle to produce an annual or six-monthly release. A few years ago, I worked with a firm that hadn’t pushed any new features into production in almost five years.

Borrowing from a loan shark

There’s also the loan shark. There will be no testing, no proper design, and only a tactical approach to change. This is frequent in feature factory organizations where the feature output rate takes precedence over everything else. Or at Legacy stores where the debt is so huge that no one is even trying to pay it off. They merely try not to be present when Knuckles comes to visit. At its most extreme, firms that do this frequently believe that hiring more people is the best way to scale up. As a result, teams will have hundreds of employees working in Big Balls of Mud systems, reluctant to make any meaningful changes since the system hasn’t been tested effectively. They’re also afraid of breaking stuff.

So they exacerbate the debt situation by increasing the interest payback by simply inserting their code for a new feature in the first place they locate. They do not consider the system’s design. There is none. They are unconcerned about the solution’s long-term viability. The code is already a mess; thus, this update will simply make it worse. This is a death spiral strategy.

There will come a day when Knuckles will knock on your door, and you will have nowhere to go. The IT team will request things that are critical to the company but are impossible to implement. Customers will begin to complain about the poor quality of the code, but the team will struggle to improve problems because each change destabilizes the system even more. They are so fearful of change that they become more conservative in their willingness to change and slower in their productivity. This may appear an exaggeration, but it is surprisingly prevalent, particularly in large corporations.

What causes technical debt?

There are several reasons for technical debt. Wikipedia lists fifteen. If you’ve worked in the software industry for any time, you’ll recognize the first three. Some individuals refer to the gradual accumulation of cruft as “bit rot.” I’d recast the second one because it’s prone to misinterpretation.

I’d rather fail to start and retain a sense of design throughout the system’s existence than force us to design from the start. And I expect many people would nod in agreement when we blame the third on corporate constraints. Let’s look at each of them and discuss how to prevent them.

The notion that “bits rot” and systems constantly deteriorate is incorrect. Instead, it’s a sign of a more significant issue. Several entrepreneurs approached me during the internet boom of the 2000s, hoping to engage me to help them create world-beating software. Their business ideas were occasionally bizarre, but they were also quite diversified. However, many of them have something in common in terms of business approach. They intended to employ a technical team to fast construct software to implement their world-beating plan. Then, once the program was completed, get rid of the technical staff because they were too expensive. The only advantage of this method was that it simplified my decision.

Every software is eternal by definition

I would not collaborate with somebody who believes that software can be written once and left alone indefinitely. This is not how software works in practice. In the actual world, software is never truly completed. Good software is created through an iterative approach. An evolutionary process then sustains it. At the very least, its dependencies must be updated regularly.

Even back then, I envisaged waterfall-style growth. People know that software maintenance takes far more time and effort than development. Most estimates at the time said that the costs were split around 80/20 in favor of maintenance. Most of the time and money spent on software was on maintenance rather than development.

In Agile development, I believe the model has evolved. Certainly in the context of proper Agile development. There is no distinct distinction between development and upkeep. Instead, it’s about iterative, incremental development that lasts until the product dies. This is more like gardening than making widgets with fire.

An evolutionary process

If we regard our software as a more organic creature that we nurture and shape over time, we begin with the idea that our current predictions are probably incorrect. As a result, we will require rectification at some point in the future. We’ll never be done if we keep our code in a livable environment where we can quickly, confidently, and securely change. Our code is the most recent snapshot of our system’s ongoing progress.

In this perspective, retaining our capacity to make changes fast and securely is the concept of system quality. Once the program is operational, high-quality software on this metric may be altered more rapidly to address any additional shortcomings that may arise over time. Teams that operate in this manner are laser-focused and have a low tolerance for technical debt. They desire a debt-free lifestyle, so I was still stressing about that awful service all those years ago.

The answer to the second problem on Wikipedia’s list follows from the first. Because a naive reading of this suggests, “well, that means I need to design everything up front,” I updated the terminology described by Wikipedia. That is a flawed concept.

Coding is design

Coding, in my opinion, designs, and design is architecture. We never get these things correctly the first time. Great systems are created by individuals who see this and operate in ways that allow it. As a result, we must immediately develop a rough-cut initial estimate of our design.

Some organizational concepts will assist us in structuring options in our code. And then we go to work. When we find an area where our initial design assumptions were incorrect, we modify our code and design and may even re-architecture it. I believe it is critical to keep the team’s feeling of creativity alive and well.

There is some agreement on where things should go in the code. Even if we need to add a new feature years after the system has been in production for years, we don’t just dump it in the first spot that comes to mind. We consider how this connects to our design and place it in our best estimation of where it should live. If we didn’t know where it was, we would look there. This entails constantly updating the model of our system’s tourism map. It also reduces the Tactical creep that the old Coast code bases would otherwise experience.

Technical debt from business pressure

Finally, how about those pesky business pressures? This is when the financial analogy’s opposite side comes into play. How do we prevent getting into debt in the first place in the realm of finance? We put money into things. We may invest in our talents and jobs to supplement our income. However, we also provide cash assistance. We pay today to reap greater rewards later. This comparison holds in software as well.

If your boss comes and applies business pressure by saying something like, “we’ve got this deadline coming, so we need you to get stuff done,” then what does that mean? Are they some kind of drug addict borrowing money from Knuckles to feed their habit of features, or do they mean: “we need to produce software as efficiently as possible and maintain our ability to continue to do that”? Because if it’s the second one, and it ought to be the second one, then working with high-quality, constantly working to minimize technical debt, working so that our code is a high quality easily maintainable place to work is the most efficient way to do work.

If we write rubbish today, my job tomorrow will be slower because I’ll be repairing the crap I wrote today. When we allow technical debt to accumulate, we borrow from our future. If we endeavor to reduce technical debt, we invest in our future productivity.

Even then, we will still make mistakes and create an unintended obligation. But we’ve only borrowed from a friend, so the consequences aren’t quite so severe.

Here at Talendor, we have plenty of highly skilled software engineers who can help reduce the technical debt of your project drastically. Click here to know more.