Reducing Technical Debt
Ward Cunningham was the first to write about the concept of reducing technical debt, and he defined it as:
“Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite.
The danger occurs when the debt is not repaid. Every minute spent on a code that is not quite right counts as interest on that debt.”
Engineering organisations can be brought to a standstill under the debt load of an unconsolidated implementation.
Ward used the technical debt metaphor to explain to his business team that creating software quickly to get feedback was good.
In doing so, however, he emphasised two key points:
- The team and organisation must be vigilant about debt repayment as their understanding of the business domain improves.
- The design and implementation of the system need to evolve to embrace that understanding better.
Since the introduction of the term, the software industry has taken some liberties with Cunningham’s definition.
These days, technical debt refers to the shortcuts we purposely take and the many problems plaguing hardware and software systems.
These include:
An unfit (bad) design is one that once made sense but no longer does, given important changes to the business or technologies we now use.
Defects. We have known problems in the software that we haven’t yet invested time in removing, such as insufficient test coverage. These are areas where we know we should do more testing but don’t.
Excessive manual testing. Testing by hand when we really should have automated tests
Poor integration and release management. Performing these activities in a manner that is time-consuming and error-prone
Lack of platform experience. For example, we have mainframe applications written in Cobol, but there are no longer many experienced programmers.
In summary, technical debt today is a placeholder for a multi-dimensional problem.
Naive technical debt
Cunningham did not intend for technical debt to refer to a team member, business maturity, or process deficiencies that lead to sloppy design, poor engineering practices, and a lack of testing.
Proper training, a good understanding of how to apply technical practices, and sound business decision-making can eliminate this kind of debt.
Because of the irresponsible and frequently accidental nature of how this debt is generated, it is referred to as naive technical debt or reckless debt.
Unavoidable technical debt
In addition, there is unavoidable technical debt, which is usually unpredictable and unpreventable.
For example, our understanding of what makes for a good design emerges from doing the design work and building valuable user features.
We can only partially predict upfront how our product and its design will need to evolve. The design and implementation decisions we made early on might need to change as we close critical learning loops and acquire validated learning.
The changes required in the affected areas are unavoidable technical debt.
Strategic technical debt
The final type of technical debt is strategic technical debt. This debt can help organisations better quantify and leverage the economics of important, often time-sensitive decisions.
For example, an organisation might deliberately take shortcuts during product development to achieve an important short-term goal, such as getting a time-sensitive product into the marketplace.
Also, a capital-strapped organisation risks running out of money before completing its product.
Getting a product with technical debt to market at a reduced initial development cost and then generating revenue to sell funds for ongoing development may be the only way for the organisation to avoid death before deployment.
Regardless of how debt was accrued, technical debt is a powerful metaphor because it raises awareness of and provides visibility into an important issue.
The metaphor resonates well with business people well versed in financial debt.
When they hear technical debt, they can quickly appreciate the insightful parallels. The most important is that, just like financial debts, technical debt requires interest payments, which come in the form of extra future development efforts.
We can continue paying the interest (by working around the problems) or pay down the debt principle by refactoring the code to make it cleaner and easier to modify.
The consequences of technical debt
As the level of technical debt rises, so does the severity of the consequences.
Unpredictable tipping point
A vital attribute of technical debt is that it grows unpredictably and non-linearly.
Each bit of technical debt, when added to the pool of existing technical debt, might do significantly more harm than the size of that new debt might imply.
At some point, technical debt achieves a critical mass, where the product reaches a tipping point and becomes unmanageable or chaotic.
At the tipping point, even small changes to the product become significant occasions of uncertainty.
This nonlinear characteristic is a significant business risk; we don’t know when the next piece of straw will break the camel’s back, but all consequences are amplified when it does.
Increased time to delivery
Taking on technical debt means borrowing today against the time required for future work. The greater the debt today, the slower the velocity tomorrow. The term velocity is used in iterative and incremental agile projects.
NOTE: Velocity measures the rate at which work is completed per unit of time. Using scrum, velocity is typically measured as the sum of the size estimates of the product backlog items completed in a Sprint.
Velocity is reported in the same units as product backlog items – usually story points or ideal days. Velocity measures output (the size of what was delivered), not outcome (the value of what was delivered).
Delivering new features and product fixes to customers takes longer when velocity slows.
In high technical debt, the time between deliverables increases rather than decreases. Technical debt is actively working against our best interests in ever-competitive marketplaces.
A significant number of defects
Products with significant technical debt become more complex, making it harder to do things correctly. Compounding defects can cause critical product failures with alarming frequency.
These failures become a major disruption to the normal flow of value-added development work. In addition, the overhead of managing many defects eats into the time available to produce value-added features.
We begin to drown at some point but are so busy treading defect-filled waters that we can’t see how to pull ourselves out of our mess.
Rising development and support costs
As technical debt increases, development and support costs start rising. What used to be simple and cheap to do is now complicated and expensive. Even small changes become very expensive due to increasing levels of technical debt.
When the high technical debt curve starts its aggressive climb, we reach a critical mass of technical debt and are at the tipping point.
Additionally, rising costs can change the economics of whether to proceed with a feature or a defect repair.
A feature that could be built (or a defect that could be repaired) at a low cost in the presence of low technical debt might become too expensive in the presence of high technical debt.
Rising costs make our products less adaptive to the evolving environment in which they must exist.
Product atrophy
As we stop adding new features or fixing defects that could rejuvenate our ageing product, it becomes less appealing to current and potential customers.
As a result, the product starts to atrophy and ceases to be a viable option for most customers.
Those who stay with the product are typically stuck with it for the time being, but as soon as the first opportunity to switch to another product appears, they will probably take it.
Decreased predictability
Making any prediction is nearly impossible for a product with high levels of technical debt. Estimates become bad even for the most experienced team members.
There is too much uncertainty surrounding how long something might take when dealing with a debt-ridden product.
Consequently, our ability to make and reasonably expect to meet commitments could be improved.
Businesses stop trusting anything development has to say, and customers stop trusting anything the business has to say.
Underperformance
Sadly, as technical debt increases, people expect increasingly lower development performance, reducing their expectations of what is possible.
Of course, lower expectations propagate through the value chain, resulting in lower overall performance organisation-wide.
Universal frustration
The unfortunate human consequence of high technical debt is that everyone in the value chain becomes frustrated. The accumulation of all those minor but annoying shortcuts makes work on the product painful.
Eventually, the joy of development disappears and is replaced with the day-to-day grind of fighting issues that no one wants to (or should have to) deal with.
People burn out.
Knowledgeable development team members begin to leave to pursue more gratifying opportunities. As they are in the best position to address the debt problem, their departure worsens things for those who remain.
Morale spirals downward with increasing intensity.
Technical debt doesn’t just suck the joy out of just technical people; it has that same effect on business people. How long do we want to keep making business commitments that can’t be met?
And what about our poor customers, who are trying to run their businesses on top of our debt-ridden product?
They also quickly grow tired of our repeated product failures and inability to fulfil our promises.
The trust that existed through the value chain is replaced with frustration and resentment.
Decreased customer satisfaction
Customer satisfaction will decrease as customer frustration increases. So, the extent of the damage caused by technical debt is not isolated to the development team or the development organisation.
Even worse, the consequences of technical debt can substantially affect our customers and their perception of us.
What can you do to prevent technical debt in your organisation and career?
