Saturday 5 September 2015
By this stage, I guess that most developers are aware of the concept of “Technical Debt” as coined by Ward Cunningham in 1992 when he wrote:
“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.” – Ward Cunningham
(Quote taken from the Wikipedia page on Technical Debt on Saturday September 5th 2015)
I’ve found that some developers fail to give Technical Debt the consideration it is due, finding it instead easier to concentrate on delivering on today’s commitments with nary a care for the future. The persistent trend in the IT industry of moving from job to job every year or two only contributes to the problem as the people who cause the pain rarely have to pay the price down the track.
What is Technical Debt?
Technical debt is what you have when some part of your code base is “sub-optimal” in some way.
The analogy with the world of finance is clear - sometimes it makes a lot of sense to go into debt, to purchase a home or to finance an education. Other times debt is a convenient way to manage a budget; anyone who puts most of their spending on their credit card and pays it in full every month is leveraging debt to make life simpler.
The key is to realise that all debt has an associated cost, often in the form of interest payments but sometimes the costs take other forms. Manage debt well and the costs are low compared with the benefit realised - this is why many businesses knowingly leverage debt to grow their business. Conversely, debt that is ignored - or that is incurred unknowingly - can easily compound into a large problem in a surprisingly short amount of time.
The same trade-offs apply to Technical debt. Sometimes it makes sense to deliver code that has obvious deficiencies - when time to market is the most important factor, when a “good enough” solution is all that’s required, or when the code is only going to be in use a short time.
Trouble arrives - and grows - when those deficiencies aren’t addressed in an appropriate manner.
Types of Technical Debt
There are many different kinds of technical debt. Some technical debt is purposeful, a form of leverage to give you an advantage; some can be avoided when you see it coming - and some creeps up on you unawares.
In this blog I want to start an exploration some of the different kinds of Technical Debt, look into their causes and identify some ways that Technical Debt can be easily avoided. Along the way, I’ll relate some real world examples of technical debt and the real world costs that were incurred.
Temporal Technical Debt
Temporal technical debt is when a previously good decision becomes a poor one with the passage of time.
Change is practically the only constant in the development field - the seemingly ever accelerating pace of innovation and invention makes it difficult to simply stay informed about what’s new. It can also render yesterdays good decisions into todays technical debt, sometimes without warning.
Back in the late 1990’s, one of the hot development platforms for Windows was Borland’s Delphi IDE. With a well designed implementation language (an object oriented variant of Pascal designed by Anders Hejlsberg, future designer of Microsoft’s C# language), a rich class library , a visual user interface designer and an optimizing native compiler, there was a lot to like.
One of Delphi’s key strengths was the way it supported connecting to a wide variety of database systems - indeed, this is where the name of the product originated. Just as ancient Greeks would go to the city of Delphi to consult with the Oracle, so Borland wanted modern Geeks to use Delphi when they wanted to access an Oracle database.
The basic abstraction of this connectivity was the DataSet - a class that encapsulated the results of a database query, a table of results organised as a set of rows and columns. Different kinds of ‘DataSet’ allowed for access to different brands of database.
Unfortunately, in early versions of Delphi, there was no in-memory form of the ‘DataSet’ included out of the box - only variants that required active database connections to different database servers. If you wanted to load your data from another source (perhaps a file on disk, or perhaps a different kind of server, a dedicated application server) then you had two choices: license one of the many third party products that arose to fill the gap, or write one yourself.
I was working for a ground breaking data visualization company, enhancing their core product. This company had taken the route of writing their own ‘DataSet’ implementation - it was incomplete in some areas, but it worked well enough for their needs and they had a great deal of code built upon that foundation.
At the time they wrote it, their ‘DataSet’ implementation was a great asset, as it allowed their product to do things that otherwise would have been impossible. Writing it was a good decision - at the time they wrote it.
Fast forward a few years, and Borland started including a native in-memory implementation of the ‘DataSet’ in the box. This version was complete, well tested, fast and reliable.
Nearly overnight, the market for third party ‘DataSet’s dried up, and everyone who had written a home-grown ‘DataSet’ implementation had a problem: to keep working with and maintaining their own implementation, or to switch.
In other words, the new release of Delphi had also included a healthy dose of technical debt that now had to be carefully managed. More on what we could have done, should have done and actually did,later.
Management of Temporal Technical Debt
The first goal is to become aware of the temporal technical debt. It has a habit of sneaking up on you - it is easy to assume that the good decisions we made years ago, good decisions that have paid off in substantial ways, will remain good decisions, that we don’t need to revisit them.
The second goal is to make a careful decision on how to handle any technical temporal debt that is identified. The correct approach to take depends on the context of your business and your team.
One way to identify temporal technical debt is to carefully review the changes in each new release of the frameworks and tools you use. Another is to stay abreast of changes in your industry by reading the blogs of key companies and individuals. While it’s not necessary for everyone on your team to do this, it should be someone’s responsibility - typically an architect or technical lead.
In the case of the data visualization company that I described earlier, the introduction of an in-the-box implementation of ‘DataSet’ was a significant item, one that we remained in ignorance of for some time.
If we had identified the introduction of the ‘DataSet’ in a timely fashion, there are a number of ways the team might have reacted.
Status Quo - we might have reviewed the new ‘DataSet’ and decided that it’s characteristics were relatively poor, compared with our in-house implementation, and that we should continue working unchanged. This would have been a good choice if, for example, the new ‘DataSet’ had size limitations (e.g. If it only supported 64K rows) or performance problems. The downside is the continued need to work around the limitations of our own implementation and the cost of ongoing maintenance.
Switch - we might have decided the new ‘DataSet’ was sufficiently good that we should switch all our development over immediately, removing all traces of our custom implementation in favour of the new one. This would have been a good choice if the new ‘DataSet’ had new capabilties or if our old ‘DataSet’ had limitations we could overcome. The downside here is that the switch would have been expensive in the short term, and risky - we would have been unable to release a new version of the product until the work was complete.
Strangle - we might have decided the new ‘DataSet’ was a good choice, but that switching our existing code base across quickly was too risky. Instead, we could have applied the Strangler Design Pattern by supporting our custom ‘DataSet’ where it was already used by requiring new development to use the new ‘DataSet’. This would minimize the up front cost of the change, allowing migration to occur piecemeal (if at all). Long term costs would also be minimized because of the reducing dependence on the custom implementation.
As you may have guessed, because we remained in ignorance of the new implementation, we ended up staying with the status quo, but without the benefits of making the decision in a considered fashion. Instead, we found that implementation of key new features took much longer than estimated because of deficiencies imposed by the in-house ‘DataSet’ and without awareness of an easier way.
Conclusion
Any kind of technical debt is dangerous if left unaddressed - but temporal technical debt is particularly dangerous, specifically because it can creep in while you’re not looking, turning yesterdays good decisions into tomorrows headaches. Your only defense is to be proactive and to stay aware of what’s happening.
Comments
blog comments powered by Disqus