Code cannot and should not replace technical documentation

I’ve written recently about technical documentation and the good and bad about it.

Over at ElegantCode, new member John Sommez has started out with some posts about eliminating comments from code, and most recently, about how unit tests can replace the need for technical documentation for developers working on a system.

I would go ahead and read what he has to say.  It’s well-written, and he’s obviously given some thought to the topics.  Unfortunately, I think the advice he gives is wrong, and would like to explain why.

As developers, we almost always run into the problem of trying to document what a system is doing and why.  And as I’ve mentioned previously, as developers, we know that detailed technical documents that describe code become stale almost the instant they are written, since the actual code, and the documents that describe that code, are almost never kept in sync, and for obvious reasons.  As developers, we look for ways to make our code better, and in good situations, we create unit tests that prove our code is better.  But we don’t always keep our technical documents absolutely up to date to match our code, and for good reason, as the effort required to do so is often too costly.  I think this is a given and John states this quite well:

“Our simple goal is to replace the “dead” documentation (meaning that it does not update automatically with the thing it is documenting), with “living” documentation in the form of unit tests.”

The problem is that with any system of any semblance of complexity, unit tests cannot document how that system works.

Some of this is for purely technical reasons.  A common infrastructure that I’ve dealt with involves a combination of, C#, Java, SQL, Oracle, and 3rd party vended applications.  Understanding the entire system involves understanding how all of the various parts interact in production.  No unit tests can provide this understanding to a developer.  To understand how the system works involves understanding the entire flow of the system, and by their very nature, unit tests cannot provide this, as they only provide an understanding (at best) of how tiny pieces of the system works. 

Maybe it is just me (though anecdotal evidence suggests it isn’t), but as a developer, I actually don’t really care about understanding an API when learning how a system works.  An API is all about implementation details, and so when writing specific code to implement specific functionality, I do care a bit about that, but that’s (ironically) about the least important thing.  What I want to know is, at a high level, what does the system need to do?  What data needs to move from source A to destination B, and why?  What is the business functionality I am trying to perform?  The developer need to, e.g.,  refactor and make class names and functions more transparent is vitally important, but only after I understand what it is for. 

Unit tests can’t capture this.  They aren’t designed for it (and unit tests as understood in the TDD sense are harmful here, which is one of the reasons why I constantly point out that TDD sucks).

More importantly, unit tests cannot capture why code was written the way it was written, as opposed to the other code that could have been written, but wasn’t.  This sounds more mystical than it needs to be.  One of the best code comments that I’ve ever read (which should have been in a technical document) had an entire block of code that was commented out above the code that actually functioned, with an explanation of why that code was originally used and discarded.  That it was placed in the code itself was obviously not a great thing, except that it was in the perfect place for a developer to see it. 

Existing code can be described very well by unit tests that cover it.  I do not dispute that.  But it cannot tell you why it was chosen.  What discussions with the end users or BAs or project managers led to that code being written as opposed to alternatives?

I’m sure almost everyone has dealt with a situation like something I’ve dealt with recently.  A piece of code referenced a lookup table that defined a threshold level.  Not that it matters, but the level was “12000.”  I suppose one can imagine that someone could create a unit test with a name like “orders_will_be_placed_in_the_processing_queue_within_a_threshold_limit_of_12000_orders_because_the_underlying_MQ_implementation_will_block_otherwise” or some nonsense, but I think it is apparent that this doesn’t really work.  As a developer, I don’t actually want to look at code to know things like this.  I want a technical document that lays out details like this.

When dealing with heterogeneous systems, unit tests cannot explain the boundaries of the systems, of what limitations are there to be dealt with.  Knowing, e.g. that the current Charles River implementation can only handle some level of orders of a certain format is totally outside of anything you can ‘document’ in unit tests.

Developers sometimes have a love affair with code.  My experience with senior level developers has been that, for the most part, they don’t actually care about APIs.  Given some business requirement like “starting with situation A, the end result should be situation B,” there are almost always multiple ways to get from A to B, and the details don’t matter.  Do I need, e.g., to take this code in C# to produce an output that gets processed in the Java part of the system?  Okay.  Tell me what the end goal is, and I’ll write the code that I think does it best.

digression: this is one of the reasons why I believe senior level developers hate code reviews so much in certain instances.  Unless you are dealing with high-performant situations, it really doesn’t matter all that much how you get from business process A to business process B.  It really, really doesn’t matter.

A well written technical document that explains the workflow of a system will pretty much always be more valuable than unit tests.  Unit tests have their own intrinsic value, but for other reasons.

posted on Tuesday, April 27, 2010 11:53 PM Print
No comments posted yet.

Post Comment

Title *
Name *
Comment *  
Please add 2 and 5 and type the answer here: