Code commenting - the myth
I have lately been embroiled in a debate about the importance of commenting in code. While I don’t yet believe that comments are completely unnecessary, I tend to think that they are largely unnecessary. Almost two years ago, I wrote ‘Software development without maps’ in which I extolled the virtues of ‘documentation’. While it can be easy to confuse ‘documentation’ with ‘commenting’ and jump to the conclusion that I’m now contradicting myself, I’d like to draw the distinction between the two. Commenting is only a specific type of documentation - a kind of documentation that is so narrow in scope that it only applies to code. [Sidenote: In general, commenting will end up covering the 'mechanisms' expressed by code, and very little of the 'policies' and justifications - which would be best covered by higher level documentation anyway.]
In essence, I see programming as an activity where one devises and manipulates abstract models. Writing code is merely a process of expression of such models. What I was trying to get at, in my previous blog post, was that the same models can be viewed at different levels of abstraction. The ‘documents’ encompass knowledge about the models at a higher abstraction level than the code. Documents, being UML diagrams, specifications, or other suitable representations. Comments, on the other hand, are at the same level as code. While it may be useful to sketch out a program using comments before the real code is written, I believe that such ‘scaffolding’ should eventually fade away. Like faint construction lines in a technical drawing, such comments are there to provide an initial framework for laying out the real lines and curves. Like faint and rough sketches on canvas, or a block of stone or wood, the comments become unnecessary as the system is painted, or moulded into shape.
As a software system takes shape and matures, the most reliable indicator of what it is doing is the code itself, not any superfluous comments. Indeed, there are various dangers associated with using and maintaining comments beyond the initial stages of coding:
As far as I can tell, the importance of commenting code is 1) seemingly over-emphasised by Computer Science departments, and 2) an unfounded myth perpetuated in programming shops. OK, after a quick trip through IEEEXplore, I found several papers seemingly extolling the virtues of code commenting. However, they all seem to cite the same paper when doing so. A paper written in 1988. A paper about a study based on PL/I and Pascal. Enough said. Clearly, there has been some research done on the topic, but a lot of it seems outdated, especially in the face of more modern concepts such as Object Oriented languages, etc. I shall delve more into this and possibly post a follow-up to this.
In any case, even I was to assume that comments are useful and will bring about world peace, I cannot find any qualitative documentation on the subject. It’s all well and good to say things like “Your code should be 20% commented” or “You need to have comments describing each function” or “Comment about why, not necessarily what, the code is doing”, but no one ever seems to have good examples of such. Such vagueness simply serves to exacerbate the problem - anyone can write practically anything they like in comment blocks and get away with it. “Whaddya mean, I wrote comments! See!” A more practical way of posing this question would be: Given that I have a programming task, how do I write good quality comments that will remain useful to future programmers, given what I know about the problem domain, and how I anticipate it to change? Is there an example of this story and how it pans out?
Taking a step back, let’s look at the real problem that commenting pretends to solve.
As I mentioned earlier, programming involves the manipulation of abstract models and expressing them in code. It therefore follows that one gains more by learning how to more effectively express programming code than by decorating code with comments. Programming code is the ultimate middle ground - it is something understandable both by the programmer and by the machine (at some level). The programmer who can write prose in comments does not hold a candle to the programmer who can get the machine to execute exactly what he wants to get done. There are two main reasons for the existence of comments in code: 1) making up for the lack of higher level documentation, and 2) attempting to mask complexity by ‘explaining it in english’.
The first case is symptomatic of a poor development process, where intentions articulated at the business level aren’t properly captured and architected into solutions at various levels of abstraction. In the absence of UML and other such higher level program models, developers are supposed to resort to code comments to explain ‘why’ certain things are being done, even though higher level descriptions would be more accessible to various stakeholders. (This is a variant of the “explain why you’re doing this” style of commenting) Comments used in such a manner simply mask a lack of traceability between what a particular client wants, what possible solutions are presented and approved, and what ends up getting implemented. The valuable historical record of the back-and-forth discussions detailing what happened and when particular decisions were made, is simply lost. Coupled with an environment where developers are added to and removed from a project in a piecemeal fashion, this is simply a recipe for a fragmented disaster, as few of those involved have a complete memory of the sequence of events.
On the other hand, masking complexity by ‘explaining in english’ is plainly disproven by decades of development on programming languages. Concepts such as functions, classes, methods, packages and libraries were developed for the specific purpose of managing complexity by breaking down code into smaller, safer, more tractable and more manageable chunks. Those allow for the implementation of layerable and composable solution patterns, as well as enabling reuse - all techniques well known to help improve the long term reliability and maintainability of software systems. If it weren’t for those, we’d still be writing long epics in FORTRAN.. if we could even get that far.. The fact that classes and methods can be given meaningful names dispels a lot of the reason for using inline comments. Got a function doing lots of things? Break it up into smaller functions that have descriptive names. The code is then easier to follow at a higher abstraction level. Want to zoom in on a specific step? Just go into that function. Easy. Up and down the abstraction ladder.
Despite the various reasons for writing comments, it all boils down to managing complexity: the complexity of stakeholders’ requirements, the complexity of the solution at hand. What I’ve been trying to say is that all of this relates to the management of the development process. By building maps and models of stakeholders’ requirements, a better understanding of the problem domain can be achieved. The process of building such maps and models also helps in the discovery and resolution of conflicting requirements and essential priorities. From there, developers can devise subsystems that - when coupled together - should aim to meet the set of requirements. Such subsystems can then be defined in terms of their interfaces and interactions with each other - all at a higher abstraction level. From there, the team can then zoom in further into each subsystem and attempt to refine the implementation further. This same process can be repeated down to applications, services, packages, classes and methods. The whole ‘stack’ describes the abstraction ladder in a consistent manner, and moving up and down this stack constitutes the ‘zooming’ action. Explaining the process in such a way presents a simple concept that can be applied by various people involved in the process. Stakeholders talk to architects and account managers to produce documents and customer-facing models, while architects present the same documents and models to developers for further refinement and evaluation. The end product of this process is a whole collection of inter-related artifacts that document the history of the project and its various aspects from different perspectives - all of which is much more useful and accessible then comments buried deep in the code.
Another way of looking at the commenting problem is one of situational awareness. Piles of comments (or code, for that matter) are essentially worthless to a programmer until he reads them. (And when he does read them, the code will provide a more accurate picture of what’s happening, rather than the comments) A programmer (A) who writes some code has implicit knowledge of what the code does and why it’s doing what it does. A programmer (B) who simply reads comments written by someone else has explicit knowledge of what the code is doing, but not necessarily any implicit knowledge. A programmer (C) who reads some code written by someone else internalises a more accurate mental model of what the code is doing. Simply put, programmer A devises a mental model and expresses it in code, while programmer C is doing the reverse process by reading the code and building a mental model. While programmer B has some chance of success at building a model, he might end up doing so ‘faster’ than programmer C, but at the expense of an inaccurate - possibly even out of date - model. This whole construction and deconstruction of mental models is exactly why the ‘abstraction ladder’ development process is powerful - it provides models of varying detail at various levels to enable almost anyone to more easily conceptualise any part of the solution and work with it.
In essence, I see programming as an activity where one devises and manipulates abstract models. Writing code is merely a process of expression of such models. What I was trying to get at, in my previous blog post, was that the same models can be viewed at different levels of abstraction. The ‘documents’ encompass knowledge about the models at a higher abstraction level than the code. Documents, being UML diagrams, specifications, or other suitable representations. Comments, on the other hand, are at the same level as code. While it may be useful to sketch out a program using comments before the real code is written, I believe that such ‘scaffolding’ should eventually fade away. Like faint construction lines in a technical drawing, such comments are there to provide an initial framework for laying out the real lines and curves. Like faint and rough sketches on canvas, or a block of stone or wood, the comments become unnecessary as the system is painted, or moulded into shape.
As a software system takes shape and matures, the most reliable indicator of what it is doing is the code itself, not any superfluous comments. Indeed, there are various dangers associated with using and maintaining comments beyond the initial stages of coding:
- comment maintenance overhead
As code is modified for various reasons - bug fixes, feature additions, refactoring for reuse - extra time is required to verify not only the proper operation of the code (which can be automated), but also to review existing comments and rewriting the ‘comment narrative’ in a way that fits the reshaped code. I consider this to be a special case of duplication of logic - in the presence of comments that attempt to explain the program logic, one needs to maintain not only the actual program logic expressed in the code, but also those comments that ‘pretend’ to be saying what’s happening. Such a task becomes especially daunting in a team of non-trivial size, as various people end up writing and rewriting chunks of this narrative - which all ends up devolving into an incoherent and unreliable mess. There are ways of managing, verifying and testing program code - and enforcing program coding style conventions - written in a diverse environment, but there is no reliable way of getting people to write a natural language in a consistent, non-monotonous way. The best we can do is introduce the editor model - should we then have a ‘comment editor’ on every programming team, to ensure that all the comments flow clearly and adopt the right style and tone? - matching comments with code
On the other hand, if programmers do not take the required amount of time to fully review and rewrite comments as they implement changes, we end up in a situation where the comments do not accurately reflect the sequence of events expressed in the code. As this compounds, over time, programmers who are introduced to the system later on bear the considerable risk of being lead astray while attempting to trace defects. The end result is code of poor quality, at the significant expense of time - time wasted following misleading comments.
As far as I can tell, the importance of commenting code is 1) seemingly over-emphasised by Computer Science departments, and 2) an unfounded myth perpetuated in programming shops. OK, after a quick trip through IEEEXplore, I found several papers seemingly extolling the virtues of code commenting. However, they all seem to cite the same paper when doing so. A paper written in 1988. A paper about a study based on PL/I and Pascal. Enough said. Clearly, there has been some research done on the topic, but a lot of it seems outdated, especially in the face of more modern concepts such as Object Oriented languages, etc. I shall delve more into this and possibly post a follow-up to this.
In any case, even I was to assume that comments are useful and will bring about world peace, I cannot find any qualitative documentation on the subject. It’s all well and good to say things like “Your code should be 20% commented” or “You need to have comments describing each function” or “Comment about why, not necessarily what, the code is doing”, but no one ever seems to have good examples of such. Such vagueness simply serves to exacerbate the problem - anyone can write practically anything they like in comment blocks and get away with it. “Whaddya mean, I wrote comments! See!” A more practical way of posing this question would be: Given that I have a programming task, how do I write good quality comments that will remain useful to future programmers, given what I know about the problem domain, and how I anticipate it to change? Is there an example of this story and how it pans out?
Taking a step back, let’s look at the real problem that commenting pretends to solve.
As I mentioned earlier, programming involves the manipulation of abstract models and expressing them in code. It therefore follows that one gains more by learning how to more effectively express programming code than by decorating code with comments. Programming code is the ultimate middle ground - it is something understandable both by the programmer and by the machine (at some level). The programmer who can write prose in comments does not hold a candle to the programmer who can get the machine to execute exactly what he wants to get done. There are two main reasons for the existence of comments in code: 1) making up for the lack of higher level documentation, and 2) attempting to mask complexity by ‘explaining it in english’.
The first case is symptomatic of a poor development process, where intentions articulated at the business level aren’t properly captured and architected into solutions at various levels of abstraction. In the absence of UML and other such higher level program models, developers are supposed to resort to code comments to explain ‘why’ certain things are being done, even though higher level descriptions would be more accessible to various stakeholders. (This is a variant of the “explain why you’re doing this” style of commenting) Comments used in such a manner simply mask a lack of traceability between what a particular client wants, what possible solutions are presented and approved, and what ends up getting implemented. The valuable historical record of the back-and-forth discussions detailing what happened and when particular decisions were made, is simply lost. Coupled with an environment where developers are added to and removed from a project in a piecemeal fashion, this is simply a recipe for a fragmented disaster, as few of those involved have a complete memory of the sequence of events.
On the other hand, masking complexity by ‘explaining in english’ is plainly disproven by decades of development on programming languages. Concepts such as functions, classes, methods, packages and libraries were developed for the specific purpose of managing complexity by breaking down code into smaller, safer, more tractable and more manageable chunks. Those allow for the implementation of layerable and composable solution patterns, as well as enabling reuse - all techniques well known to help improve the long term reliability and maintainability of software systems. If it weren’t for those, we’d still be writing long epics in FORTRAN.. if we could even get that far.. The fact that classes and methods can be given meaningful names dispels a lot of the reason for using inline comments. Got a function doing lots of things? Break it up into smaller functions that have descriptive names. The code is then easier to follow at a higher abstraction level. Want to zoom in on a specific step? Just go into that function. Easy. Up and down the abstraction ladder.
Despite the various reasons for writing comments, it all boils down to managing complexity: the complexity of stakeholders’ requirements, the complexity of the solution at hand. What I’ve been trying to say is that all of this relates to the management of the development process. By building maps and models of stakeholders’ requirements, a better understanding of the problem domain can be achieved. The process of building such maps and models also helps in the discovery and resolution of conflicting requirements and essential priorities. From there, developers can devise subsystems that - when coupled together - should aim to meet the set of requirements. Such subsystems can then be defined in terms of their interfaces and interactions with each other - all at a higher abstraction level. From there, the team can then zoom in further into each subsystem and attempt to refine the implementation further. This same process can be repeated down to applications, services, packages, classes and methods. The whole ‘stack’ describes the abstraction ladder in a consistent manner, and moving up and down this stack constitutes the ‘zooming’ action. Explaining the process in such a way presents a simple concept that can be applied by various people involved in the process. Stakeholders talk to architects and account managers to produce documents and customer-facing models, while architects present the same documents and models to developers for further refinement and evaluation. The end product of this process is a whole collection of inter-related artifacts that document the history of the project and its various aspects from different perspectives - all of which is much more useful and accessible then comments buried deep in the code.
Another way of looking at the commenting problem is one of situational awareness. Piles of comments (or code, for that matter) are essentially worthless to a programmer until he reads them. (And when he does read them, the code will provide a more accurate picture of what’s happening, rather than the comments) A programmer (A) who writes some code has implicit knowledge of what the code does and why it’s doing what it does. A programmer (B) who simply reads comments written by someone else has explicit knowledge of what the code is doing, but not necessarily any implicit knowledge. A programmer (C) who reads some code written by someone else internalises a more accurate mental model of what the code is doing. Simply put, programmer A devises a mental model and expresses it in code, while programmer C is doing the reverse process by reading the code and building a mental model. While programmer B has some chance of success at building a model, he might end up doing so ‘faster’ than programmer C, but at the expense of an inaccurate - possibly even out of date - model. This whole construction and deconstruction of mental models is exactly why the ‘abstraction ladder’ development process is powerful - it provides models of varying detail at various levels to enable almost anyone to more easily conceptualise any part of the solution and work with it.