A Case for Specs

Synopsis: Recent editorializing has challenged the value of functional specs, or at least their place in current software development. And to be sure, many of us have seen situations where specs were more reverse-engineered than prescriptive. But saying that specs are counterproductive is a case of discarding the baby with the bathwater. I propose that they are perhaps 80% as necessary as ever, and only 80% as prescriptive as once was the case. The 20% change is accounted for as follows: 1) reverse engineering the specs is a fact of modern (highly visual and interactive) software life and it is not a bad thing; 2) smaller projects are now accomplishing great things with quicker tools than in the past, so some of the need for advance documentation has been supplanted by pre-built solutions. The ideas in this article might help your team maintain a productive perspective on this recent shift in the role of specs.

A recent thread on IXDA.org's discussion group for user experience designers and various software folks elicited a seemingly endless stream of posts (see "Functional Specs") as to the value and nature of functional specs. The current in the stream became whitewater, an absolute torrent, when folks started debating the merits of Jason Fried's article, "Getting Real, Step 1: No Functional Spec." Here are the two key paragraphs of his thesis:

1. "Functional specifications documents lead to an illusion of agreement. A bunch of people agreeing on paragraphs of text is not real agreement. Everyone is reading the same thing, but they’re often thinking something different. This inevitably comes out in the future when it’s too late. “Wait, that’s not what I had in mind…” “Huh? That’s not how we described it.” “Yes it was and we all agreed on it — you even signed off on it.” You know the drill."

2. "Functional specifications document are “yes documents.” They’re political. They’re all about getting to “yes” and we think the goal up front should be getting to “no.” Functional specs lead to scope creep from the very start. There’s very little cost in saying “yeah, ok, let’s add that” to a Word document."

I took the liberty of painting Jason's words (1) good and (2) bad, er, I mean green and red with apologies to the protanopes(?) in the audience. Perhaps you can already see where I'm going with this. The first paragraph, the good, green paragraph is true as true can be... as if straight from the hand of God. It describes the project I'm on right now and almost every nasty specs dilemma I've experienced. Thank you, Jason.

But in the second paragraph, the focus starts to change. Instead of addressing how functional specs are misused, the framing becomes "functional specs are bad," and eventually morphs almost completely into "design documentation is a waste of time on software projects." At least that's the way the bowling ball is spinning by the end of the article. Now, let's keep in perspective, it's just an article, and its unstated premise must be inferred by reasonable people as addressing smaller teams and projects. This is almost certainly justified by today's greater prevalence of web projects that deliver great systems, often in weeks.

But too many of us experience the nasty problems cited in the good, green truth on projects of all size. What's a PM (project manager) to do? And why is this such a tough, contentious nut to crack? My guess is that specs are a topic of fiery discussion because it's not a computer problem... it's one primarily of good writing, and increasingly, images... both of which are beyond many folks' skill set. If it were an IA (information architecture) or UX (user experience) problem, the thread would fizzle out after 10 or 15 posts.

How Did We Get in this Mess?

Why, oh why? In the beginning code was a black box that ran start-to-finish, uninterrupted by users... it was not "interactive." The more that the code accomplished, the more crucial it was to prepare written instructions—marching orders—for the folks that did the coding. And life was good, if only in a boring sort of way. As the command line gave way to character-based menus, the premise was still sound... or if you prefer business-speak, the value proposition was still workable.

Challenge #1: Despite all the disagreement over specs, could anyone disagree that the value of written instructions increases in proportion to the size of a team, and perhaps the distance between team members, creative license notwithstanding? Think offshore. Think Space Shuttle. Would you fly in that thing if creative license trumped written plans?

In 1992, with the proliferation of Windows 3, all that changed. People were making systems without anyone else telling them precisely what it should look like or how it should work. The tools were that powerful. And the end-product simultaneously became both richly interactive and two-dimensional. (Which brings up the interesting question, is a batch program or command line interface one-dimensional? But I'll leave that to the reader, as an extra credit problem to solve at home.) This has resulted in all the problems laid embarassingly bare by Jason.

But it is incorrect to posit that "written instructions are bad because of the problems we currently have with them." Let's look more closely at the problems.

The Timing... and Direction

Specs now often lag instead of lead the implemented code, and this is ridiculed as engineering hypocrisy. The solution is to accept that the timing is a two-way street. Just because solutions evolve in the laboratory doesn't mean we shouldn't write down every last detail of the experiment that worked.

Now, if you're on a project that's so small or un-dynamic that you can get away with saying "the system itself is the spec," that's your call. Who has a right to say you're wrong, except perhaps someone who holds your paycheck. But if the project is complex or has a lot of people, or those people change over time (sound familiar?) or are an ocean away, someone will eventually need not just the cake but the recipe. This is not advanced thinking, or "rocket surgery," as I think, Steve Krug coined the term.

Let me tell you a story. Just yesterday, our lead architect came out of an 11th-hour meeting telling me about newly agreed to changes to our system. Although our system is visual and highly interactive, it also has formulas like Excel, and we just added a new formula. Being so late in the game, and being a topic of such technical nature that only he could explain (and code) it, he dictated the explanation that I edited right into our end user documentation (!) 30 seconds after the meeting. In this, a rarest twist of fate, the end user was the first, not the last to know of a new function... while it was still a glimmer in the developer's eye. I published the results and alerted our spec writer to transcribe the new facts into the design documentation. Does this disprove the legitimacy of specs in an "agile development" shop? Hell, no. It simply highlights the need to make the traffic go both ways on the street of Good Specs. Why should written instructions be castigated just because they are not assured of achieving perfect foresight?

And what about that Space Shuttle? I have been known to use it as a paragon of prescriptive design... or at least a hyperbolic example of the need to put plans on paper. Is it some sort of proof or validation of "prescription???" Yes, but maybe not how you think. There were two spectacular failures in the shuttle program. So the written specs failed, right? Well, that's probably a matter of opinion and debate... one I'm not interested in. What I am interested in is that the failures should result in the specs being edited with the resultant design improvements (note to file: rubber o-rings freeze). In other words, even a monumentally prescriptive project needs reverse engineering of its specs. If you can't get it out of your head that this is embarrasing, that's your problem. The value of the specs is not that they are perfect; it is that they are on papyrus or paper or LCD... somewhere where they stay still and can be seen at a glance without as much interaction and experimentation as the actual system.

Recipes are the ultimate analogy. We make a cake, we taste it, it tastes too sweet. So we reduce the sugar and edit the recipe. We taste it again and like it so we file the recipe away. We don't say "wow, recipes are a horrendous waste of time." In other words, trial-and-error is not disproof of the value of specs.

Do This: Expect your specs to be updated both prescriptively and retroactively. The fact that some design is fleshed-out in the laboratory does not invalidate the worth of written instructions.

The Medium

Specs often try to use text for non-text designs. This is where the situation really blows up. Compared to this one, the timing thing is just a logistics and integrity issue... "So what, our spec is outdated, nobody reads that thing anyway."

Many criticisms of specs deride their "text-ness" and justifiably so. If you were the designer at Ferrari, would you ask the plant to produce the body for a new car from a paragraph of prose? Duh, I think not. Well, it's no different with a graphical interface. Therefore the first part of the solution to many specs dilemmas is to put graphics before rather than after the pertinent text. Why doesn't this happen? Because not everyone is articulate with pictures. And not everyone has the tools, imagination, or time.

Hold it, how could they not have the time? If the implemented design becomes visual, and you're telling me that the problem is that the implementation often occurs before the spec has the details, then the time is spent, right? Wrong. The "design" that comes when a (code or UI) developer creates a page with software tools is not a "designed product." It is the result of feasibility and chance... the factors in play when the page is constructed, including everything from skill level to prior art, tools, frameworks, and technology. It is not a design solution conceived in the harsh light of multiple opinions and broad considerations. It is often a poor stab in the dark. Sentiments about general software usability tell us it is often a problem.

Somehow, as lamented in Jason's first paragraph, "spec" has become synonomous with "paragraphs of text." Well, that's not the specs' fault. It's ours.

Challenge #2: Would anyone disagree that diagrams are not just valuable, but crucial instructions when making a visual product?

The flaw in the syllogism is that genuine design takes time, imagination, skill, planning, and —brace yourselves— trial-and-error! Oh horrors, tell me it isn't so. When we cite the age-old cliche that a picture is worth 1000 words, it's not simply proverbial: by designing and then drawing a picture of a software component, whether dazzling home page or boring Winmac dialog, you have potentially done thousands of words worth of spec-ing. There's no getting around this. The work is not reduced, only the words. Design takes time. The more intricate the solution, the more time it takes.

Do This: Provide as many pictures as it takes to get the job done and put them first. Don't try to reduce into words items that are irreducible or inherently 2- or 3-dimensional.

Because diagrams and mockups take more time to make, they often end up at the end, or following the text. That is the first step on the well-intentioned road to Spec Hell. Mothers, don't let your development team grow up to do this.

Using Judgment

Design pictures can be napkin notes, line drawings, or Photoshop masterpieces... whatever gets the job done. Technically one might argue that only "whatever is the least that gets the job done" is appropriate, because a medium that is more elaborate (as in labor) than necessary is likely to include unintended information or subtle translation. This doesn't mean that the design work can't be done right in the development tools, but if it isn't someone with design skills, you'll be no closer to good specs than you would with the traditional up-front work.

This applies to depicting logic or flow: you must use a medium that presents the message in the fewest possible words or the spec will be inauthentic to the point of being unsupportable. As soon as the flow needs to change, the burden of updating the recipe will destine it to datedness and it will die on the spot.

Do This: When the solution is highly conditional, use a flowchart to document the conditional elements.

I'm guessing many folks consider flowcharts a "necessary evil" sort of thing, just like text-rich specs, or perhaps even more tortuous. But you gotta admit, the automatic connectors in Visio (and I assume other tools) make this once painful exercise more realistic.

Detail Level

The subject of detail level is a common sore point with specs. The now cliche strategy of the software development world is to limit functional specs to "the what, not the how" of the design. This is intended to give the developers some latitute to implement the best engineering solution and also prevent the sales and marketing folks from overstepping their bounds. (That's called protectionism.)

Challenge #3: Who could disagree that precision of instructions is valuable in proportion to the cost of mistakes?

Maybe detailed specs won't always prevent costly mistakes, but the converse can't possibly be more true. Where you draw your line between functional specs and other more detailed levels is actually immaterial. What matters is who is the sender and who is the recipient of each level of spec.

Who Are Specs For?

To my way of thinking, the word "requirement" should only be used in the context of documents sent from sales and/or product departments to the software development group. They require the system to deliver certain functionality. I recommend calling it just "Requirements," and leaving out the word "function" in any form. A requirements document could be very close to just a features list... even if it includes items down to the very fields of data that must be stored. Maybe an appendix is the right way to handle fields.

After the sales/product folks write up what they want, someone who writes for a living and also knows the technicalities of the product should clean up the document and send it back to them to make sure it is correct. Such a person is called a technical writer... you know, Dilbert's much maligned co-worker Alice. If you don't have Alice clean up the wording, you will just continue to complain that specs are just compliance docs.

The next document is your design document. You can throw the word "functional" in there if you want. Then you get to decide what gets pulled out as detailed design or database design. That's fine, whatever works for you, but from this point on those breakouts are immaterial to your success or failure at employing specs. No one really struggles with the detail level or quantity of documents that are used entirely within the software development group... all of the strife is 1) the line between requirements and design, and 2) how much design to forecast now that development tools are faster than planning, writing, and drawing.

Do This: On a project of any significant size, eventually document everything... but use judgment as to how much is needed in advance. The stuff you don't document in advance, you should go back later and put in your design docs. The better your cookbook, the better your cakes will be.

I suspect that the "functional specs" that we variously criticize are really dysfunctional specs... poor writing, missed points, circular references, and text deconstructed from images. To the extent that they include good information reverse-engineered from solutions, that is not a problem... that's just a byproduct of wonderful improvements in our tools.

See Also: A very short article delineating the various levels of specs

Revisionist History

• 2006-July-13: Edited more carefully
• 2006-Jun-27: Edited and retitled
• 2006-Apr-2: Created