How software developers suffer from the illusion of requirements

Proponents of Agile development have long argued that users are not capable of clearly stating a comprensive, accurate, consistent set of requirements for a desired system. The truth is more problematic. Most software projects have no requirements to state – at least, that is theme of new research in Requirements Engineering.

Suppose we’ve been hired by a hospital. The hospital has just finished digitizing all patient records. Physicians need to access patient records on the computers in examination rooms but, due to privacy legislation, records must be visible only to the physicians. That is, a nosey patient shouldn’t be able to read someone else’s information over the physician’s shoulder. Based on this description, we devise two approaches – 1) information is displayed using codes that only physicians can decipher; 2) a matching pair of filter screens and glasses installed such that the screen can only be read by someone wearing the glasses. The requirements, therefore, are the features common to both approaches. Filter-glasses, for example, are not required because the secret codes approach doesn’t need them.

The first problem is that these two approaches share few features. They both involve screens displaying patient information, but that’s about it and this is too general to drive the design of either the codes system or the filter system).

The second problem is that, even if these two options share many features, there could be a third option that we haven’t thought of. For example, we could give physicians tablets, which could simply be placed face down when not in use to avoid prying eyes. Therefore, we cannot say with certainty that X is a requirement unless we are sure that we have considered every possible solution.

The third problem is that unlike the hospital example, many software projects begin with unknown, ambiguous or conflicting goals. Trying to state a set of necessary conditions for success where we can’t define success is meaningless.

To state a single requirement, therefore, we need an unambiguous goal on which everyone agrees, have enumerated every possible solution, and all solutions must overlap. Good luck with that.

Of course, if you demand that an analyst produce a set of system requirements, most unemployment-fearing analysts will produce a set of statements labelled requirements. And that’s where the real problems begin. A wide variety of cognitive phenomena including system justification, miserly information processing and status quo effects lead developers to accept that most statements labelled requirements must be, in fact, necessary for success. This “requirements illusion” powerfully curtails innovation and creativity.

The solution is to accept that many projects simply have no requirements. This entails abandoning tendering for software projects, creating flexible contracts and attempting to understand what users do rather than what they want.