2. Requirements¶

Effective software design begins with the establishment of clear, achievable, and unambiguous requirements.
— IEEE Computer Society, on Requirements 

In brief, requirements are the stated rules of what a future software project must accomplish and support, and occasionally what it must not support. The needs of the client discovered here are intended to be met through software. In order for a project to be successful, they must be documented as accurately as possible. An explicit requirements step helps ensure that clients and users, rather than developers, drive functionality.

Fig. 2.1 What the client asked for. ¶

The requirements phase is one of the most important for a project, yet often overlooked as bureaucracy by the inexperienced. However history tells us a significant portion of projects that fail,  fail early on this, their first step.  It is not hard to imagine that regardless of the quantity and quality of “ninja/rockstar” code cranked out in later phases, none of it matters if the project doesn’t deliver what the client needs.

Get the project’s requirements wrong and when lucky, you’ll only have to go back and redo weeks or months of work (D’oh!  in Fig. 1.3). Not-so-lucky, and the whole project could be flat-out canceled as a failure.

On the other hand, taking the task seriously is not a guarantee of success either. The real-world requirements of a project may not be fully understood at its conception, misunderstandings may and will occur, the client may change their mind in-process, and regrettably often do.

Requirements Risk

Excerpted from Professional Software Development Ch. 1, by Steve McConnell:

Recent surveys have found that the most frequent causes of software project failure have to do with requirements problems—requirements that define the wrong system, that are too ambiguous to support detailed implementation, or that change frequently and wreak havoc on the system design.[9] But requirements problems are not new. Back in 1969, Robert Frosch observed that a system could “satisfy the letter of the specification and still not be very satisfactory.” [10]

[9] Cole, Andy, 1995. “Runaway Projects—Cause and Effects,”

Software World, Vol. 26, no. 3, pp. 3-5

Dennis, MA, “Charting the Seas of Information Technology,”

The Standish Group, 1994.

[10] Frosch, Robert A. “A New Look at Systems Engineering,”

IEEE Spectrum, September 1969.

2.1. Considerations¶

The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is so difficult as establishing the detailed technical requirements, including the interfaces to people, to machines, and to other software systems.
No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.
— Fred Brooks, The Mythical Man-Month, 2nd Ed. (Ch. 16)

Now that sufficient fear has been instilled, there are a few considerations to note before proceeding.

2.1.1. Problem Definition¶

If a good problem definition hasn’t been specified, you might be solving the wrong problem during construction.
— Steve McConnell, Code Complete (Ch. 3)

Problem Definition is often not a formal step but rather a simple exercise interviewing the client, thinking, and writing a few paragraphs about the problem at hand to be solved. Other terms for this step may include a product definition or problem statement. This documentation may be folded into a project charter by a project manager.

When defining problems, it’s best to state them in general terms as clearly as possible and avoid mention of potential solutions. Additionally, do not use jargon —the problem definition should be readable by everyone.

Fig. 2.2 Solving the wrong problem.   ¶

Tip: The Five Whys

The root cause of any problem is the key to a lasting solution.
— Taiichi Ohno, Former Executive VP, Toyota Corp.

When dealing with problems that may at first appear nebulous or uncertain, pushing on a few steps further and asking the five whys  often improves clarity considerably. An effort is made to find the root causes  of a problem rather than merely the symptoms, by repeatedly asking why, like a five-year-old . This problem-solving technique originated at the Toyota company in Japan.

As an example, imagine a welding robot stops in the middle of its operation, what are the root cause(s)? (courtesy Toyota )

“Why did the robot stop?”
The circuit has overloaded, causing a fuse to blow.

“Why is the circuit overloaded?”
There was insufficient lubrication on the bearings, so they locked up.

“Why was there insufficient lubrication on the bearings?”
The oil pump on the robot is not circulating sufficient oil.

“Why is the pump not circulating sufficient oil?”
The pump intake is clogged with metal shavings.

“Why is the intake clogged with metal shavings?”
Because there is no filter on the pump.

The “five whys” technique is useful at every stage of the life cycle where hard questions come up and complete solutions are needed; use to your advantage.

2.1.2. What Are We Building?¶

In the classic book on software project management, The Mythical Man-Month, Fred Brooks describes the concept of the “Programming Systems Product,” to differentiate it from simple programs. Management may not realize the differences between a program, a system, and a product, and therefore may not be cognizant that a full “systems product” (aka Platform in modern parlance), can take up to an order of magnitude longer than a corresponding program. For this reason they may press for an unrealistic schedule. The table below illustrates:

Program	Programming System (3x)
Programming Product (3x)	Programming Systems Product (9x)

Starting in the top left quadrant, we have a program. According to Brooks’ findings, moving to the right one column to build a “programming system” takes approximately three times as much effort as a single program, due to the necessity of building multiple compatible programs, interfaces, and libraries. Moving downwards one row to the creation of a product (bottom-left) also takes about three times as much effort as a program, due to increased requirements for documentation, quality, maintainability, and general polish for customers. As the work categories multiply, the resulting system-product (shown at bottom right), was found to take nine times longer.

This gets us to a first approximation when figuring what we are trying to build. Intended audience is considered below, while various criteria are weighed in the next chapter on Software Design.

2.1.3. WTF are Key Stakeholders?¶

Fig. 2.4 Let’s open a dialog… ¶

Warning

Apologies, the suit-speak  gets thick in this section. Tread carefully.

In this section we’ll discuss “buy-in” from “key stakeholders,” uggh, phrases only one notch below the dreaded “synergy.” Stakeholders  of course are the folks that care about the project and have a vested-interest or role, which affect or are affected by it. The word itself refers to the practice of driving of a stake into the ground to claim territory, hence the phrase “having a stake.” Common members include:

Customers, clients, and their representatives
Software users
Organization management
Project management and sponsors

Development and Quality Assurance teams
External suppliers, subcontractors, and consultants
Grouchy systems and database administrators (who too rarely get consulted).

Stakeholders must be identified and prioritized to make certain an appropriate amount of attention is given to them and their expectations are met.

Once all relevant stakeholders are identified, we try to determine where the power lies.
— Merrie Barron, PMP, CSM 

Key stakeholders are those directly affected by the project and whose approval is of primary concern, without which the project may be at risk of failure or cancellation. Secondary, sometimes hidden stakeholders are those the project affects or impacts indirectly. Unfortunately, they may be overlooked, a situation that creates unexpected risks.

While the term stakeholder is mildly cringe-worthy, “vested-interest affected by the project” doesn’t roll off the tongue as easily—so it looks like the word is here to stay. “Buy-in” refers to getting their full support of course. While an important concept and necessary for project success, don’t use that phrase around developers (haha), unless you’d like to get kneecapped .

Keeping in touch with stakeholders is an important part of your job, to say the least. For big projects certainly, but what about internal company projects? You bet. From “How to Train Your Microservices,” at Built To Adapt: 

So now, I’ve gotten into a habit: before I even write a single line of code, I interview everybody that potentially will use the solution that I’m going to write, and I keep them in lockstep with me and my team just about every week. We keep them engaged, helping to influence the direction I’m trying to echo out in code. It’s gone so much better, because they feel invested. They don’t feel like in six months I’m revealing this big, mysterious thing. They feel like this is just something they’ve seen through iterations. And what’s empowering about that, too, is if you can get the spiritual leaders of the different departments that you’re trying to encourage to use your solution, they’ll help sell it for you.

2.1.4. Take Your Time (Do It Right)¶

The sooner you start to code, the longer the program will take.
— Roy Carlson, University of Wisconsin 

You’ll be tempted to hurry through these so-called “upstream activities,” (Definition, Requirements, & Design) so that you can quickly get to the fun stuff. Bad idea. Performing them properly will save several times what it’s going to take to change direction later on. A great old-school  jam comes to mind:

It seems we never take the time to do…
All the things we want to, yeah
Now, baby we can do it, take the time, do it right… ♪
— The S.O.S. Band 

Fig. 2.5 You know you ought to slow down… ♪¶

Cue that up (“the Man” no longer allows song embedding ) and consider the often overlooked tasks that late changes to a project entail, besides construction:

Meetings to coordinate
New design work
Fixes for resulting code breakage
Fixes for broken tests
Revising/rewriting obsolete documentation
Restarting the QA process
Project rescheduling and additional time pressure

Most of this costly rework can be avoided with a little diligence  up front—taking the requirements phase seriously. Remember the old saying, “if you don’t have time to do it right once, how will you find the time to do it twice?”

2.2. Process¶

A customer walks into a bar. He asks for a beer made out of wine. The project manager agrees.
Both question the bartender's competence.
— Daniel Méndez, Software Engineering Fellow 

Fig. 2.6 Requirements phase sub-steps¶

The sub-steps of the requirements phase are as follows:

Problem Definition (above)
Elicitation
Analysis & negotiation
Documentation
Validation
Maintenance

2.2.1. Elicitation¶

Elicitation  is working with the client, users, and others with a vested interest in order to understand their needs. The word elicitation highlights the fact that a bit of detective work is often needed to draw out the complete requirements. Sources include stakeholders, business goals and policies, the problem domain, and occasionally regulatory law. 

Additional human factors that must also be considered  for a successful project:

All vested-interests (aka “stakeholders”) must be involved early and significantly, or they may not support the project. In rare circumstances they may even work against it, ensuring failure.
Conflicts between clients or departments happen and must be resolved.
Frequent communication with domain experts may be necessary.
Discovery of hidden requirements may be necessary.

From Software Requirements (3rd Ed.), by Karl E. Wiegers:

Requirements elicitation typically takes either a usage-centric or a product-centric approach, although other strategies also are possible. The usage-centric strategy emphasizes understanding and exploring user goals to derive the necessary system functionality. The product-centric approach focuses on defining features that you expect will lead to marketplace or business success.

A risk with product-centric strategies is that you might implement features that don’t get used much, even if they seemed like a good idea at the time. We recommend understanding business objectives and user goals first, then using that insight to determine the appropriate product features and characteristics.

Types of Requirements

Elicited requirements may fall into one of the following categories:

Product: A constraint on the software itself.
Process: A constraint on the development of the software.
Functional: Necessary functionality or features.
Non-functional: Constraints on quality, performance, safety, maintainability, etc.
System: When the software must work in concert with a larger system, platform, network, etc.

2.2.2. Analysis & Negotiation¶

If you don't know where you're going, you will not get there.
— Andrew S. Grove, founder/former CEO Intel, "High Output Management" (Ch. 6) 

Information gathered during elicitation is considered and classified at this stage. Impractical or unrealistic goals need to be identified for modification, additional stakeholders may be identified, and any conflicts should be sought out and resolved.  Repeatedly asking “why?” to get to the core business needs of each requirement will improve chances at success. Knowing the real purpose of the project will allow for better decision-making when disagreements and feature requests come up.

It’s not uncommon for requirements to be given that aren’t practical to accomplish, either due to the open nature or intractability of the problem, lack of time, resources, and budget (all phrases for the same thing). This is where negotiation process begins.

2.2.2.1. Prototyping¶

Prototyping may be done during the Requirements or Design phases and is helpful for ferreting out issues or flaws in the modeling of the problem. Prototypes can drive discussions in ways that words alone cannot, generating not just answers but new questions as well. As the cliché goes, a picture is worth a thousand-words. A survey of case studies affirmed that prototyping can lead to improved product fit in regard to user expectations, with improved designs. (Gordon and Bieman, 1991  )

Use of low-fidelity “mockup” or wire-framing software instead of conventional UI libraries can avoid a focus on cosmetic issues during the prototype sub-phase. The focus is on creating rough sketches to illuminate functionality and “user stories” without getting bogged down in unnecessary detail, and is analogous to story-boards in the film world.

Popular mockup software:

Balsamiq  (Proprietary)
Invision  (Proprietary)

Warning: The Iceberg Secret

You know how an iceberg is 90% underwater? Well, most software is like that too—there’s a pretty user interface that takes about 10% of the work, and then 90% of the programming work is under the covers.
That’s not the secret.
The secret is that People Who Aren’t Programmers Do Not Understand This.
— Joel Spolsky, Blogger/co-founder StackExchange 

Fig. 2.7 “Every night in my dreams, I see you, I feel you… ♪”   :-P¶

A few words of caution when dealing with suits and non-programmery types in general regarding your mockups. These folks don’t know that the user interface is just the “tip of the iceberg” of your project, hence the “wire framing” recommendations above. Take heed—don’t produce pretty mockups or they’ll think the project is done before it starts!!! 

Important Corollaries:

If the UI looks unfinished, they’ll think it is unfinished.
If the UI looks finished, they’ll think it is very close to being finished.
Give management several versions of the graphic design to choose from, to narrow input to safe subjects.
A polished web site will attract more investment than a much more functional one.
When demoing, make screen shots beautiful.

Solutions:

Build your UI so that unfinished parts look unfinished.
Leave out features that are not implemented, such as menus buttons, sections, etc. As complete features are added in, progress will be perceived more accurately.

2.2.3. Documentation¶

The overarching goal of preparation is risk reduction.
Part of a programmer’s job is to educate bosses and coworkers about the software-development process, including the importance of adequate preparation before programming begins.
— Steve McConnell, Code Complete (Ch. 3)

Written specification of information obtained through elicitation and analysis is recorded in this step, which produces a document formally called a Software Requirements Specification (SRS) . A conceptual data model  (CDM) should be prepared as well, discussed in the next chapter.

Many of the qualities of successful requirements documents are listed below :

Clarity:: Writing and critical thinking skills, FTW !
Complete:: Must address relevant cases, features, and users.
Concise:: Partially at odds with completeness, this quality facilitates understanding without drowning the reader in unnecessary detail. Don’t over-document (unless paid by the page of course).
Consistent:: Requirements must not contradict each other. They must be compatible logically, strategically, and grammatically. 
Achievable:: Unless you’ve a breakthrough design for a flux capacitor , your time-travel project will fail.
Unambiguous:: If two people may interpret the specs differently, they will. This quality is especially important on platform or protocol work, on top of which others will build.
Verifiable:: Measurable goals should be stated, rather than “works well”, etc.
Prioritized:: Rank the relative importance of each requirement.
Appropriate:: Consider the scale of the project in question when writing. More lengthy and formal processes will likely be required as the project grows.
Version Controlled:: Requirements docs should be in a version control system with other docs for continued maintenance.

Protip: When documenting, involve a technical writer when possible.

Hint: Traps

Software Requirements: 10 Traps to Avoid, courtesy processimpact.com :

Confusion about what a requirement is

Inadequate customer involvement

Vague and ambiguous requirements

Unprioritized requirements

Building functionality no one uses

Analysis paralysis

Scope creep

Inadequate requirements change process

Insufficient change impact analysis

Inadequate requirements version control

Templates

Below are templates and guidelines for requirements documentation, which may be helpful to get started.

2.2.4. Validation¶

One of the most common causes of defects—ambiguous requirements—can be prevented by writing comprehensive acceptance tests when each requirement is captured.
— Clarke Ching @ StickyMinds.com 

Fig. 2.8 Unclear requirements, courtesy Barron & Barron   ¶

In this step the produced documentation is reviewed by the team (both customer and developer) with a skeptical eye, looking for inconsistencies, problems, lack of clarity, and missing cases. Requirements must be verifiable, and may be validated through the authoring of acceptance tests  or checklists. 

Requirements should be iterated upon until reaching a level of quality and detail that allows the Design phase to begin in earnest.

Cost Estimation

…In nearly all software development projects the cost of implementation is one of the key factors in evaluating requirements.
Otherwise how does the person funding the project decide whether or not to approve it?
— DougWebb @ HN 

While we’ll discuss the subject of cost in more detail in the Software Quality and Project Management chapters, and time-based Estimation further as well, it is useful to consider cost of implementation here briefly as a chief driver of Requirement acceptance and determination of project feasibility. The more costly a requirement and its estimated implementation is, the less likely it is to be accepted. Of course accurate estimates are a necessary prerequisite and can make or break a project.

Return on Investment

From Software Requirements (3rd Ed.), by Karl E. Wiegers:

If possible, quantify the contribution the feature makes towards the business objectives, so that people can make scoping decisions on the basis of facts rather than emotions. Will a specific feature contribute roughly $1,000, $100,000, or $1,000,000 toward a business objective? When an executive requests a new feature that he thought of over the weekend, you can use quantitative analysis to help determine if adding it is the right business decision.

2.2.5. Feasibility Study¶

How do you cancel a project early? The standard means is by conducting a feasibility study early in the project to determine whether the full-scale project is workable.
— Steve McConnell, IEEE Software Best Practices Column 

But, is this project feasible? Good question. After requirements have been validated, a feasibility study  should be conducted. In short, it’s a process where technical, economic, legal, and operational factors are considered alongside the requirements, in order to decide whether it actually makes sense to move forward with the project.

2.2.6. Maintenance¶

While it is often imagined that the requirements documentation is “done” as we move into the next phases of the traditional lifecycle, requirements gathering should be an iterative process, with documentation maintained as such. As understanding is improved the documents will need to be revisited to reconcile with new realities.