7. Models & Methodologies¶

The suspense is terrible... I hope it'll last.
— Willy Wonka & the Chocolate Factory 

Now that we’ve discussed a full software development cycle, let’s step back and consider how to conduct it effectively. In this chapter we’ll discuss several popular models and methodologies used in the service of software engineering. Process models (sometimes called paradigms) are abstract concepts found helpful to focus and direct thought. They try to answer development questions such as what to do, e.g.: which phase of the development life cycle should be worked on and in what order? Methodologies and frameworks on the other hand are more concrete and specify how to do each step.  Factors in the choice of approach include requirements (and their clarity), project size, cost of change, time to market concerns, desired flexibility, and industry norms. 

Fig. 7.1 Social network analysis. ¶

Remember, there is no single development practice applicable to all projects that guarantees success. As always, it’s your job as an engineer to choose from among the right ones for the job at hand.

7.1. Code and Fix Model¶

Code-and-Fix is a reactionary, … risk-enriched approach.
— Lucky/Phillips, Software Project Management for Dummies

Fig. 7.2 No time to lose! ¶

This tried and not-so-true “model,” sometimes called “cowboy coding” doesn’t have much to say about how to run your project. Code is “whipped-up” and fixed until done. While it should be actively discouraged, this author won’t lie to ya—it works fine on small projects (and prototypes) with an experienced architect/developer at hand. But, projects tend to grow. Add multiple stakeholders, conflicting requirements, devs with differing experience levels, and it breaks down quickly. Code and fix often results in “big balls of mud”  at best, disaster at worst. The most common problems are that:

Fixes are more expensive to perform once code is deployed in the field.
Poor design makes fixes even more expensive.
Due to lack of communication , the end-product may be a poor match to user needs and expectations.

7.2. Waterfall Model¶

Don't go chasing waterfalls,
Please stick to the rivers and the lakes that you're used to…
I know that you're gonna have it your way or nothing at all,
But I think you're moving too fast ♪
— TLC 

Fig. 7.3 The SDLC as modeled under waterfall¶

You may have realized that previous chapters describing Lifecycle Planning were described in the form of a traditional, sequential waterfall model (with nods toward modern practice such as TDD, pair programming, and an autonomous QA team). Under waterfall work is completed in large batches. It proceeds step by step (with some overlap) from start to finish as each phase is completed, “falling down” as if it were water—the origin of the term. Wikipedia on Waterfall :

The waterfall development model originates in the manufacturing and construction industries: highly structured physical environments in which after-the-fact changes are prohibitively costly, if not impossible. Because no formal software development methodologies existed at the time, this hardware-oriented model was simply adapted for software development.

The model is still useful and applicable to situations where a project must be proven correct beforehand, such as the aforementioned skyscraper, shipping software widely on physical media, e.g. embedded firmware, cartridge, or optical disc that’s costly or impractical to fix in the field, compilers, security software, government or enterprise contracts, or perhaps flight software for a space probe never to return, all without a second chance to “get it right.”

However, it is a myth that deliverables are frozen at each step under waterfall  (see below). Even in the “old days” it was understood that delivering a useful product is more important than shipping garbage on schedule. This meant creating prototypes, going back, iterating, and fixing issues at each stage as needed. In chapter seven of the book Rapid Development Steve McConnell likens this process to a salmon swimming upstream—it isn’t impossible to backtrack, just difficult.

Fig. 7.4 “You keep using that word…” courtesy The Princess Bride .¶

The term in current usage however has become a synonym or catch-all term for a rigid and problematic development model that doesn’t fully exist—especially as described by Agile proponents, in order to emphasize the difference in focus. Statements like “that’s bad because it’s waterfall…” may be overheard. Often they’re describing a predecessor, the little-known stagewise  model, which froze deliverables, lacked prototyping, and recognition of feedback loops.

Yes, I did try to dissuade the sales rep that we were not using any waterfall, watering hole, water table, water can or water cooler methods but as you can suspect sales folks can’t sell you a solution if you don’t have a problem and he was really trying to create a problem.
Well of course we iterated and revisited tasks in previous phases.
— Prof. David Dischave 

Even so, many cracks were found early on. Though once dominant, a more pliable, realistic waterfall is still not flexible enough for many projects, especially those that moved on to “internet time” . Fred Brooks in “Don’t Build One to Throw Away—The Waterfall Model Is Wrong!” from The Mythical Man-Month (2nd Edition, 1995) wrote:

The waterfall model puts system test, and therefore by implication user testing, at the end of the construction process. Thus one can find impossible awkwardnesses for users, or unacceptable performance, or dangerous susceptibility to user error or malice, only after investing in full construction.

Despite newer paradigms and methodologies taking hold (which we’ll discuss shortly), the traditional phases of the life cycle continue on in iterative or remixed  forms. As such, the traditional SDLC continues to be a useful model of software engineering.

7.2.1. Emergent Properties¶

You’d probably want to tar and feather a teacher who gave you a programming assignment, then changed the assignment as soon as you finished the design, and then changed it again just as you were about to turn in the completed program. But that very process is an everyday reality in professional programming.
— Steve McConnell, Code Complete (Ch. 5)

Another significant problem with waterfall or other highly structured upfront development models is that for many projects, the full problem set is poorly understood at its start. Indeed, for innovative projects, full commitment to ill-fated requirements or design cements bad decisions, leading to unfortunate outcomes. A full team taking six months to discover that big mistakes have been made is just too damn costly, except perhaps for government work. Requirements may also transform for reasons out of the project’s control, e.g.: a change in the competitive landscape, or a fickle client:

I don't know what I want, but I'll know it when I see it!
— Anonymous

To describe this situation succinctly we say that the requirements in these cases are emergent—meaning that they are understood, evolve, and improve over time. Accordingly, the best problem definitions and solutions are often discovered as the work is done or even later, when software faces the real world for the first time.

Sub-Optimal Decisions

Often the true consequences of your architectural decisions are only evident several years after you made them.
— Martin Fowler 

Looking even further down the line, it’s often revealed that poor architectural decisions were made at design time. Unfortunately, there’s not a lot that can be done years after a project goes into production, where such lessons-learned either prompt a costly refactoring, are folded into the design of the next-generation, or simply suffered through.

Accordingly, over the years approaches that embrace change and focus on smaller batches or “iteration cycles” have become more popular.

Mind. Blown?

It’s recommended to pause here for a moment to contemplate the ramifications of the section you’ve just read—which are bigger then they might first appear. A re-read might be worthwhile.

In short, in some circumstances, everything you’ve read in the book up this point may not apply to your situation. In fact, some of the advice you’ve read may even be wrong for your next project! (WTF?) Consider the context of a project before diving in to a process blindly.

7.4. Spiral Model¶

Long before the Agile Manifesto, when I was working at NASA, we talked about the reality that the state of knowledge about a project is at its lowest at the beginning, and that any realistic project planning and management process must acknowledge and accommodate that fact.
— vannevar @ Hacker News 

Fig. 7.8 Simplified Spiral Model of development.¶

From “A Spiral Model of Software Development and Enhancement” Barry W. Boehm, TRW Defense Systems Group :

A primary source of difficulty with the waterfall model has been its emphasis on fully elaborated documents as completion criteria for early requirements and design phases. For some classes of software, such as compilers or secure operating systems, this is the most effective way to proceed. However, it does not work well for many classes of software, particularly interactive end-user applications. Document-driven standards have pushed many projects to write elaborate specifications of poorly understood user interfaces and decision support functions, followed by the design and development of large quantities of unusable code.

To alleviate these issues, the Spiral model performs a number of shorter, elongating, lighter-weight iterations through the life cycle during the project time-frame. It focuses on carrying out the most risky activities first (aka, the hardest parts), in order to discover problems and misconceptions earlier. It can be therefore described as an evolutionary, risk-driven development model (RDD anyone?), rather than a “document or code-driven approach.”

7.5. Agile Model¶

Even on well-managed projects, however, requirements change by about one to four percent per month (Capers Jones 2000).
— Steve McConnell, Code Complete (Ch 24)

Small Batches

Where as under the waterfall model our project might have one full iteration per year, and Spiral once a quarter, the Agile paradigm takes the idea that more frequent iteration is better to its logical end. While cycles of as little as two weeks are possible, one month is common to minimize excessive planning overhead.

Fig. 7.9 Iteration frequency comparison: Waterfall, Spiral, and Agile.¶

Iteration frequency is but one facet however. As described  by Wikipedia:

Agile software development is a set of principles for software development in which solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development, early delivery, and continuous improvement, and it encourages rapid and flexible response to change. Agile itself has never defined any specific methods to achieve this, but many have grown up as a result and have been recognized as being “Agile.”

Huh? What did all that mean?

Self-organizing:: Micromanagement from the “suits” upstairs is unnecessary for motivated pros to get their work done, thank you.
Cross-functional team:: Think team sports. Everyone needed to build a complete product (developers, designers, testers, admin/ops folks, customer representative(s), and project manager) working toward the goal.
Adaptive planning:: Often we don’t know what we need or want until we’ve seen a solution, lived with it a while, and experienced its flaws. Agile development acknowledges this reality, and aims towards better solutions sooner.

Continuous improvement:: Building on TQM, a goal to analyze and improve process and product frequently. Small features and bug-fixes land much more quickly than if they had to wait for a traditional development cycle. i.e. monthly instead of yearly.

While agile methods were developed in parallel during the late 80’s and 90’s as teams were thrown together to meet urgent need, the “Agile Manifesto”  was published in 2001. It states:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

The manifesto is based on the following twelve principles :

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity–the art of maximizing the amount of work not done–is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Hint: Back to Basics

In reaction to excessive Agile hype the resulting backlash, manifesto contributor Dave Thomas penned this piece, Agile is Dead (Long Live Agility)  and gave an interesting talk  about it:

Here is how to do something in an agile fashion. What to do:

Find out where you are

Take a small step towards your goal

Adjust your understanding based on what you learned

Repeat

How to do it:

When faced with two or more alternatives that deliver roughly the same value, take the path that makes future change easier.

And that’s it. Those four lines and one practice encompass everything there is to know about effective software development.

In brief, the Agile model shortens feedback loops  while encouraging the continuous improvement reminiscent of and complimentary to Total Quality Management (TQM).

7.5.1. Scrum Framework¶

Agile is a general philosophy regarding software development. Scrum is an implementation of that philosophy pertaining specifically to project management.
— RibaldEddie @ stackoverflow 

Fig. 7.10 A Rugby Scrum, courtesy AllBlacks.com ¶

Scrum is the most popular agile framework for software development, taking its name from the Rugby football  term for a cross-functional team pushing in a single direction.

Scrum (n): A framework within which people can address complex adaptive problems, while productively and creatively delivering products of high value.
Scrum employs an iterative, incremental approach to optimize predictability and control risk.
— The Scrum Guide 

Hint:

Agile and Scrum are not the same thing.

Rooted in empiricism , scrum is not a technique for building software, but instead “a framework that software can be built within.” Let’s begin with a few helpful definitions:

Story:

A feature request that describes who, what, and why in a concise manner. e.g.: “As a $ROLE, $FEATURE is needed to…” 

Sprint:

The basic unit of planned development time, a development cycle extending from one week to one month.

Backlog:

A list of potential things to do.

Blocker:

Any impediment preventing a dev team or individual from meeting their goals.

Done:

“It compiled” is not done—done refers to shippable software, ready for users, that’s been:

Built
Integrated
Tested (ahem)
Documented (ahem ahem)

Velocity:

The number of stories or “points” completed each sprint. The purpose of which is to aid in future estimation, not to be used as a performance indicator. As such, it should be converging to a horizontal line over time.

Fig. 7.11 The Scrum process, courtesy Wikipedia ¶

As seen above, the Scrum framework is relatively simple. Feature request stories are selected from the product backlog, added to a Sprint, worked on until complete, then released quickly to gather feedback and drive future decisions. It consists of the following roles, workflow, and artifacts:

Roles:

Product Owner (PO) — represents vested-interests (aka stakeholders) as the voice of the customer. Adds and manages feature requests into the product backlog in “story” format.
Dev team — cross-functional and self-organizing, limited to nine members (not including Owner and Scrum Master) to avoid communication overhead.
Scrum Master — coaches and facilitates product and sprint goals.

Artifacts (Work products):

Product backlog — a prioritized list of things to do.
Sprint backlog — a list of things to do in the current sprint.
Product increment — the sum of completed product backlog items.
Burn-down chart  — a “graphical representation of work left to do over time.” This also gives us an idea what the team’s velocity will be.

Sprint workflow and events:

Start: Sprint planning meeting — where the team plans the current sprint, selecting items (or portions of) from the backlog that can be completed.
1. What can be done this Sprint?
2. How will the chosen work get done?
Daily Scrum or “stand-up” — a short daily meeting (max: 15m) to share knowledge, and discuss three things:
1. Team tasks worked on yesterday
2. Team tasks to work on today
3. Blockers & Risks
Think huddle. First stand, yes up, to prevent excessive talking—no big discussions, thank you!  Blockers should be attacked immediately after the meeting. Chat may be an option for remote workers. 
End: Sprint retrospective — Review completed work (that is done) with Product Owner, reflect on past sprint, and discuss strategies for improvement.

What went well, what didn’t? Implement any improvements next cycle.

Limitations

Scrum is less useful in situations where the development team is scattered geographically, team members work on different tasks with different skill sets, and projects that have many external dependencies, or long histories. It is also a poor fit where quality control is paramount, large amounts of testing need to be completed before release, and ad-hoc decision making can be detrimental to quality objectives. 

7.5.2. Kanban¶

Kanban aligns inventory levels with actual consumption.
— Wikipedia 

Fig. 7.12 Simple Kanban board, courtesy Wikipedia ¶

Kanban, literally signboard or billboard in Japanese, is “a scheduling system for lean, just-in-time manufacturing,” that was originally developed at Toyota. Focused on inventory management, supplies and finished goods replace those just purchased—as happens on the shelves of a supermarket.

In a software context, Kanban also includes a signboard with several columns, To Do, In Progress, and Done, at its simplest. Cards or post-it notes describing units of work are attached to the board under the appropriate column. Task progress is visualized through the movement of cards to right until Done status is reached. It’s considered best practice to limit the number of items “in progress” at once to improve focus and limit multitasking. Additional columns are used to model more complex tasks. Below is an example common for Agile/Lean development, each post representing a work unit or story  .

Backlog	Ready	Coding	Testing	Approval	Done
	 	 		 	 
 		 		 	
 			 

As Compared to Scrum

Similarly to other Agile approaches, Kanban promotes a philosophy of learning, continuous improvement, and removal of obstacles, while encouraging self-organization. In contrast, it is more compatible with conservative organizations, as it does not prescribe titles, roles, or specific procedures to be followed, leaving existing structures intact. Interestingly, it can also be more flexible, as tasks can make their way through the pipeline at different speeds.

ryanmarsh, Agile coach on HN: 

Scrum is a batch oriented process. Batch processes are useful when the outcome of a process is uncertain and requires inspection and reconfiguration of the inputs after each batch until a stable and predictable process can be produced and the variables understood.

Kanban is a throughput oriented process. It is useful for reducing waste and optimizing for consistent and high output in a process where the inputs are already well understood and the output is of a more consistent quality.

Kanban’s workflow is compatible with Scrum, meaning the two can be used in concert. Alternatively, Scrum is a good choice while the problem space and design is being discovered, while Kanban can be useful later when things settle down.

Once you're in steady state maintenance/incremental improvement mode, kanban is probably the appropriate approach. But for initial product development, and major new feature releases or product re-writes, scrum with its measured discipline and time restrictions is better.
— vannevar at HN 

7.5.3. Extreme Programming/XP¶

The basic advantage of XP is that the whole process is visible and accountable. The developers make concrete commitments about what they will accomplish, and show concrete progress in the form of deployable software… This allows business-people to make their own commitments with confidence, to take advantage of opportunities as they arise, and eliminate dead-ends quickly and cheaply.
— Kent Beck

Fig. 7.13 COWABUNGA, DUDE! ¶

“XP” is a number of preexisting ideas grouped together under a dumb name. This methodology had the severe misfortune of debuting shortly after the “Extreme everything” (X-Games, MTV Sports  w/Dan Cortese, Extreme underwater-basket-weaving, Extreme yogurt-eating, Extreme wedgie) craze of the late 90’s. 

While surveying Agile techniques, we’ve been moving down from abstract models with lofty ideals, to frameworks to work within (helping to organize our work), finally arriving at more concrete methodologies. This is where we start to find out how to do things. XP focuses on delivering high-quality code in small increments. It values communication, simplicity, feedback, courage, respect among participants, and works best with teams of twelve or fewer.

Extreme Programming is successful because it stresses customer satisfaction. Instead of delivering everything you could possibly want on some date far in the future this process delivers the software you need as you need it.
Unproductive activities have been trimmed to reduce costs and frustration of everyone involved.
— extremeprogramming.org 

XP’s 12 Core Practices, courtesy Lee Copeland at Computerworld :

Customers define application features with user stories.

Small code releases are put into production early.

A common system of names and descriptions is used.

Teams emphasize simply-written code that meets requirements.

Unit tests are written upfront and run throughout.

XP teams frequently revise and edit the overall code design, a process called refactoring.

Programmers work side by side in pairs, continually seeing and discussing each other’s work.

All programmers have collective ownership of the code and the ability to change it.

Code is integrated and released to a repository every few hours and in no case hold on to it longer than a day.
// rec. reduced

Programmers work only 40 hours per week; there’s no overtime.

A customer representative remains on-site throughout the development project.

Programmers must follow a common coding standard so all the code in the system looks as if it was written by a single individual.

That’s a good introduction. There are other practices and benefits, which are broken down into categories below :

Improved Feedback:
- Pair programming
- Planning game 
- Test-driven development
- Customer/rep on the team.
Continuous process:
- Small releases
- Continuous integration of code to find issues early.
- Refactoring or design improvement
Shared understanding:
- Coding standards
- Collective and egoless code ownership
- Simple design

Coding:
- Implement the simplest thing  that could possibly work.
- Customer is always available.
- One pair integrates at a time
- Optimization comes last.
Testing:
- All code must have unit tests.
- All code must pass all unit tests before it can be released.
- When a bug is found tests are created before the bug is addressed.
- Acceptance tests are run often and the results are published.
Programmer welfare:
- Sustainable pace
- No overtime

XP Criticisms

Many types of system projects require upfront planning to succeed. When we avoid “excessive” requirements gathering and design planning, we risk the same disasters as those who didn’t do them before Agile.

Also, because of the interdependence of XP practices, each of which support the others, it has been criticized as a “daisy-chain of poisonous snakes.” If one gets loose (meaning an XP practice is skipped), pain may follow. In other words, an XP project may become “house of cards” that falls if not performed adequately. 

7.5.4. Lean Startup¶

There is nothing quite so useless, as doing with great efficiency, something that should not be done at all.
— Peter F. Drucker, Management Educator 

Fig. 7.14 What the customer really needed.  ¶

The “lean startup” methodology, sometimes described as discovery-driven planning, supports continuous innovation. It is a set of ideas built on top of the lean manufacturing and total quality management movements discussed previously. In short, Agile concepts are applied to entrepreneurship to accelerate the discovery of customer wants and needs, in order to avoid the waste of constructing products or functionality no one wants.

That was a mouthful, so lets unpack it a bit. When dreaming up new products, we engineers love to brainstorm about how to build them. Ok, but that’s not the first question to consider, is it? Rather, it is “should this product be built?” and (if a commercial endeavor) “can we build a sustainable business around it?” As theleanstartup.com  explains:

Too many startups begin with an idea for a product that they think people want. They then spend months, sometimes years, perfecting that product without ever showing the product, even in a very rudimentary form, to the prospective customer. When they fail to reach broad uptake from customers, it is often because they never spoke to prospective customers and determined whether or not the product was interesting. When customers ultimately communicate, through their indifference, that they don’t care about the idea, the startup fails.

Instead of building in isolation, the lean startup method requires the product be exposed regularly to potential customers to test that the assumptions it is built upon are valid. That’s the gist of it, a set of techniques to gather more rapid feedback, to “fail faster,” thereby finding our way to success earlier .

Minimum Viable Product (MVP)

If we do not know who the customer is, we do not [yet] know what quality is.
— Eric Ries, The Lean Startup (Ch. 6) 

Fig. 7.15 Build-Measure-Learn feedback loop¶

To that end, the goal of an MVP is to “test business hypotheses and begin the learning process as quickly as possible.”   The simplest product that could work is built to demonstrate an idea and is tested against potential customers under a rapid Build-Measure-Learn feedback loop.

Feedback / Validated Learning

Under lean startup, we aim to handle feedback as quickly as we would the steering wheel of a car, in contrast to the meticulously planned launch of the Space Shuttle.  If the product demo doesn’t fit with customers,  it may be improved or abandoned depending on how close it hit the target, a process author Eric Ries calls “validated learning.” Unnecessary, costly work on the wrong thing is avoided.

Unfortunately, it can be difficult to find potential customers willing to give feedback and contribute to a minimal, pre-release product, one of its main criticisms. 

A Success Story

This "prove-everything-at-each-step" side hustle method is how most businesses should be started.
— codyzazulak1 at HN 

So, how to find out what we need to know, when folks are not inclined to help? Here’s a wildly successful example of how to test a business idea with a low upfront investment, “…an Internet Millionaire in His Spare Time”  at Bloomberg with associated discussion.  In short, before taking the plunge, the subject sold his hypothetical product idea on Facebook, refunding orders that came in. As they did, desire was demonstrated. Later, the first production batch was crowdfunded through Kickstarter.

7.6. Criticisms¶

Beware of Methodologies. They are a great way to bring everyone up to a dismal, but passable, level of performance, but at the same time, they are aggravating to more talented people who chafe at the restrictions that are placed on them.
— Joel Spolsky, "Big Macs vs. The Naked Chef" 

Unfortunately, these “big-M” Models and Methodologies may or may not work for your team. While they often satisfy the needs of the smart folks that invent them, there are no guarantees in this world for an inexperienced team “painting by the numbers.” 

The important point is to use your brain to match method to circumstance, rather than blindly following directions. Experience makes this possible. 

7.7. Conclusion¶

My fellow engineers, please stop asking "Is this process good or bad?" and start asking, "Is it well-suited to my situation?"
— Jocelyn Goldfein, Director of Engineering and Investor 

As we’ve seen, in the frequent cases where project requirements are vague and temporary failure not catastrophic, industry is moving towards iterative styles of development—reducing both cost and risk by uncovering problems and poor design more quickly. The Agile model and associated methods and frameworks don’t solve engineering issues per se. Rather, they increase their resolution—the visibility and insight into the health of a project as it moves forward.

Tip: Balance

Seek out the right proportion between upfront planning and late flexibility for every project.

Use methodologies to your advantage, avoid unthinking dogma.

TL;DR

Process models focus on what to do and in what order, while methodologies specify how to do it.
A “code and fix” strategy often results in a “big ball of mud” or worse.
Requirements (and design) of innovative projects are emergent.
If the problem set of a project is well understood, requirements are stable and risks are low, and failure/fixes/changes are costly to perform after the fact, a sequential development model such as waterfall may make sense.
If a project has important risks to manage and requires more feedback, the Spiral Model would be a good choice.
If requirements are vague/challenging/not understood, if you’re breaking new ground, if the product needs to respond to competition, and cost of change is low, choose an iterative approach, such as a methodology complementary to the Agile Model.
Scrum is a structured project management framework that software can be built within. Managers and business owners enjoy its increased predictability.
Kanban is a technique that promotes “just in time” delivery and self-organization.
A Minimum Viable Product, or MVP is the simplest product that could work. It is built to demonstrate an idea for testing against potential customers, to find a solution they’ll pay for, in the shortest amount of time.
Consider the application or balance of multiple approaches when managing a project. Don’t be afraid to throw out facets of them that aren’t applicable or working.