6. Release & Maintenance

Software being "Done" is like a lawn being "Mowed."
— Jim Benson‏ @ourfounder 
_images/clipper.jpg

Fig. 6.1 “Ship it!”,  Clipper ship ‘Oriental’ 

This chapter marks the closing of the traditional software development life cycle—the end of a project and birth of a new software product. We’ll follow it from release, through the postpartum period of support and maintenance, until retirement.

6.1. Release

Real artists ship.
— Steve Jobs 
_images/cell_11.jpg

Fig. 6.2 What marketing advertised.  

Yes, they do (as the author is painfully reminded while trying to finish this damn book). So, the time has finally come in to ship the product. Is it good enough? Is it… done? 

Prelude

By this time, new feature development will have frozen  long ago, shifted in favor of stability, bug-fixing, and general polish. Planned testing will have completed with all known defects documented and listed in a database for final analysis. The most important requirements and quality objectives, which were documented at the start of the project, will have been validated through acceptance testing. Unless the project proceeded perfectly, a few less-important (or difficult) features will likely have been cut, backed out, disabled, or postponed by the time the project approaches release.

6.1.1. Release Criteria

In the competitive commercial software market, software companies feel compelled to release software the moment it is ready. Their task is treacherous, treading the line between releasing poor quality software early and high quality software late.
A good answer to the question, "Is the software good enough to release now?" can be critical to a company’s survival.
— Steve McConnell, IEEE Software Best Practices Column 

Where should we be?

To make an edumacated decision, we’ll first need release criteria, the list of objectives that need to be completed prior to shipping. Unless they’ve been specified properly, there won’t be concrete goals to work towards. In short, they’ll be project requirements (at agreed quality), minus any cut features.

Where are we?

Without stability and measurable data it isn’t possible to know with any degree of certainty where the project is in comparison to its goals and therefore make a rational decision regarding whether to ship.

What’s left to do?

When criteria are set and functionality/quality measured, everyone on the team can see progress and knows when the project is ready to release. 

_images/heyzooss.jpg

Fig. 6.3 Hey! You’re good!  

Once the preferred release-date has been scheduled, what’s left to do? Quality goals, such as the elimination of blocking issues (showstopper/data-loss bugs) or restrictions on the number of open bugs, are met. Open bugs that can’t or won’t be fixed by the deadline (in support of stability) are postponed as well.

See also:  Online Resources

Release Criteria: Is This Software Done?, courtesy Johanna Rothman :

  1. Define success.

  2. What’s important for this project?

  3. Draft release criteria.

  4. Make release criteria SMART.

  5. Gain consensus on release criteria.

Example release criteria:

  • All code must compile and build for all platforms.

  • Zero high priority bugs (Zarro Boogs )

  • All open bugs documented in release notes with workarounds.

  • All planned QA tests run, at least ninety percent pass.

  • Number of open defects decreasing for last three weeks.

  • Feature X unit tested by developers, system tested by QA, verified with customers A & B before release.

  • Ready to release by June 1.

Ship it!

Now witness the firepower of this fully armed and operational battle-station.
— Emperor Palpatine, "Return of the Jedi" 

During the final hours, the full team will typically perform last-minute ad-hoc testing until the “lever is pulled.” Congrats! Let’s take it for a spin:

“Fire at will, commander!” (@ 1:07)


6.2. Continuous Delivery

Continuous delivery takes automation from source control all the way through production.
— Wikipedia 

As an alternative to the one-time or infrequent release schedule described above, we have continuous delivery, also abbreviated as CD. CD is the rapid delivery of release-candidate level code fit for deployment at potentially any moment throughout its history. Background from “Continuous Delivery” by Jez Humble and David Farley follows :

In the late 90’s I paid a visit to Kent Beck, then working in Switzerland for an insurance company. He showed me around his project and one of the interesting aspects of his highly disciplined team was the fact that they deployed their software into production every night. This regular deployment gave them many advantages: written software wasn’t waiting uselessly before it was used, they could respond quickly to problems and opportunities, and the rapid turn-around led to a much deeper relationship between them, their business customer, and their final customers.

CD builds on the foundation of best practices that have developed over the years to shorten the feedback loop during development of a project:

Benefits of Continuous Delivery

CD can benefit projects where it’s appropriate in the following ways   :

  • Faster time to market—new features are not tied up in the development pipeline.

    • Quicker feedback on new features

  • Low risk releases—more frequent releases are by definition smaller and less-likely to break.

    • Release is practiced often and regularly, therefore process issues get fixed.

    • Releases are no longer scary.

    • Happier teams, through reduced frustration and stress.

  • Lower costs—through investment in build, test, deployment and environment automation.

Continuous Deployment

Continuous deployment is simple: just ship your code to customers as often as possible.
— Timothy Fitz 

Continuous delivery is related to but often confused with continuous deployment, which goes one step further—the frequent push of builds to “production,” meaning that customers are now directly working with “freshly built” software. In other words, the builds are not just ready to be deployed at any moment, they are deployed! This strategy can make sense in environments with a higher risk tolerance.

Remember: continuous delivery of candidate builds, continuous deployment to production.

See also:  Books

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, by Humble & Farley

There are a number of great nuggets of wisdom in this book, the only drawback is that it is a bit repetitive. Worth the read, but you’ll find yourself skimming the later chapters.

See Also:  Online Resources

The cycle time from conception [of a feature] to production has decreased from several months to two to five days.
— Lianping Chen 
  • Continuous Delivery: Huge Benefits, but Challenges Too, by Lianping Chen. This piece  is a case study on how to move an existing project from an infrequent release schedule to CD.

  • Continuous Delivery vs Continuous Deployment, by continuousdelivery.com 

  • Software Inventory, by Joel Spolsky   

6.3. Maintenance

Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance.
— Kurt Vonnegut, Jr. 
_images/cell_09.jpg

Fig. 6.4 How the customer was billed.  

Once a software project has been shipped and exercised by users in the real world, bugs surface, new requirements emerge, and underlying platforms shift over time. The maintenance phase of the product’s life-cycle begins. While not as glamorous perhaps as some of the other phases, it’s an important consideration for customers in regard to total cost of ownership (TCO ); with necessary work typically financed through support contracts.

Often the same team of developers that created the project will maintain it, supporting several lines of development (branches) simultaneously. In other circumstances, a separate team may take over maintenance. Work may even be outsourced to an external entity. The maintainer is the individual or organization that maintains the software.

Building vs. Maintenance

One of the things they don’t get around to mentioning at University (and if they did, may have been ignored), is that the majority of the development work in industry is in maintenance. That is, patching, updating, and fixing other’s already-written code.

New design and engineering, sometimes called greenfield development , i.e. where the “fun” is, is a bit more rare. Still, software maintenance is a useful learning activity for junior engineers, both in learning what to do, and in some circumstances what not to.

6.3.1. Considerations

The fundamental problem with program maintenance is that fixing a defect has a substantial chance of introducing another.
As a consequence… program maintenance requires far more system testing per statement written than any other programming.
— Frederick P. Brooks, Jr., The Mythical Man-Month (Ch. 11)

If the maintenance team is not the same as the development team, comprehension of the code base can be an issue. Research has shown that approximately half of total effort is dedicated to understanding the software to be updated . This is the point where time spent on readability, documentation, and writing tests earlier pays great dividends. “Pay it forward,”  as they say.

The First Rule…

The objective of software maintenance is to modify existing software while preserving its integrity.
— SWEBOK v3, Ch. 5

Khaaaaaan!!! 

The first rule  of software maintenance is, don’t break anything!  Yes fix bugs of course, but don’t break existing functionality. If you do, the Wrath of Khan  will rain down from above.

Suprise Prevention

Best practice allows only fixes to a maintenance branch, with enhancements (new features) and changes to interfaces prohibited—and only considered under extreme circumstances. The reasoning should be clear from the paragraph above, the product must remain stable. That’s what customers pay for, surprise prevention. It’s just too risky to do otherwise.

Maintenance tasks fall into the following categories, courtesy of ISO/IEC/IEEE 14764   :

Corrective:

Reactive modification of a software product performed after delivery to correct discovered problems.

Adaptive:

Modification of a software product performed after delivery to keep a software product usable in a changed or changing environment.

Perfective:

Modification of a software product after delivery to improve performance or maintainability.
(Generally recommended to be done in a new version)

Preventive:

Modification of a software product after delivery to detect and correct latent faults in the software product before they become effective faults.

More simply, the reasons for a maintenance team to release updates to a shipping product are:

  • Fixing functionality, that never worked or no longer works correctly.

  • Closing security holes

Somewhat rarely you may see an important feature “backported”  to a previous release, as may happen under a pricey support contract. The port must not interfere with existing functionality of course. Everything else must be pushed forward to an upcoming release in a professionally managed project.

6.3.2. Maintenance Release

Each maintenance release of a line of software, often called a “point-release”  or ocassionally a “hot-fix”, passes through what might be called a miniature life-cycle process. From SWEBOK v3, Ch. 5:

Maintainers perform analysis, design, coding, testing, and documentation. They must track requirements in their activities—just as is done in development—and update documentation as baselines change.

This is where the existing test suites—unit, integration, and system, together with additional QA regression testing prove their mettle  in prevention of new defects. Maintenance deliverables should receive a full round of automated and manual spot-testing before release.

Long-Term Support Release

A long-term support  release (LTS) is commonly available for operating systems and other long-lived platform software, generally extending common support periods to five and ten year periods. This allows larger, slower-moving “enterprise”  (think corporate or government) organizations to further optimize their processes for stability, thereby reducing upgrade and training costs.

6.4. End-of-Life

Does this mean I’ll need to re-install Zoo Tycoon?
— the Onion, on Ending Support for Windows XP 
_images/cell_10.jpg

Fig. 6.5 “They paved paradise and put up a parking lot…” 

Years (to decades) later, as a venerable  software product arrives at the end of its useful life, the organization responsible for maintenance decides to retire it—meaning the end of any and all marketing, sales, maintenance, or support activities regarding the product. This phase of the cycle is called end-of-life (EOL) . Software dropped early by its maintainer may also be called abandonware.

When a product has time-based releases with a regular schedule, maintenance and support periods are generally tied to it, an example of which are the releases  of the Ubuntu distribution of Linux. A similar strategy for proprietary software may be termed a “sunset policy”  or “planned obsolescence” . In rare cases, proprietary software may be turned over to its user-community as open-source, to allow them to provide service and/or further upgrades .

The retirement of popular software can be contentious. Recently Microsoft’s end of support  for Windows XP found its way into the headlines. It was such a successful product that there were still millions of users of it with no pressing need to upgrade. Following a large number of complaints and cries of hardship, Microsoft capitulated and supported it another two years after its originally scheduled EOL date, providing for a full twelve years of support without additional charge.

6.5. Conclusion

“This is how the world ends, swallowed in fire, but not in darkness…”

This chapter concludes our discussion of the traditional (yet modernized) software development life cycle. The information presented up to this point will be invaluable as we discuss new developments in software engineering, Models & Methodologies, and Project Management in general.

TL;DR 

  • To make a rational decision regarding when to ship a product, you’ll need:

    • Release criteria, a list of goals to work towards.

    • Code stability

    • Current completion and quality data, to enable easy visualization of progress.

    To reach a verdict, compare where you are with where you need to be.

  • The majority of the development work in software is in maintenance activities.

  • The first rule of software maintenance is, don’t break anything!

  • Breaking changes must never be added to a maintenance release—customers pay for surprise prevention.

  • Approximately half of total effort in software maintenance is dedicated to understanding the software to be updated.

  • Maintenance releases benefit from additional regression testing.

  • A long-term support (LTS) release is a commonly available for operating systems and other long-lived platform software.

  • As a software product arrives at the end of its life, it is retired, a cessation of all associated activity.

  • Continuous delivery builds on continuous integration and takes automation from source control potentially all the way through deployment to production.