6. Release & Maintenance¶
Software being "Done" is like a lawn being "Mowed."
— Jim Benson @ourfounder
This chapter marks the closing of the traditional software development
life cycle—the end of a project and birth of a new software product.
We’ll follow it from release,
through the postpartum period of support and maintenance,
Real artists ship.
— Steve Jobs
Yes, they do (as the author is painfully reminded while trying to finish this damn book). So, the time has finally come in to ship the product. Is it good enough? Is it… done?
By this time, new feature development will have frozen long ago, shifted in favor of stability, bug-fixing, and general polish. Planned testing will have completed with all known defects documented and listed in a database for final analysis. The most important requirements and quality objectives, which were documented at the start of the project, will have been validated through acceptance testing. Unless the project proceeded perfectly, a few less-important (or difficult) features will likely have been cut, backed out, disabled, or postponed by the time the project approaches release.
6.1.1. Release Criteria¶
In the competitive commercial software market, software companies feel compelled to release software the moment it is ready. Their task is treacherous, treading the line between releasing poor quality software early and high quality software late.
A good answer to the question, "Is the software good enough to release now?" can be critical to a company’s survival.
— Steve McConnell, IEEE Software Best Practices Column
Where should we be?
To make an edumacated decision, we’ll first need release criteria, the list of objectives that need to be completed prior to shipping. Unless they’ve been specified properly, there won’t be concrete goals to work towards. In short, they’ll be project requirements (at agreed quality), minus any cut features.
Where are we?
Without stability and measurable data it isn’t possible to know with any degree of certainty where the project is in comparison to its goals and therefore make a rational decision regarding whether to ship.
What’s left to do?
When criteria are set and functionality/quality measured, everyone on the team can see progress and knows when the project is ready to release.
Once the preferred release-date has been scheduled, what’s left to do? Quality goals, such as the elimination of blocking issues (showstopper/data-loss bugs) or restrictions on the number of open bugs, are met. Open bugs that can’t or won’t be fixed by the deadline (in support of stability) are postponed as well.
See also: Online Resources
Release Criteria: Is This Software Done?, courtesy Johanna Rothman :
What’s important for this project?
Draft release criteria.
Make release criteria SMART.
Gain consensus on release criteria.
Example release criteria:
All code must compile and build for all platforms.
Zero high priority bugs (Zarro Boogs )
All open bugs documented in release notes with workarounds.
All planned QA tests run, at least ninety percent pass.
Number of open defects decreasing for last three weeks.
Feature X unit tested by developers, system tested by QA, verified with customers A & B before release.
Ready to release by June 1.
Now witness the firepower of this fully armed and operational battle-station.
— Emperor Palpatine, "Return of the Jedi"
During the final hours, the full team will typically perform last-minute ad-hoc testing until the “lever is pulled.” Congrats! Let’s take it for a spin:
6.2. Continuous Delivery¶
Continuous delivery takes automation from source control all the way through production.
As an alternative to the one-time or infrequent release schedule described above, we have continuous delivery, also abbreviated as CD. CD is the rapid delivery of release-candidate level code fit for deployment at potentially any moment throughout its history. Background from “Continuous Delivery” by Jez Humble and David Farley follows :
In the late 90’s I paid a visit to Kent Beck, then working in Switzerland for an insurance company. He showed me around his project and one of the interesting aspects of his highly disciplined team was the fact that they deployed their software into production every night. This regular deployment gave them many advantages: written software wasn’t waiting uselessly before it was used, they could respond quickly to problems and opportunities, and the rapid turn-around led to a much deeper relationship between them, their business customer, and their final customers.
CD builds on the foundation of best practices that have developed over the years to shorten the feedback loop during development of a project:
Full build automation, to eliminate clumsy manual labor.
Daily builds and smoke tests, that find errors more quickly.
Continuous Integration, to further reduce errors and waste.
Automated test suites, for increased confidence.
Benefits of Continuous Delivery
Faster time to market—new features are not tied up in the development pipeline.
Quicker feedback on new features
Low risk releases—more frequent releases are by definition smaller and less-likely to break.
Release is practiced often and regularly, therefore process issues get fixed.
Releases are no longer scary.
Happier teams, through reduced frustration and stress.
Lower costs—through investment in build, test, deployment and environment automation.
Continuous deployment is simple: just ship your code to customers as often as possible.
— Timothy Fitz
Continuous delivery is related to but often confused with continuous deployment, which goes one step further—the frequent push of builds to “production,” meaning that customers are now directly working with “freshly built” software. In other words, the builds are not just ready to be deployed at any moment, they are deployed! This strategy can make sense in environments with a higher risk tolerance.
Remember: continuous delivery of candidate builds, continuous deployment to production.
See also: Books
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, by Humble & Farley
There are a number of great nuggets of wisdom in this book, the only drawback is that it is a bit repetitive. Worth the read, but you’ll find yourself skimming the later chapters.
See Also: Online Resources
The cycle time from conception [of a feature] to production has decreased from several months to two to five days.
— Lianping Chen
Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance.
— Kurt Vonnegut, Jr.
Once a software project has been shipped and exercised by users in the real world, bugs surface, new requirements emerge, and underlying platforms shift over time. The maintenance phase of the product’s life-cycle begins. While not as glamorous perhaps as some of the other phases, it’s an important consideration for customers in regard to total cost of ownership (TCO ); with necessary work typically financed through support contracts.
Often the same team of developers that created the project will maintain it, supporting several lines of development (branches) simultaneously. In other circumstances, a separate team may take over maintenance. Work may even be outsourced to an external entity. The maintainer is the individual or organization that maintains the software.
Building vs. Maintenance
One of the things they don’t get around to mentioning at University (and if they did, may have been ignored), is that the majority of the development work in industry is in maintenance. That is, patching, updating, and fixing other’s already-written code.
New design and engineering, sometimes called greenfield development , i.e. where the “fun” is, is a bit more rare. Still, software maintenance is a useful learning activity for junior engineers, both in learning what to do, and in some circumstances what not to.
The fundamental problem with program maintenance is that fixing a defect has a substantial chance of introducing another.
As a consequence… program maintenance requires far more system testing per statement written than any other programming.
— Frederick P. Brooks, Jr., The Mythical Man-Month (Ch. 11)
If the maintenance team is not the same as the development team, comprehension of the code base can be an issue. Research has shown that approximately half of total effort is dedicated to understanding the software to be updated . This is the point where time spent on readability, documentation, and writing tests earlier pays great dividends. “Pay it forward,” as they say.
The First Rule…
The objective of software maintenance is to modify existing software while preserving its integrity.
— SWEBOK v3, Ch. 5
Best practice allows only fixes to a maintenance branch, with enhancements (new features) and changes to interfaces prohibited—and only considered under extreme circumstances. The reasoning should be clear from the paragraph above, the product must remain stable. That’s what customers pay for, surprise prevention. It’s just too risky to do otherwise.
Reactive modification of a software product performed after delivery to correct discovered problems.
Modification of a software product performed after delivery to keep a software product usable in a changed or changing environment.
Modification of a software product after delivery to improve performance or maintainability.
(Generally recommended to be done in a new version)
Modification of a software product after delivery to detect and correct latent faults in the software product before they become effective faults.
More simply, the reasons for a maintenance team to release updates to a shipping product are:
Fixing functionality, that never worked or no longer works correctly.
Closing security holes
Somewhat rarely you may see an important feature “backported” to a previous release, as may happen under a pricey support contract. The port must not interfere with existing functionality of course. Everything else must be pushed forward to an upcoming release in a professionally managed project.
6.3.2. Maintenance Release¶
Each maintenance release of a line of software, often called a “point-release” or ocassionally a “hot-fix”, passes through what might be called a miniature life-cycle process. From SWEBOK v3, Ch. 5:
Maintainers perform analysis, design, coding, testing, and documentation. They must track requirements in their activities—just as is done in development—and update documentation as baselines change.
This is where the existing test suites—unit, integration, and system, together with additional QA regression testing prove their mettle in prevention of new defects. Maintenance deliverables should receive a full round of automated and manual spot-testing before release.
Long-Term Support Release
A long-term support release (LTS) is commonly available for operating systems and other long-lived platform software, generally extending common support periods to five and ten year periods. This allows larger, slower-moving “enterprise” (think corporate or government) organizations to further optimize their processes for stability, thereby reducing upgrade and training costs.
Does this mean I’ll need to re-install Zoo Tycoon?
— the Onion, on Ending Support for Windows XP
Years (to decades) later, as a venerable software product arrives at the end of its useful life, the organization responsible for maintenance decides to retire it—meaning the end of any and all marketing, sales, maintenance, or support activities regarding the product. This phase of the cycle is called end-of-life (EOL) . Software dropped early by its maintainer may also be called abandonware.
When a product has time-based releases with a regular schedule, maintenance and support periods are generally tied to it, an example of which are the releases of the Ubuntu distribution of Linux. A similar strategy for proprietary software may be termed a “sunset policy” or “planned obsolescence” . In rare cases, proprietary software may be turned over to its user-community as open-source, to allow them to provide service and/or further upgrades .
The retirement of popular software can be contentious. Recently Microsoft’s end of support for Windows XP found its way into the headlines. It was such a successful product that there were still millions of users of it with no pressing need to upgrade. Following a large number of complaints and cries of hardship, Microsoft capitulated and supported it another two years after its originally scheduled EOL date, providing for a full twelve years of support without additional charge.
This chapter concludes our discussion of the traditional
software development life cycle.
The information presented up to this point will be invaluable as we discuss new
developments in software engineering,
Models & Methodologies, and
To make a rational decision regarding when to ship a product, you’ll need:
Release criteria, a list of goals to work towards.
Current completion and quality data, to enable easy visualization of progress.
To reach a verdict, compare where you are with where you need to be.
The majority of the development work in software is in maintenance activities.
The first rule of software maintenance is, don’t break anything!
Breaking changes must never be added to a maintenance release—customers pay for surprise prevention.
Approximately half of total effort in software maintenance is dedicated to understanding the software to be updated.
Maintenance releases benefit from additional regression testing.
A long-term support (LTS) release is a commonly available for operating systems and other long-lived platform software.
As a software product arrives at the end of its life, it is retired, a cessation of all associated activity.
Continuous delivery builds on continuous integration and takes automation from source control potentially all the way through deployment to production.