Software Testing and the Development Lifecycle

There are many perceptions of what software testing is all about. Some people see software testing as something developers do as they write code. For some testing is performed all at once when the code is complete. Others believe that testing should be performed along-side development activities.

It is a well-established fact that the sooner a defect is found the less expensive it is to fix. Consider the definition: "Testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results" - The Complete Guide to Software Testing, Bill Hetzel, 1988. Testing is a set of continuous activities performed with the objective of evaluating the software against pre-defined results/expectations.

Testing the application throughout its development lifecycle, emphasizing the early lifecycle activities helps increase the overall effectiveness and efficiency of this effort. Further, early involvement allows evaluation of important planning, design and development decisions with respect to how these decisions aid or impair the testability of the application.

Stages of the Software Development

Test resources have at least one associated activity within any phase of a project lifecycle regardless of the development methodology followed:

Project Planning - When the project plan and associated schedule is created, Testing reviews the plan to ensure that all testing tasks have been included and have been scheduled for an appropriate length of time. As well, assigning resources to these tasks will allow any potential resource allocation problems to be raised at the beginning of the project while there is still time to do something about any shortfall. Testing will also be a critical participant in initial and on-going Risk Analysis/Management.

Requirements and Specifications - While the requirements are being gathered and written, Testing begins writing a Test Strategy to outline the overall approach to testing and related assumptions and constraints. Testing also reviews the requirements for clarity, completeness, ambiguities, and testability. This means preventing defects from occurring in the code and having to be found during a future testing phase, when it is more expensive to fix.

Design - While Development is documenting an architecture and detailed design for the application, Testing continues Requirements Analysis and begins creating Test Cases (for inclusion in the Test Plan) and any other Test Assets (test coverage matrices, test data, test automation frameworks, etc.). Testing also reviews the design documents for accuracy and testability. Design for testability and the easier all areas of the application can be tested - improving quality while saving time and money.

Coding (unit) - Once coding begins, Testing must work towards completing Test Cases, Test Plans and Test Assets. Identification of Build Verification Test Cases (BVTs) and prioritization of the remaining Test Cases are completed at this point. Testing begins performing some unit testing (developing scripts and automation where applicable), depending on the scope of testing outlined in the Test Plan.

Coding (integration) - As integration begins, Testing can begin executing BVT Test Cases as part of "Smoke Testing" or "Stability Testing". Testing will then move on to high-priority Test Cases in areas where sufficient development has occurred. As integration progresses, Testing is able to test more and more of the application's functionality, reporting all defects. When the application nears functional completeness, Testing completes one or more test passes (execution of all Test Cases). Finally, focus shifts to system, scenario, stress, and performance testing. If Testing did not have all information available before or the requirements have changed, test execution and creation and/or updating test case steps may have to occur at the same time.

Coding (code freeze and bug fixing) - During this phase, Testing's primary activity is regression of defect corrections. Testing also participates in the triage of defects (reprioritization to reflect risk, schedule slips, or removal/addition of features) as needed.

User Documentation - Testing reviews all user documentation (manuals, help files, etc.) for accuracy, usability and completeness.

Release - Testing can sometimes be responsible for the final builds that lead up to the release. When the release decision is made, Testing collects and archives all project data, documentation, source and tools. Testing also participates in the project Post Mortem, where problems and solutions encountered in a project are reviewed for learning purposes.

Support and Maintenance - As externally-found defects are reported to the support personnel, defect reports are analyzed by Testing to determine if the defect is reproducible and whether it was known prior to release or not.