QA Labs brings you practical tips and tools on testing, quality assurance (QA), and related topics through this monthly e-newsletter.
This month's article author: Cip Ticea, editor: Trevor Atkins.
Quality Approach As A Strategic Differentiator
Analysts have noted that the era of "deadline-driven" and "do more for less (cost saving)" is ending. More and more companies today have adopted a new mantra: "do the right things the right way". Time to market, cost and now quality are key ingredients of today’s corporate strategies.
It is important to note that real life will drive company executives to commit to this approach only if quality is perceived as strategic market differentiator.
The Challenge
For those that work in QA/QC area, it is important to know what to do in order to bring quality up to this "market differentiator" statute.
The roadmap to put it in practice usually starts with getting executive's formal "go do it" type commitment. Once you have the commitment, the next step is to write it down in the form of a policy. This policy would cover the basic principles on which "quality" is to be built. The policy will spell out corporate business strategies in the form of related engineering principles. Now you have the money (a.k.a the commitment) and the standards and rules (a.k.a the policy). The next step is to define the best possible testing process that fits your newly defined needs and within your constraints.
In an ideal world, every company ought to have their testing process defined. One mistake companies might make is to develop or adopt a testing process and then expect that it will address all their testing needs. The magnitude of the impact of this mistake is directly proportional to the diversity of projects a company holds in its portfolio. The higher the diversity, the more likely that a single testing process/framework will fail to respond to the company's overall testing needs.
In support of this statement consider the following challenge: assume you are the QA manager in company with a diverse project portfolio. What would work best for you: to be given a prescribed, well laid-out formalized process and testing artifacts (i.e. templates, tools, etc) or to have the flexibility to create your own testing process and artifacts on a per project basis? Many of us would answer this question with: "it depends."
Looking at this from another angle, our possible choice could be somewhere between the two ends of the spectrum: "no process" which might equate to full flexibility and "very detailed process" which equates to rigidity.
One can approach the answer from a perspective of economics. What the executives want to hear is that we are productive and that we "do the right things the right way". Therefore we make clear at the corporate level that it is necessary to have a testing policy and a set of test practices which one could use to quickly assemble a custom fit-for-purpose test process for each given project.
The key principle behind this rationale is that quality can be improved while time-to-market and cost go down by re-using and refining components of a well architected test framework. In this context, it pays to know how to architect/tune-up your testing process to achieve the highest productivity and quality possible.
The Options
The external context of the testing process often dictates what flexibility means from the testing perspective. There are many variables that come into play and in support of what has been suggested above we will focus on the relationship between a given Software Development Lifecycle (SDLC) model and associated testing techniques.
There are many off-the-shelf SDLCs available today, each with its own related testing approach and/or philosophy. Each one of them is centered on one or more of the following main activity classes: requirements capture, design, coding, testing, roll-out, and maintenance.
It is common that a project is matched to a preferred SDLC rather than to a preferred testing process. Also, the "real world" suggests that very few companies follow a prescribed SDLC to the letter. Most often, the project manager ends up customizing an SDLC by borrowing approaches and techniques from existing, "proven", SDLCs they have had success with in the past. It is less common that a project manager has the ability to select testing techniques to match the chosen project execution approach. This is where the QA/Test architect comes in.
A well architected testing framework should integrate seamlessly with the SDLC of choice for any "normal"–type project a company might undertake. Being able to assemble that fit-for-purpose test process and techniques for a given SDLC could help project’s bottom line.
The following table highlights testing techniques employed in a few of some popular SDLCs as they are reflected in today’s literature and/or research:
|
SDLC Model
|
Testing approach (The WHAT = activities & the HOW=techniques)
|
|
Boehm's Spiral Model
|
SDLC key note: Uses prototyping and re-planning with reassessment of risks and constraints with each prototype.
Common techniques:
- Risk-driven testing: During the risk analysis phase of the SDLC, testing is used to identify risks. Defect analysis and defect patterns are key to identifying the "soft" areas of a product.
- Time-boxing techniques. Testing starts early in product development life cycle.
- Risk-based test success/stop criteria. Test coverage is not an explicit goal. Test strategy aims at uncovering as many risks as possible and help builders decide if they can move on to the next spiral or not.
Notes:
- Within the product development phase of the spiral, testing is very similar to that seen in waterfall life cycles except smaller in scope and duration.
- Time available for testing increases as there is a longer time between the start of testing and going live.
|
|
Personal Software Process (PSP) |
SDLC key note: Training the developer to understand how to identify, analyze, and improve individual processes. The goal is to enable each individual to take charge of improving their own productivity and deliver on pre-established quality goals.
Common techniques:
- Reviews (code inspection, personal review, walkthrough).
- Early defect removal. Developers are encouraged to focus on removing defects early, record and manage defects, and conduct regular defect analysis.
- Strive to improve the testing process continuously.
- Personal Defect Management (fix your own defects soon after they have been injected; personal design & code reviews following prescribed checklists/guidelines).
|
|
Waterfall and/or waterwheel SDLC
( i.e. Giddings Domain-Dependent Life Cycle)
|
SDLC key note: Waterfall model with feedback loops, and experimentation or prototyping. Relies on large test phases which start after coding has completed.
Common techniques:
- Relies on internal deliveries of code to testers who look at the product from both a clear-box and black-box perspective.
- Clients are involved in testing at the end during acceptance testing.
- Additional, post-delivery testing is performed to validate product.
Notes:
- Testing improvements are usually possible at the end of the project (i.e. as part of the post-mortem/close-out analysis). It takes a longer period of time to get better because the opportunities to trial and implement new ideas are relatively far between.
|
|
Agile Programming SDLC
(covers approaches such are XP Programming, SCRUM, etc)
|
SDLC key note: Incremental approach relies on continuous (automated) testing by developers, on oral rather than written communication, and close collaboration.
Common techniques:
- Test-driven development. Developers are expected to write test cases before the design and coding are done. These test cases are to be executed by the developer once the coding is complete. Very important is that test cases are updated to improve readability/usefulness.
- Automation. When test cases are "stable" they are automated, preferably in groups such that they can be associated with a system design package/module.
- Good enough quality. The aim is to deliver "good enough" quality. Usually the good enough depends on factors related to the complexity of the system, criticality of its core functionality (i.e. are people's lives at stake or not? Is this product vital to company's growth strategy? etc.), customer's tolerance to errors, etc.
- Testing as risk mitigation strategy. Integration-type testing is regarded as main mitigation strategy for risks related to putting together code changes from various development teams.
- Acceptance tests (a.k.a. "User-stories") are usually written by the "customer/business" representatives. These tests should be completed before the developers starts designing and coding.
|
|
STEP (Systematic Test and Evaluation Process) SDLC
(Similar SDLCs: V- and W-model, Component Test Process Life Cycle)
|
SDLC key note: Life cycles that emphasizes the place of and control of software testing.
Main activities are: Develop test strategy and plan; Design and implement fit-for-purpose test environment; Execute Test and collect results; Evaluate test process and test infrastructure and plan improvements (from one iteration to another).
Common techniques:
- Risk-driven testing. Concentrate on testing what matters.
- Evolutionary development of the test plan. Start with an outline plan and then add details as project advances. Start planning based on Requirements (acceptance test plan & cases), then based on high-level design (system test plan & cases), detailed design (integration test plan & cases) and coding (unit test plans & cases).
- Test design at each stage is completed as soon as possible.
- Employs a bottom-up approach to executing testing (i.e. unit tests first, module, system, etc).
- Map test cases to requirements, design, and code units with full coverage as a declared goal.
- Discourages un-structured testing (i.e. ad-hoc).
- Encourages/relies on collaboration among senior resources to identify risks, set test objectives, and take decisions fast through techniques such is "brainstorming meetings".
- Focuses on maintaining visibility and traceability of testing effort, deliverables and overall testing progress.
- STEP uses IEEE standard document templates as a recommended guideline.
|
This information is not a comprehensive listing of today's techniques. The intention is to show that testing is approached differently depending on the SDLC practiced by the project team.
From a practical perspective, each project has its own special needs which would be best served by a unique collection of testing techniques.
Conclusion
Starting in 2004, many companies have started to put "quality first" in their mission statements - a trend that Gartner predicts will continue. Good testing practices are key ingredients to convert this from a statement of intention into competitive advantage.
Corporate commitment to continuous improvement, well defined test process, and fit-for-purpose tools can dramatically reduce costs and time to market while overall product quality increases.
It is not enough to start testing early as much as to do the right things the right way.
QA Labs is a powerful player on your team supplying the critical competitive advantage you need today. Our mission is to help you make your software products succeed in the marketplace, whatever the climate. We work with you to make wise choices that reflect project constraints, industry trends, and business considerations. We are the largest independent software quality assurance and testing service provider in Canada. For more information, please visit www.qalabs.com.
Reduce costs - Take advantage of lower labour costs and today's