Upcoming Course Announcement: Register now for QA Labs' two-day Introduction to Automation course presented through the University of British Columbia Continuing Studies on November 7th and November 16th, 2002.

Toolkit Testing - A Lightweight Alternative

• Are you ready to take on the challenges of the new project?
• Are you knowledgeable of the latest in testing tools and techniques?
• What will be the ramification of not testing the product adequately?
• How will this impact your future business?
• Can you afford not to test?

Exhaustive software testing standards, frame-works, and techniques, promote the notion that a robust variety of testing techniques and structures will increase the likelihood that defects will be uncovered. But with tight project budgets and short timelines, the accompanying bureaucracy and documentation can greatly reduce the interest in formalized testing, structured or otherwise.

It is well understood that a higher quality product demands a higher upfront price and compromises regarding quality versus cost are made every day. However, the best approach is not likely the most structured or complicated one. Rather a sophisticated approach is required that maximizes the value of the resources available within the organization and without. A good starting place to developing this sophisticated approach is to examine the fundamental skills that make great testers great, enabling them to draw upon their almost innate ability to find the crucial defects quickly.

From this examination it is very probable that you will generate ideas for:

• Managing iterative development and test cycles
• Creating reusable sets of tests and test data
• Compressed delivery schedules
• Newly refactored architectures
• Standardized project metrics
• Migrating from simple to complex deployments
• Changing customer expectations

A few simple principles can provide the framework from which to grow and adapt the test approach. Keeping things simple will make it easy for the benefits and costs to drive further evolution. While there are continuously new development tools and programming languages, many testing requirements remain the same, and simply require additional emphasis within and by the test team.

• Examples - having previous projects from which examples of the "best of breed" can be drawn for each type of process, document, or test technique is invaluable in giving the next project a giant jump start in how to approach the test effort. Asking, "how did we do it last time and what can we improve?" will drive forward improvements to your testing toolkit strongly, and with frequent project iterations, rapidly.

From these previous project examples you can begin to derive reusable tools for your toolkit:

• Guidance Process - generic process frameworks and best practices that can be applied to most project types and be ingrained as habit as much as in any documentation.
• Templates - light-weight documents focused on capturing the critical information and not on keeping resources busy with technical writing.
• Checklists - lists of test, lists of tasks, matrices of test configurations that allow you to rapidly document and check off what has been done and see what is left to do

Note: If you do not have your own history of previous examples there are many resources on the web where others share their experiences and advice such as at www.stickyminds.com

In one such example, James Bach of Satisfice.com provides a number of whitepapers and articles on "Exploratory Testing" wherein he has a set of mnemonics and heuristics in his toolkit. One of these mnemonics is SFDPO where the letters stand for Structure, Function, Data, Platform, and Operations.

• Structure - what the product is
• Function - what the product does
• Data - what the product processes
• Platform - what the product depends upon
• Operations - how the product will be used

Using rules and checklists such as this allow you to quickly focus your test idea generation and ensure that you have systematically visited the major aspects of the product.

"SFDPO is not a template or a test plan, it's just a way to bring important ideas into your conscious mind while you're testing. It's part of your intellectual toolkit. The key thing if you want to become an excellent and reliable exploratory tester is to begin collecting and creating an inventory of heuristics that work for you. Meanwhile, remember that there is no wisdom in heuristics. The wisdom is in you. Heuristics wake you up to ideas, like a sort of cognitive alarm clock, but can't tell you for sure what the right course of action is here and now. That's where skill and experience come in. Good testing is a subtle craft. You should have good tools for the job." - James Bach

However, even with the best tools and techniques, a test team can't create the kind of return on investment managers require as long as the test efforts don't start early and don't involve all appropriate stakeholders and participants. When developing your testing processes, look for those improvements where:

• Errors are detected and corrected as early as possible in the software life cycle
• Project risk, cost, and schedule effects are lessened
• Software quality and reliability are enhanced
• Management visibility into the software process is improved
• Proposed changes and their consequences can be quickly assessed

"... when all the pieces come together - the right people, the right processes, the right time, the right techniques, the right focus - then we can achieve truly impressive returns on our testing investment. Significant reductions in post-release costs are ours for the taking with good testing. In cost of quality parlance, we invest in upfront costs of conformance (testing and quality assurance) to reduce the downstream costs of nonconformance (maintenance costs and other intangibles associated with field failures)." - Investing in Software Testing by Rex Black.

For more information.

QA Labs' test experts construct effective test strategies tailored to your timeframe and resource realities. A well-scoped and actionable testing strategy is critical to ensuring the continued survival of a given product line, or the successful launch of a new application to the market. QA Labs paves the way for immediate and on-going quality improvements by establishing critical artifacts, practical automation, and integrating quality best practices - all while getting those bugs logged.

Our clients hire us knowing they have hired a company, not an individual or group of individuals. We bring our comprehensive experience to the table and can provide the exact resource skill sets needed at exactly the right time. QA Labs offers a clear alternative to hiring individual contractors, and even to hiring new permanent full-time employees.

Contact us at services@qalabs.com or visit our website for more information on our QA and Testing services.