Testing Web Applications Part 2: Preparing for Battle!
In “Testing Web Applications Part 1: Priming the Team,” I discussed some of the reasons why testing a web application is different than testing something like a desktop application. In this article, I am going to outline some key points to help you keep focused and hopefully allow you to avoid common pitfalls when planning your testing effort.
When to Plan:
Day 1. Period. As soon as the marketing requirements are gathered and have stabilized/been signed-off, test planning should begin in earnest. This rule of thumb should be applied to all software development projects, but for the web application project this rule must be applied. The timeframe for these kinds of projects is so short and they are under such intense pressure to go live at a specified date, that test planning must begin as soon as possible.
How to Plan:
I’ve listed a condensed set of items to think about when planning to test a web application. Often this type of information is not made available to testers in any written form...or even in any oral form! Most test teams get stuck with an incomplete picture when starting their test planning activities. The items below are designed to help you fill in the gaps, where information is often miscommunicated or not communicated at all. Think ahead to two weeks before release when the product manager says tothe test team: “You’ve tested on IE4.5 on Mac 9.0, right?” This is one way to ensure you can answer the question.
1. Define the goals of the site.
This is useful if only to eliminate the non-purpose elements from your test planning. For example, if the site's primary purpose were one of information exchange, not e-commerce, then the test planning would have less detail or focus on the e-commerce functionality of the site.
2. Define the audience of the site.
Knowing this will help you focus on the type of user and the most common functions they will perform at the site, as well as their expectations! Useful User Scenarios can then easily be created to help define your testing scope and focus.
3. Define the quality criteria of the site.
Perhaps reliability and security are the key areas on which to focus your testing; perhaps it is speed and performance. Asking these kinds of questions will help you focus on the most important areas. If performance and load testing are criteria for your web application, and you've never done this kind of testing before, get help from people who are more technically savvy in this area. A good place to start is with in-house development resources, maybe they can create some scripts that test the web server directly and that can simulate many users.
4. Define the functional requirements of the site.
Then count them. This will allow you to generate ball-park estimates of the number of test cases required to test the functionality of the web application. A quick rule of thumb is to have a minimum of four test cases for each distinct functional requirement: a valid case, an invalid case and two boundary cases (upper and lower). A closer inspection of the functional requirements themselves will reveal areas that will require more test cases than the minimum to achieve adequate functional coverage.
For example: 100 functional requirements x 4 test cases per requirement = 400 test cases. This is a minimum. You will still have to go through the requirements and their associated test cases to ensure the level of test planning is adequate for all the requirements. Having test cases that are traceable back to the originating requirements will make this task much simpler.
5. Define the non-functional requirements of the site.
Since this area is what will multiply your testing effort the most quickly, it is a good idea to get a complete understanding of the scope desired by project management. This is one area where test planning can reveal the huge impact adding an extra browser or operating system will have on the test schedule. This is also the one area where prudent negotiation, pre-planning, and risk management can keep your test schedule under control. The clearer your understanding of the testing required on one operating system/browser combination, the more easily you can communicate the costs of additional testing for other platforms and/or browsers.
For example: 400 test cases x 4 environments = 1,600 total test cases. Each additional test environment (i.e., operating system/browser combination) adds 400 test cases to your test effort. Imagine if it takes 1 person-week (5 working days) to execute 100 test cases. Each additional test environment would add 4 person-weeks of testing.
6. Write all this information down in a document.
Try not to be too verbose or overly formal: this document is a high-level view for the test team to use throughout the project. It is not designed for use by the rest of the project team, nor as a replacement for other project documentation. Hopefully, much of this information will already be written in some document(s) to which you can refer. Once that is done, have it reviewed by the project stakeholders for accuracy.
7. Create your test cases and prioritize them.
I suggest creating them in this order: valid functional test case; invalid functional test case; boundary functional test cases, User Scenario test cases. Thus, you would create a set of valid functional test cases before your boundary functional test cases. Likewise, you would create User Scenarios by linking together your valid functional test cases (notice that the User Scenarios are comprised of functional test cases, and thus the “ball park number” mentioned in step 4 above is still valid). If your timeframe is very short, create the User Scenarios before the boundary functional test cases (these will give you more bang for your buck); otherwise create them afterwards to ensure you test all boundary conditions. Prioritize them in order of importance and frequency. The test cases you will run the most frequently or that test critical functionality would be of a higher priority than those you would run infrequently or only once.
In “Testing Web Applications Part 3,” I will address the act of testing this web application, including some ideas for tools and tricks to simplify the tasks that lay ahead.
Originally published by Software Productivity Center in their May 17, 2000 (Issue 19) edition of E-SSENTIALS! newsletter.
Back to Resources.
