Company Announcement: QA Labs celebrates its 6th anniversary this month having completed over 200 person years of project activity in testing, test planning, test automation, performance testing, consulting, and training for companies with mobile devices and large web/client-server applications. Visit http://www.qalabs.com/expertise/ for more information.

Interviewing for Process Improvement

Are you interested in finding and fixing the pitfalls of your testing process?

Perhaps you want to know how to develop a customized, practical and cost effective approach aimed at detecting your test (process) problems and their causes. You might also want to learn how to put together that fit-for-purpose, down-to-earth set of recommendations that stand the best chance of being adopted and implemented by your organization.

An attempt to place one's organization on an absolute scale is difficult, costly, and often impractical. It is difficult for an organization's employees to relate to it mostly because they do not always understand the evaluation method or trust the data they've been benchmarked against, as neither "seems" to acknowledge their uniqueness. Without buy-in from key employees, top managers will face a much harder job in pushing through with acceptance.

Each Problem Has Its Own Remedy

There is no silver bullet. The method that is introduced hereafter is optimized to fit a certain type of organization and class of problems.

In particular, for this article, the working assumptions are:

• You work in a (medium-to-large) IT shop, or
• In your organization there are at least a couple of (un-related) projects running in parallel at any given time, or
• Your organization develops and/or maintains more than one (functionally distinct) application

Though these conditions are fairly generic, their impact on how the investigation will look in the end is significant.

Goals And Underlying Principles

First, let's set the expectations and goals of what we are trying to accomplish:

1. Develop a custom and practical interview approach best suited for surfacing problems and their underlying causes that plague one's testing process.
2. The interview approach should be based on an initial "best" strategy yet it should be flexible enough to accommodate emerging strategies if and when they show up.
3. Focus on the testing process and its dependence on relevant processes and factors such as requirements management, development process, configuration management and organizational constraints, just to name a few. This principle suggests that one cannot achieve significant improvements of the testing process by ignoring its "surroundings" (i.e. constraints).
4. The interview approach should be based on clear, logical methodology and models such that it could easily be kept up-to-date and/or improved based on industry latest trends and best practices as well as the organization's own experience.
5. Interview approach guides the investigator towards a set of recommendations that stands the best chance of being accepted and implemented by his/her organization.
6. This process delivers the "WHAT" can be done to improve the testing process along with its rationale (i.e. the "WHY").
7. Wherever possible the "WHY"s would encompass qualitative ROI analysis information to help with decision-making and prioritization.
8. The HOW, WHO, WHEN and WHERE are not explicitly delivered by this process. These would make the scope of the implementation phase, which would normally follow after this process is done.

Moving From Problems To Resolutions

Before diving into the internals of the approach itself, list some of the pre-conditions, input, controls, output and follow-ups. The following lists have been simplified, however they should provide a pretty good idea about what is involved.

Pre-Conditions

A few of the most important prerequisites are:

• The organization is aware of a testing process problem. The organization's upper management supports a solution that involves process changes.
• An outline of the interview approach exists. It is used to get an understanding of main activities, and to negotiate the level of service and/or deliverables needed. The intent is to avoid investments that carry little or no benefit to the organization.

Input

In addition to the interviewer's or consultant's own experience the most common input is:

• The problem statement
• Organization details, if available (i.e. org chart, department interdependencies, internal processes, internal politics, etc.)
• Industry best practices, up-to-date technology and process trends
• Up-to-date interview methodology and testing process models

Main Activities

• Review and validate the problem statement
• Preliminary information gathering, analysis and documentation
• Create a plan to address interview and logistics set-up
• Get agreement on the plan and associated effort/budget
• Design the interview approach and questionnaire(s)
• Review and get approvals from main stakeholders
• Interview middle management
• Analyse results and adapt interview strategy and questionnaires
• Interview top managers/sponsors
• Interview team leads and senior technical staff
• Analyse data and prepare recommendations
• Present recommendations

Output

The main output is the set of recommendations for WHAT can be done and WHY it will help.

These recommendations could target any or all of the following organizational layers:

• Corporate wide recommendations (i.e. policies, guidelines, infrastructure, etc.)
• Recommendations for Business departments (i.e. guidelines for end-users responsible for acceptance testing, guidelines for business analysts responsible for requirements gathering and analysis, etc.)
• Recommendations for IT/R&D (testers and development) departments
• Recommendations for IT/team units (test groups, development groups, project teams, etc.)

Controls

The metrics normally attached to this process are of two kinds:

a) Qualitative metrics

These are collected from your organization, via a "satisfaction" questionnaire first at the end of the engagement and second about 3 months later. The aim is to collecting the perception of the usefulness of your recommendations and how soon the pay-offs show up (i.e. this is a perception measurement that outlines how soon people get bought-in by a given set of recommendations).

b) Quantitative metrics

These metrics are meant to capture the implementation progress of our recommendations. The most common ones are the ROI and the break-even point achieved by the organization (i.e. how soon does the pay-off offset the investment).

What's Next?

The most common follow-up would start with analyzing the recommendations (i.e. the WHAT and WHY) and then concentrate on creating an implementation plan that adds the HOW, WHO, WHEN and WHERE, thereby taking your first step closer to the actual needed process improvements.

At this stage one common approach would have you loop through these phases:

• Create implementation plan
• Execute the plan
• Measure progress
• Analyses measurements and draft corrections

.and repeat.

Conclusions

Mixed with the consultant's own experience in the form of up-to-date methodware and models, an adaptive interview approach and customized questionnaires can account for the organization's specific needs/requirements around process improvement.

This process will help you flesh out customized, practical recommendations to top management, in order to correct deficiencies that are demonstrated to be impediments or damaging to improving costs of quality within the organization.

One of the driving goals of this process is to ensure that what comes out (i.e. of the recommendations) stands the best chance of getting implemented and adopted by the organization's top managers as well as its employees. They are designed with a practical end in mind. The end result is a set of recommendations, which are tailored to one's needs, organizational specifics and growth plans.