Friday, September 12, 2008

Software Testing from Scratch

Five Steps to Tackling Everyday Testing Issues

Summary: Many people in the software testing field did not choose their career, but were placed in the position. As a result, very few individuals are fortunate enough to have actually been trained in testing before they started or even after they’ve been testing for months. The newly appointed tester is left with many questions about how, what, and when to test. This article will outline a straightforward, five-step approach to tackling these issues.

You're assigned to test and you've never done it before. You didn't request the position and you have no training. Now what? This article will outline a straightforward, five-step approach to tackling everyday testing issues. Each part builds off of the previous level in a progressive manner that can be applied to most methodologies and projects.

Step 1: Identify. When you don’t have a previous history from which to build, you have to establish a baseline of expectations, areas to test and strategies
Step 2: Inventory. Catalog these items
Step 3: Prioritize. Use your inventories to evaluate where the most immediate needs are and where to delegate tasks
Step 4: Implement. Decide how to accomplish the tasks you have identified and put your plans into action
Step 5: Analyze. When the smoke clears, take a careful look at what worked and what didn’t. Use this information to begin the identify step for the next project

Or, if you like mnemonics, “Insight Is Preferred in Advance.”

Step 1: Identify
The first step is to identify what you want to accomplish through testing. If you or your manager think the objective is “bug-free software,” you’re probably new to the testing game. Even simple projects have a huge number of variables that can cause defects. You have to consider the hardware environment, operating systems, range of inputs, volume of data, data integrity, among other factors. It’s a daunting quantity of information to work with. An essential part of software testing is breaking down these areas and finding ways to manage large amounts of data for maximum results. In other words, you’ll need to narrow your scope.

Start by picking up a good testing book and read the first chapter or two. The first two chapters of these books usually give a little history of testing and begin to talk about the scope of what it can accomplish, without getting into the fine details. Start to look at testing’s role in terms of what must be done, what should be done, and what can be ignored. As you begin to identify these items, start brainstorming lists of resources needed, areas to be tested, and potential risks. You’ll also want to be the leading authority on your product.

Much of this knowledge is gained from the program requirements. If there are no documented requirements for your product, interview the project manager, developers, and the people that requested the project be built. Find out what they want the product to do. Record this information and use it to evaluate the functionality of the program. If available, look at manuals or software for earlier versions of the product. Get call logs from customer support on existing products or look at other companies’ products that provide similar features.

Finally, identify resources you will need, including equipment, time, and personnel. Also, try to identify as many areas as you can that are products of the testing process, e.g. defect reporting, defect resolution, configuration management, test tracking and so forth. The amount and scope of these items will be directly proportional to your experience. Since nothing in technology remains static, it’s okay to add new areas that you identify as you gain experience and more time to improve your process.

Step 2: Inventory
Categorize your information and be sure to list each item under the correct category heading. List contingencies that relate to each category. Keep your inventories broad to begin with, and then fill in specifics as you go along.
The most important area to inventory is the requirements. If your requirements are already in an outline format, this may seem redundant, but it’s still very useful.

Here’s why: it helps you think more carefully about what is important. You don’t want to make a list full of redundancies or omissions. Select the parts of the requirements that are important to the functionality of the program. Solidify those areas in your mind and make connections to related parts of the program. Make a list of everything you think should be tested to verify the requirements. If you don’t have documented requirements, inventory the information collected in step 1 to use in lieu of them. The earlier you begin your inventories and planning, the more successful your testing effort will be.

There are several reasons for this:
* Testers look for things that are broken or will break. With experience, they will begin to identify and predict problems very early, when they are easier and less expensive to fix.
* Test plans are not easy to write. The sooner you know what you are going to be dealing with, the better you will be able to identify and plan out your testing needs.
* Begin to identify specific areas where you need to do more research e.g., if you’re working on a web-based ordering system, you can probably skip the articles on embedded software testing and focus on areas like web load testing. One bit of advice about early involvement: participate wisely.

Pick your battles, be constructive in your suggestions and realize that even great ideas start rough. Use this time to formulate ways of testing the project in development and provide useful feedback in areas you think need improvement. Remember that the designers are the source of the most important information you need to form test plans, so keep those lines of communication open. Inventories take time but are reusable. From the requirements inventory, for example, one can design a master test plan, create a traceability matrix, identify areas for risk analysis and check for test coverage, all of which are useful for managing the testing process. It is also easier to see where the inevitable requirements changes may impact different components from our inventory than would combing the program requirements page by page.

Step 3: Prioritize
You should now have more than enough stuff to keep any one person busy for a year or two. In the unlikely event you don’t, go back to identifying areas that will either affect how your product will work or what your product is supposed to do. Look for more specific testing articles and books related to the type of product you’re testing e.g., web-based, embedded, GUI, multimedia. When you feel that you’ve inventoried everything related to your project, it’s time to prioritize. How you prioritize your testing tasks will depend on your point of progress in the project cycle. If you’re not involved in the project until the last month of a nine-month cycle, be prepared to do a lot of ad hoc and exploratory testing. Keep track of your results to refer to when you get to step 5.

On the other hand, if you’re fortunate enough to be involved from the very beginning, you’ll want to get your test plans started early and invest more time in reviewing requirements before they are coded. Take a hard look at your inventories and start making decisions. Decide what can be delegated and to whom. What is mission critical? (e.g. You probably want test cases before you start collecting metrics.) What can be discarded if necessary? Being able to say that you can drop some tasks if you run out of time is just as important as identifying what is essential. Prioritizing can be a very difficult step. This is where resources like a master test plan or advice from a consultant can be useful in deciding where the biggest benefits will come from.

It is very hard to be objective and to be realistic about what can be achieved. Realize that you may miss something or emphasize the wrong module to test. This is perfectly normal and you’d be out of a job if developers didn’t make similar mistakes in their jobs as well. You will get better with experience and it does get easier once you’ve established your baseline. Keep in mind, it’s okay to miss some of your goals. That’s why you identified the most important ones up front. Hopefully, your deferred goals can be achieved on the next project.

Step 4: Implement
Now that you have a list of tasks and have prioritized and delegated wherever possible, get to work. Use your requirements inventory as an outline to create your testware. Make sure you cover the critical areas first and if you start running out of time—make some noise about the remaining areas that will be neglected. Show where you prioritized your tasks and explain what will happen if the schedule isn’t changed. If the project manager decides to disregard this, at least you have a record of why those last few modules seem a little buggier. Likewise, if you’ve identified a concept that needs to be implemented in your development process like a Change Control Board or purchasing a bug-tracking tool, use your inventory to build a report on why the concept is necessary and useful. Put the same kind of effort into planning process improvement that you would put into testing your program.

If you work in an organization where status reports or project plans aren’t part of the culture, try it out. You may be surprised at how much more agreeable people are when you’ve already planned it out for them. Supporting your reasoning with a report is not just more professional; it could be downright necessary if your manager doesn’t have a testing background. It’ll increase your credibility, show initiative, and give historical documentation that you can refer to in future projects if the same problem occurs again.

Step 5: Analyze
The most visible part of analyzing your end results is a post-project review or retrospective. This is where you take a good, hard look at your process and evaluate what to carry over to the next project, what can be dropped, and what should be added. Get as much involvement and feedback in this step as you can, preferably from the entire project team, but at least from your testing team. Moderate and record the feedback, but avoid censure.Take all this information and start back at step 1for the next project. You may be surprised at the progress you’ve made.

I’ve found this process to be a useful method for dealing with any new task or project, large or small. On the surface, it is simple and straightforward. There are countless more advanced tools and resources available for all types of testing, but I think they can be difficult to use effectively at first. Interestingly, I often use all five of these steps in evaluating how to apply more complex methodologies. Keeping “Insight is preferred in advance” in mind helps you to keep your focus to an appropriate scope and lay the groundwork for future projects.

No comments: