thoughts from the test eye

A project manager, that had no knowledge about testing whatsoever, asked me how he should go about testing? The question was vague since he did not know where to start. I wrote him a quick email listing a few things to consider:

• Create a list of all use cases. For each use case consider possible errors. Identify how these use cases and the errors from these could be tested and verified.
• Create a list of techniques used. Identify common errors for the specific techniques. Identify issues that affect these techniques in one way or another. These issues and common errors can be used as a base for testing.
• Create a list of the major areas of functionality that need to be tested. Create a matrix and categorise the areas in two columns (column one: risk anything goes wrong, column two: cost if anything goes wrong) from 1 to 5, where 1 is most costly or highest risk. Sort the categories.
• Identify important milestones when there are deliverables for testers.
• Identify what deliverables you expect from testing. This can be test reports, bugs. Consider when and how often you want these. Be reminded that all administration will take time from regular testing.
• Which test resources are available and to what percentage. Investigate when they are away and if they have other tasks that they work with.
• What testing has been done so far? What is documented? What did they find?
• Is there a backlog of bugs? Are they well tended and classified? Can they be reused and are they still valid?
• Decide on how you want to work with bugs and enhancements
• Have the testers received all possible documentation needed to start working? This can be support documents, help-files, design/functional specifications and so on. These documents can be used for test ideas.
• What will the test environment look like? Will it be a virtual one or a full customer-like environment with hardware and software?
• Have you considered how many prototypes that should be available for the testers? Consider that many might break. Testers without something to test is a no-no.
• Is there any other configurations that we should consider that is outside the test environment that these builds/prototypes need to run in?

If your testers are new to testing consider this:

• Use the requirements to create user scenarios that in turn can create more test ideas.
• Some tests might need to be planned long before they are executed. Example of this can be EMC testing or Safety testing, that might be performed externally in other test labs.
• System testing should be done by testers, thus not a developer or product manager.
• Do not test everything. Focus on costs and risks first. All areas that go untested should be a calculated risk done by stakeholders or decision makers.
• Test everything. All major functionality should have at least one test. Important areas should have deeper testing while less important can have shallow testing.
• Consider long term investments in testing. Will there be more versions of the product that need even more testing. It might be good to invest in test automation if it gives a ROI. Should test cases and test result be saved for the future, in most cases you would like that.

All of this information is usually structured in the various documents that expert testers are used to such as test plans, test strategies and so on. But for someone that did not know anything about testing I just gave him this.

I got no response from the project manager on this though, so either it was total crap or I scared him away.