Multi-Dimensional Software Testing Rikard Edgren
Thare are many ways to look at the problem of testing software, and it is rarely wise to use only one; that’s why there are so many mnemonics, e.g. SFDPOT, CRUSSPIC STMPL, HICCUPPS, FCC CUTS VIDS (http://www.satisfice.com/tools/htsm.pdf, http://www.developsense.com/articles/2005-01-TestingWithoutAMap.pdf, http://www.testingreflections.com/node/view/2823)
Here are some questions for your confusion:
a) The functionality, what the software does from A to Z; what elements the software works with; the source code, from beginning to end; the requirements, the use cases, the test cases; the product as a whole, including documentation and support. None of these give the full picture.
b) The actual tests, their artifacts, the communication.
a) The attributes of the functionality; Usability, Performance, Security, Reliability, Installability, Compatibility, Scalability, Accessibility.
b) The process (both the documented and the real) used when testing; your techniques and approaches, your tools.
a) The (potential) users, which you probably can’t bring down to a couple of personas, but it is good to know what the users are like; what language they speak; what most of them knows.
b) The testers, your (different) skills. Will the developers also do testing? Will customers test the software, in order to accept the product or find bugs?
a) What do you think the users are trying to accomplish with the software? Can the software be used in many more number of ways as there are users? Are there any unintentional effects that are desirable? Why should the user not switch to your competititor?
b) What are the main drivers for the testing effort: Find all relevant bugs; provide enough information for the project to succeed; minimize customer complaints; learn and explore?
a) The environments where the software will operate and interact. Specific hardware and software?
b) Which environments can you afford to use; how far can you get with Basic Configuration Matrix (http://thetesteye.com/blog/2008/05/bcm-basic-configuration-matrix/)
a) Will the software only be used at one given time, at one place? Will upcoming releases replace the old version, or is it necessary to be forward compatible with currently non-existant software?
b) When will you test the software, and in which shape will it be?