Multi-Dimensional Software Testing Rikard Edgren
Thare are many ways to look at the problem of testing software, and it is rarely wise to use only one; that’s why there are so many mnemonics, e.g. SFDPOT, CRUSSPIC STMPL, HICCUPPS, FCC CUTS VIDS (http://www.satisfice.com/tools/htsm.pdf, http://www.developsense.com/articles/2005-01-TestingWithoutAMap.pdf, http://www.testingreflections.com/node/view/2823)
Here are some questions for your confusion:
1) What?
a) The functionality, what the software does from A to Z; what elements the software works with; the source code, from beginning to end; the requirements, the use cases, the test cases; the product as a whole, including documentation and support. None of these give the full picture.
b) The actual tests, their artifacts, the communication.
2) How?
a) The attributes of the functionality; Usability, Performance, Security, Reliability, Installability, Compatibility, Scalability, Accessibility.
b) The process (both the documented and the real) used when testing; your techniques and approaches, your tools.
3) Who?
a) The (potential) users, which you probably can’t bring down to a couple of personas, but it is good to know what the users are like; what language they speak; what most of them knows.
b) The testers, your (different) skills. Will the developers also do testing? Will customers test the software, in order to accept the product or find bugs?
4) Why?
a) What do you think the users are trying to accomplish with the software? Can the software be used in many more number of ways as there are users? Are there any unintentional effects that are desirable? Why should the user not switch to your competititor?
b) What are the main drivers for the testing effort: Find all relevant bugs; provide enough information for the project to succeed; minimize customer complaints; learn and explore?
5) Where?
a) The environments where the software will operate and interact. Specific hardware and software?
b) Which environments can you afford to use; how far can you get with Basic Configuration Matrix (http://thetesteye.com/blog/2008/05/bcm-basic-configuration-matrix/)
6) When?
a) Will the software only be used at one given time, at one place? Will upcoming releases replace the old version, or is it necessary to be forward compatible with currently non-existant software?
b) When will you test the software, and in which shape will it be?
Don’t forget “For whom?”.
The stakeholders that have interest in the application/product/project.
This would (and should) affect which testing activities to focus on.
That’s very true!
It is probable (but not certain) that the test activites will differ if the test team’s primary “customer” is the project team, the (different) stakeholders, the companies buying the product/service or the end users.
As I see it, there might be obvious stakeholders and stakeholders that might be more or less hidden.
A “customer” might be such an obvious stakeholder. It might then just be a matter of recognizing which of these “customers” that matter and take those expectations into consideration when selecting test activities.
The project manager is an example of an obvious stakeholder.
But how about the hidden stakeholders? And what are their interests?
See “The hidden project stakeholders” http://blog.thetesteye.com/?p=16 for more info.
[…] was originally a response to Rikard’s post “Multi-Dimensional Software Testing”, but here I have developed my thoughts a […]
[…] it make no sense to script all tests in advance and aim for 100% Pass? If you agree that quality is multi-dimensional , do you want to spend at least some thought and time for all orthogonal aspects? All due respect to […]