Testworthy Rikard Edgren
I have had some problems with the notion of Risk-Based Testing.
I mean, aren’t all testing based on risk in some sense, making the term redundant?
When using risk techniques, you come up with a list of areas to investigate first or most.
But what about those items that are extremely rare, but with very high impact?
What about not so risky things that would be extremely cheap to fix if we knew about the problems?
Imagine that Security is down-prioritized; that probably wouldn’t mean that passwords can be displayed in clear text, or publically available to anyone?
This is handled in many risk-based strategies by using light testing on items with lower risk.
But is the list of items in the risk assessment broad enough, and contain all relevant areas?
And how do you go from the general risks to the details involved in the actual testing?
There is also the notion of serendipity to consider; just a quick look at something might render very valuable information you didn’t know you were looking for.
So what about the tests that just take a few seconds to execute?
And what if the interpretation of the risk assessment contains mistakes?
I haven’t found a good way to express another way to look at it, until I stumbled on the word testworthy.
A test is testworthy if we think the information we can get is worth the time spending on it, regardless of risks, requirements, test approaches et.al.
The benefit of risk assessment rather is that stakeholders get involved with a language they understand.
Not sure if testworthy can help there…
Just need some clarifications.
Which definition of Risk-Based Testing are you using?
Or is it Risk-Based Test Management you are talking about; and in that case, which definition?
Testworthy can also be translated to Valuable, which is something we should consider in most things we do.
I was in a hurry before and my comment was perhaps a bit short and too direct.
First of all, I like the word “Testworthy” because it makes sense and differ from those tests that are Valuable. E.g., if there are 50% of the total Plausible tests that are Valuable there might be 25% that are Testworthy. In that sense a Testworthy test must be Valuable but not vice versa. Don’t you think?
But I don’t agree with (some of) your statements or questions regarding Risk-based Testing. Therefore I would like you to describe which definition for Risk-based Testing you are using (I know of at least two). And my follow-up question might have been leading, but I sensed that some of your statements were regarding Risk-based Test Management, and there might be several definitions for that as well.
I think “Valuable” is problematic, because we don’t really know in advance how valuable a result of a test will be. A Pass and a Fail might be valuable, and a Don’t Know as well.
Testworthy captures this better, I think.
All of my text, except the first paragraph is about (Product) Risk-Based Test Management (the last one I read was Veenendaal’s PRISMA)
I think thinking in risks is good, but when it becomes the sole baseline when choosing what to test, it is problematic in the ways described in the post.
But I don’t have a specific definítion I adhere to (is there a reliable software testing dictionary??)
Thanks for pointing out my ambiguous choice of terminology.
I see Valuable as in “The information from the test is of value no matter what the information is; it is valuable because it reduces my uncertainty for something I care about (Information = Reduced uncertainty)”.
It is an interesting journey when selecting tests; to go from:
Possible –> Plausible –> Valuable – >Testworthy
And not end up in Risky/Sloppy/Gambling.
Do you go straight to Testworthy without considering the other?
I like this presentation about different Risk-approaches and how they can be used: http://www.kaner.com/pdfs/QAIRiskBasics.pdf
Good example of valuable, so yes, it can be used.
I don’t have a whole theory around possible/plausible/valuable/testworthy. I have just started using “testworthy”, and I like it!
Kaner’s presentation is good. Risk-based test design is something you can’t do without. And as Kaner points out, you can’t do risk-based test design only; you need many different test approaches/techniques.
I’m a structural analyst. I use the term “testworthy” for a design that – based on the results of FEA – is deemed worthy of being tested. There is no need to test a design that probably will fail on the test bench or that is unlikely to meet expectations.
[…] Feel free to exchange to questions that suit you better, or to areas of your concern, e.g. which bugs to fix. This is not a water-proof method, but might yield the very best results, and is more straightforward than the notion of testworthiness. […]