fast and frugal tree for test triage Rikard Edgren
There are situations when you have to choose to run a test or not.
Some organizations quantifies properties like time, risk and get a prioritized list.
Most probably just use their intuition, but if that’s not enough, or you want to explain and share the reasoning, you can try using a fast and frugal tree (Yes, I got this idea when reading Gigerenzer’s Gut Feelings.)
Fast Frugal Test Triage
If the answer is Don’t Know to any of the question, keep the test on the list until you know more.
Feel free to exchange to questions that suit you better, or to areas of your concern, e.g. which bugs to fix.
This is not a water-proof method, but might yield the very best results, and is more straightforward than the notion of testworthiness.
Your decision tree says that if you can get new, relevant information, and if the test is fast (whatever that means), then you should run it – even if it isn’t important to anyone.
I think I’d reverse the bottom two decisions.
The order I’d probably use would be:
1. Can we get new, relevant information by running the test?
2. Is it important, to someone?
3. Is it really fast to execute and evaluate?
The reason I put the tree like this is serendipity/the unknown.
If we only run tests we know are important, we might miss information that fast tests provide.
(BTW, I updated your comment so the last two were switched.)
Also, the tree can be different for different situations, e.g. in the beginning of a project, the first question probably is:
is it possible to run this test?
I like this! Easy, yet solid.
However, if serendipity/unknown is important, I guess that it will be hard to ever say “No” to the first question since serendipity could be applicable in both the “Yes” and “No” cases. Saying “No” means “We cannot get new, relevant information by running the test” which will have to be a really narrow case in order to not reveal anything by serendipity. Right?
How about reframing the first question to “Is the test (question) open enough that we can get new, relevant information by running the test?”
Good point, Henrik.
However, we want to distinguish tests to not run as well.
And it is not a fool proof system.
So we will make a (small) mistake for those tests that doesn’t seem relevant, but would have enabled interesting findings.
But that’s probably OK, just as the intuition this decision tree mimics, it won’t be perfect.
And any reframing is allowed, the purpose of an exercise like this is to make your intuition a bit more transparent.
As I’m writing this, I realize it would be possible to make sub-trees for what “relevant” and “important” is; but to make that frugal, you’d probably need to be context specific.
I love this chart. Great idea. I will hang it on my wall and share it with my testers. Thank you!
This is great 🙂 so simple and yet many still don’t seem to ask the first question.
I will be sharing this.
Great idea.
Inspired me to create a bit bigger one – http://bit.ly/eWKb7O
I think i’m going to use it for explaining why and how I test. Making that helped me understand what things to consider when considering running a test.
And to remember that it is a heuristic not a Rulebook 🙂