test design technique name competition Rikard Edgren
When I read about the “classic” test design techniques, I don’t recognize the way I come up with test ideas.
Sure, the implicit equivalence partitioning is used pretty often, and I get happy the few times a state model is appropriate, but the testing I perform seldom has the unit/component focus that these techniques have.
Rather I start with a What If question I believe is important, e.g. what if the Client-Server connection is lost here.
And then I use various sources to find out how this can occur, and write test ideas in an easy-to-read format:
– investigate behavior if Server goes down
– check what happens if network cable is unplugged
– can the function be performed on a really slow network (use NetLimiter)
– check behavior in our load test environment
– what if the connection is interrupted before/in beginning/at the end?
– verify ability to Cancel operation
– are error messages adequately informative? (remember Security, Secrecy…)
– how will a user feel during interruption?
– look at information in the log files
The focus is on the analysis part, to find out what we want to test.
There is a mix of “what” and “how”, non-functional is not a limiting factor, and the details are left for the test executor to work out.
I think this way of designing tests is extremely common, and yet we don’t have a name for it! Or do we?
We should have a name competition.
Vote for one of these; come up with your own suggestion; or refute the whole idea…
* Straightforward Test Design
* Common Sense Testing
* plain English Test Design
* Test Idea Generation
* One-Liner Tests
* Test Design Un-Technique
* Error Guessing
* Test Design
* Risk-based testing
* Analysis-Focused Test Design
* Test Charters
* Zxcvbnm
* Simple Test Design
– So are you promoting a test design technique that anyone can perform?
Yes, I think anyone should be able to write, and understand these test ideas.
To do it good, you just need to understand what is important, see the whole, and realize how it is best achievable in tests.
And if you can write all test ideas with a granularity so all interested parties can get a grip of the whole testing effort, you will get good feedback, and have a great start of your testing story.
Why not just call it ‘What if test design’.
Exploratory Test Design
Intuitive Test Design
Heuristic-driven Test Design
Experienced-based Test Design
Creative Test Design
Wearing-Gloves-While-Testing-based Test Design 😉
Black-Box Test Design
Investigative Test Design
System Test Design (since the complete “system” is seldom described anywhere)
Your example is definitely system based, but I don’t see any reason in limiting it do this. Would you not use a similar strategy if you were asked to test just one component?
I kinda like “What-If Test Design”.
Or “Weakest-Link Test Design”
or maybe “Broken-Arrow Test Design” (arrow as in a diagram)
Well, this is of course based on my interpretation of your thoughts, so it might not be what you meant. 😉
I just got another idea;
“Holistic Test Design”
Many good suggestions! Keep’em coming.
In the design, I don’t feel limited in any ways (Free-Form Test Design Technique?) and can use any technique or skill associated with exploratory testing , experience-based testing, or error-guessing + “Mary had a little lamb”, checklists, Test Idea Triggers, Zoom In/Out et.al.
Originally, I think I wanted a name for a technique to put next to Equivalence Partitioning and the others.
But your suggestion indicates more of a system perspective, which I think is more important and better.
OK, I got it:
Exploratory Test Planning
We just call it a test case. On larger projects, this would be a negative scenario during integration testing. Usually the design documentation or operations guide details the timeout/retry processes.
Opportunistic Test Design
I called this Rapid Test Preparation, with inspiration from the RST.
Still, what you describe above is a part of all the preparation done.
I’d like to add Context-driven test design.
We normally call it:
System Break Down.
But the thing is – the ideas for all these What-If’s are what actually matters.
So you can have What-If’s based on functionality,
What-If’s based on Standards,
What-If’s based on Tester Knowledge and expertise,
What-If’s based on Boundary & Equivalence…
So who really cares how the whole What-If System Break Down is called? – That’s just Test Plan & Design.
I agree somewhat with Kobi, aren’t these just a part of the design and plan? Do you need a name for it? That’s just things we need to test.
Things-we-should- test-design.
I just saw Barry Schwartz talk about “Practical Wisdom”
http://www.ted.com/talks/barry_schwartz_using_our_practical_wisdom.html
With that in mind I would suggest to call it “Practical Test Design”. You create test cases based on your testing skills, based on your knowledge about the product and with the will to find all important bugs.
There is a point in not bothering with a name for this; that it is part of something bigger.
At the same time, names make our language and communication richer, it enables us to take things further, and discuss them better.
So the competition continues, and I’m specifying it to be about the technique of designing tests in a natural, plain fashion, “utan krusiduller” in Swedish.
It can be of any type, e.g. “investigate accessibility”, “reliability on concurrent usage”, “verify that the Help matches the functionality”, “use Unicode characters at applicable places” et.al.
The bigger picture is – bigger, and can not be solved in this competition.
We have a lot of good suggestions, and they have about one vote each.
The prize of this competition is a richer testing community.
Inquisitive testing?
It took some time, but finally I think I came up with a winner.
It is not “no flourishes”, since it is better to state what you got than what you don’t have.
Humanistic Test Design
Test design for people, that focus on what’s important, in an understandable way.
The old techniques can still be used, especially when using computers to evaluate Pass/Fail.
But when you are aiming at what’s valueable, not coverage, you are already using humanistic test design.