Common Sense Partitioning Rikard Edgren
– I saw you tested “42”. How come you didn’t try “43”?
– That’s obvious. It would be the same. No need to test something that would give the same result.
– OK, so I guess you are familiar with equivalence partitioning?
– Beg your pardon?
It is quite often said that testers don’t know the basic test design techniques.
This is both true and false.
“All” testers know how to use basic equivalence partitioning, it is common sense: you put things in groups, so you don’t have to test similar things that would yield the same information.
But they don’t know the name, and they don’t know the theory behind partitioning many types of groups.
But if they are testing all important partitions, is it a problem?
Might be, but probably not. I think a tester with knowledge about the product and its usage will make a common sense selection of the partitions that are needed to be tested for that particular situation.
For cases where you need to be very thorough this will include inputs, outputs, combinations; and extensions to examples like these:
Numericals: normal, inside/outside boundaries, min/max data type, different data types, decimals, precision
Strings: A-Z, ÅÄÖ, Unicode, empty, many, extremely many, control chars, leading/trailing spaces, things that mean other things (*, SQL, links, special values)
– You know you can’t be called a real tester if you don’t know how to do equivalence partitioning.
– Be so it. I just test what is important in my project.
I’m surprised that you haven’t pointed out the obvious logical problem with all this, Rikard: You can’t know the outcome of a test that you HAVE NOT PERFORMED. In other words, 43 will be the same UNLESS THERE IS A BUG.
Gee, if it’s okay not to test 43, based on an assumption that it will not fail, then why not skip testing with 42, as well. Skip all tests. Just assume the product has no bugs. (This is basically the same issue as Hume’s famous attack on inductive logic.)
We have an answer to this, of course. I expect you know the answer. But it deserves to be made explicit.
James, if 42 and 43 are within the same partition, why test both?
I agree you can never be certain about the outcome of testing 43 without actually testing it, but you can’t test everything, right?
I agree you can never be certain about the outcome of testing 43 without actually testing it, but you can’t test everything, right?
You have to be careful about using equivalence partitioning and boundary analysis since you most likely don’t know for sure exactly how it is implemented in the application…so even if you can’t test every possible combination due to time/cost it could be worth doing more than one test within an equivalence partitioning and/or a boundary.
James, not sure if I understand you here, but I’ll try to elaborate.
I don’t see equivalence partitioning as a logic, fool-proof way to reduce the number of tests.
I see it as a pragmatic way to make a “better” set of samples to test.
As Stian says, we can’t be certain of the outcome of 43 without testing it.
Actually, after testing it, we can only be certain it didn’t fail under the specific conditions used in my test.
But this is most often good enough.
Based on what we learn from tests, we can apply the Broader Partitioning Heuristic – If “this” was OK or Not OK, we can skip “that”.
If error handling fails for “A”, we might skip long strings or Unicode.
If we suspect something fishy we might apply Brute Force Heuristic – test everything.
We can’t prove to 100% that something works, but we can make educated judgments about what users will experience.
Stefan: “You have to be careful about using equivalence partitioning and boundary analysis since you most likely don’t know for sure exactly how it is implemented in the application…so even if you can’t test every possible combination due to time/cost it could be worth doing more than one test within an equivalence partitioning and/or a boundary.”
That is sound advice and worth applying for critical and/or especially important areas of the software under test. Thanks, Stefan
I completely agree. Many testers need up-skilling by formal training on testing. Especially those migrated to testing from Business Analysis.
Regards
Reghu
http://www.testingthoughts.blogspot.com
Good point about testing more than one item for each partition.
I select this by importance; the most important partitions need more tests than the less important.
Reghunath, I would rather say that some testers need more training.
The point of my post was that many already know this, and do a better job without the formal training.
I would rather say the opposite: the formal theory need more traing in reality; they say nothing about importance, and nothing about common sense judgment that make us do effective testing.
Rickard, I agree that commonsense and domain knowledge can do some good testing. But, such testing focuses more on what the analysts know and perceive and is often over focused on what they feel important and lean on other areas.
I appreciate the context driven school of thought that highlights this kind of knowledge, but is not a substitute for formal training ans structured thinking.
http://www.testingthoughts.blogspot.com
Yes, formal testing techniques capture things that aren’t important.