The Metrics Tumour Rikard Edgren
quantitative numbers in a world of qualitative feelings
I am not against measurements in general, they can surely be useful. I use length when building things, weight for baking, time for appointments etc.
I often use numbers for various things in my bug reports.
But metrics are something different; metrics are measurement plus value.
“Should have at least 80% code coverage on unit tests” is a typical example, where “peer reviewed and accepted” would give better results.
“2% better defect detection percentage since last release!” says less than conversations with support department.
“95% Pass on test cases” means nothing at all.
The measurements with value cannot judge what is important;
reality is impossible to aggregate;
metrics are dangerous.
There are many good software testers that advocate metrics. At a couple of occassions I have had the chance to have “the talk” with some I respect.
It boils down to the same thinking: metrics must have a lot of context in order to be useful, so much details that I believe you could throw away the numbers, or only have them as a footnote.
This is not done, because management demands numbers, that’s what they are used to base decisions on.
But for software and testing that might not be well applicable.
If kids want candy, it doesn’t mean you have to give it to them.
You give them better things, out of respect and care.
Testing provides information, so you should find out what is important to different people, and give them what they need, not necessarily what they want.
Metrics will hide importance, and also skew the development efforts.
Try to take them out, with surgical precision.
Then you are left with one final catch:
Qualitative information is hard to aggregate, trust is needed.