The impact of a good or bad bug report Martin Jansson
You are on a quite large company where there are several QA divisions, several layers of management, several listeners to each step of the development process. It is the final weeks of the release. You are about to enter a bug which seem serious but you are not sure. You can take at least two paths here…
Take the easy path and you report what you have seen so far and scribble down a few quick repro steps on what you think might be the bug. You do not dig deeper cause its been a long week and it is your right to go home now.
… or …
Take the harder path and you really try to hone down what is wrong. You try to find some repro steps on what it is. You try to see how often this bug appear and what impact you think it has. You extract log files, config files and all other information that you think might help the developer. You proof read the bug report and make sure you have gotten everything in there. You let someone else take a look before entering it.
As the submit of the bug takes place you see that there were many who got the bug in the mailboxes as a final delivery at the end of the week.
Chosing the easier path might have come to a situation where you as an observer…
- do not believe the bug
- ignore the bug because it seem too vague
- think the testers are crap and become unsure if their previous bugs were infact not correct as well
- are scared that the quality of the release is in danger because you have no trust in the bugs
- become angry because the testers do not take their job seriously and enter a bug in such bad state in such a critical moment
- become angry as a tester thinking that the other test team gives you a bad reputation
The list could go on for some time…
Chosing the harder path might have come to a situation where you as an observer…
- are thankful as a developer that you did not have to work the weekend to try to find the reprosteps
- are thankful as a project manager that you were able to make a bug classification more easily before everyone had left for the weekend
- are impressed as a manager that the testers give the extra effort to save so much time for others
- are thankful as a tester that the other testers take pride in their work and deliver at every step
The easy path is quicker while the harder path takes a bit longer time. Depending on your thouroughness and completeness the result will differ enourmously. Naturally this is not always the case, but I’ve seen this happen so many times.
“but I’ve seen this happen so many times.”
Have you seen the easy path or the hard path often?
My experience is that one bug once in a while doesn’t make a big difference, but many bug reports together can bring the phenomena you describe, of course depending on the style of the developers you are working with.
The thorough bug report takes time from “normal” testing, but quite often you find more bugs while digging into the first one.
… I have experienced an even easier path that some people tend to take in a similar situation:
Do not report the bug at all!
You do this of several reasons, but mainly it means that by reporting this you will get into trouble: this will render more job and perhaps give you attention you might not want.
The last thing about attention is interesting because it might reveal other bugs that you should have seen earlier (sloppy testing) and therefore you have turned yourself into a scapegoat for many problems!
So by not reporting the bug at all, this might be discovered late enough for you to have moved on to a new project and thereby not risking your job or reputation…
In quite large companies where project includes outsourced team members and several departments from different countries, your effort is often measured by something else than a “job done well”. E.g., a subcontractor might be measured by the number of bugs that they are responsible for; and they will try to shun the bug around to other departments or make you as reporter do insane tests in order to win some time (I have experienced such cases myself). This will lead into situations where you consider not to report the bug because it ultimately will render more (unnecessary) work in order to “support” the blameable culture that is bred within such large companies.
I am sure all of us have taken the easy path once in a while and sometimes we have seen the effect of both.
“The thorough bug report takes time from “normal” testing”, that depends. If you spend 5 hours to give a good report and it does not give more work to other testers, it might be better than spending 30 min on a bad bug report that gives 10 or more hours of work to other testers. In larger companies the ripples of information spreading are so much larger than in smaller companies.
“Do not report the bug at all!”, that is a terrible path… I must have mentally blocked it.. I think this is too common and sad to say I think many managers stimulate this behavior by thinking it is good that no bugs are found.
Jonathan Kohl has written a good blog about bug reports, you can find it here:
http://www.kohl.ca/blog/archives/000207.html