Jerry Weinberg has, in his book ”Perfect Software and other illusions about testing”, expressed a very important observation, namely who is responsible for pinpointing the bug. The tester finds the bug, tries to reproduce it, then adds as much information that he/she has such as log files, configurations, test data and so on. When you estimate time for testing I think you most often consider the time that you do to find the bug and then make a report. As Jerry points out, the time it takes to pinpoint the exact location is not taken into consideration. It might also be unclear who has this responsibility.

This might not be an issue where there is high testability, when it takes little time to report the bug and when the actual pinpointing takes little time. When testing complex systems (a system consisting of several sub system, different hardware, many interfaces, a vast amount of ways to get information on what actually went wrong), the time for pinpointing can be a factor that is underestimated and that can really be a major issue in a project. Reproducing a bug can take several hours or possibly days. If you, after having reported the bug, are asked by the developer to add extra or missing information to the bug it might cost you days to get into the same erroneous state. In some cases it might be close to impossible because of missing equipment or missing/broken units. When it takes this long time it is inevitable that a conflict grows in the project. It is also inevitable that the testers start to doubt how much time they should spend on bug reporting (in the worst case scenario consider not doing it) and if they can help developers.

How do you handle pinpointing? Is there any need to increase testability to make it easier to report bugs and get information from the sysem? How have you discussed and communicated this within our projects? Is your testlead and project manager aware of this time sink and uncertainty?

If you want to taste some of Jerry Weinbergs knowledge I recommend reading this book.

You are on a quite large company where there are several QA divisions, several layers of management, several listeners to each step of the development process. It is the final weeks of the release. You are about to enter a bug which seem serious but you are not sure. You can take at least two paths here…

Take the easy path and you report what you have seen so far and scribble down a few quick repro steps on what you think might be the bug. You do not dig deeper cause its been a long week and it is your right to go home now.

… or …

Take the harder path and you really try to hone down what is wrong. You try to find some repro steps on what it is. You try to see how often this bug appear and what impact you think it has. You extract log files, config files and all other information that you think might help the developer. You proof read the bug report and make sure you have gotten everything in there. You let someone else take a look before entering it.

As the submit of the bug takes place you see that there were many who got the bug in the mailboxes as a final delivery at the end of the week.

Chosing the easier path might have come to a situation where you as an observer…

• do not believe the bug
• ignore the bug because it seem too vague
• think the testers are crap and become unsure if their previous bugs were infact not correct as well
• are scared that the quality of the release is in danger because you have no trust in the bugs
• become angry because the testers do not take their job seriously and enter a bug in such bad state in such a critical moment
• become angry as a tester thinking that the other test team gives you a bad reputation

The list could go on for some time…

Chosing the harder path might have come to a situation where you as an observer…

• are thankful as a developer that you did not have to work the weekend to try to find the reprosteps
• are thankful as a project manager that you were able to make a bug classification more easily before everyone had left for the weekend
• are impressed as a manager that the testers give the extra effort to save so much time for others
• are thankful as a tester that the other testers take pride in their work and deliver at every step

The easy path is quicker while the harder path takes a bit longer time. Depending on your thouroughness and completeness the result will differ enourmously. Naturally this is not always the case, but I’ve seen this happen so many times.