Not too long ago I had a test lead that I pretty soon recognised as someone that didn’t share my philosophy in software testing.

One day I reported to him that I had run all regression tests that were assigned to me.

All tests were executed on the same build (we had monthly iterations and this was done in the first week); and a couple of bugs were found and reported. I also verified a couple of bugs that I had reported earlier and that had been fixed and included in the current build. The suite took 3 days to execute and it consisted of rigorously documented tests that had little to offer to your fantasy. The system that was tested did not have any UI and the final results could only be checked via a secondary tool and/or by monitoring several log files; so it was a bit cumbersome to test.

I guess that I was pretty fast or efficient because the suite was supposed to take 5 days or so to run, so I was rather glad to have done this in almost half the time; and believe me when I say that I wasn’t sloppy.
And especially if it is a rather boring task I tend to think that it is better to do it as quick as possible, without being sloppy, in order to get rid of the boring stuff.

Well, after I had said that I was finished I expected to hear from my test lead what to do next.
Perhaps having a look at the new functionality that we should have tested last iteration but didn’t have time to do (i.e. we were a bit behind the schedule)?
Or perhaps helping the other testers out with their suite?
Maybe verify some bug fixes reported by others that were included in the build?

No, this was nothing that he said. Instead his response was:
“Hmm, OK… Could you test it again?” And I saw that he was trying me with this sentence because I caught a nervous smile in his face that he was trying to hide.

I just shook my head and said “Are you serious? What sense would it make to run it twice?”

He did not have any good answer to that. And of course I did not run the same tests once again…

But that incident made me wonder how many times he had done that to other people before. And in general, I wondered how many test suites are rerun several times on exactly the same configuration just in order to make it look good for some reason.

Well, it is always controversial to criticise the making of the Test Plan (http://en.wikipedia.org/wiki/Test_plan ). But here is an attempt that will leave some open questions for further discussions.

According to my experience, a test plan is a mandatory document that test managers and test leads often promote but seldom question.
Sometimes it is promoted and actually needed; sometimes it is promoted but not needed; sometimes it is not promoted and not needed but still produced anyway.
Why is this happening?
Is it because this is something we can do (by can I mean that this is something we done several times and therefore have developed a skill)?

A test plan is usually done in the beginning of the project and is an old remain from the V-model and Waterfall methods, where all activities are specified and planned in advance (hence the word plan).
But if these project models are out of date, why are we still doing the test plan as if nothing has happened?
Some might say that this, nowadays, is a living document and should change. But isn’t it about time that we call this document something else then? Or break it up into separate documents/artefacts/etc?

One of the main reasons of having a test plan, is the ability to communicate all the how’s, why’s, when’s, what’s, who’s, do’s, don’ts, etc, that regards the testing effort during a project. This is great, but since there will be so many things to cover, it is easy to forget some important aspect or info. Especially if this is something that should be produced in the initial project phase.
And also, what if these things change over time (ever been in such situation?). Is it then fair to expect that the test plan always is updated, at any given time?
Is it also fair to expect that all stakeholders should update themselves by reading this document say once a week? (These documents tend to be large after a while).
And is it fair to say that different stakeholders have different interests in the testing effort? Might they have different quality criterion on the info?

I would like to propose that in many (most?) cases it is good enough to have a Test Strategy to deal with what the testing mission is about; a wiki for documenting the latest useful info; the iteration plan or sprint backlog or similar to keep track of current tasks/work. This way we can address the right stakeholders with information served in the way it would be expected.
Is there something important that we miss if we would do it this way instead?