Test Plan – an unnecessary artefact? Henrik Emilsson
Well, it is always controversial to criticise the making of the Test Plan (http://en.wikipedia.org/wiki/Test_plan ). But here is an attempt that will leave some open questions for further discussions.
According to my experience, a test plan is a mandatory document that test managers and test leads often promote but seldom question.
Sometimes it is promoted and actually needed; sometimes it is promoted but not needed; sometimes it is not promoted and not needed but still produced anyway.
Why is this happening?
Is it because this is something we can do (by can I mean that this is something we done several times and therefore have developed a skill)?
A test plan is usually done in the beginning of the project and is an old remain from the V-model and Waterfall methods, where all activities are specified and planned in advance (hence the word plan).
But if these project models are out of date, why are we still doing the test plan as if nothing has happened?
Some might say that this, nowadays, is a living document and should change. But isn’t it about time that we call this document something else then? Or break it up into separate documents/artefacts/etc?
One of the main reasons of having a test plan, is the ability to communicate all the how’s, why’s, when’s, what’s, who’s, do’s, don’ts, etc, that regards the testing effort during a project. This is great, but since there will be so many things to cover, it is easy to forget some important aspect or info. Especially if this is something that should be produced in the initial project phase.
And also, what if these things change over time (ever been in such situation?). Is it then fair to expect that the test plan always is updated, at any given time?
Is it also fair to expect that all stakeholders should update themselves by reading this document say once a week? (These documents tend to be large after a while).
And is it fair to say that different stakeholders have different interests in the testing effort? Might they have different quality criterion on the info?
I would like to propose that in many (most?) cases it is good enough to have a Test Strategy to deal with what the testing mission is about; a wiki for documenting the latest useful info; the iteration plan or sprint backlog or similar to keep track of current tasks/work. This way we can address the right stakeholders with information served in the way it would be expected.
Is there something important that we miss if we would do it this way instead?
Some companies use the Test Plan as a check point when reaching a gate, thus there must be some kind of testing planned before going to the next stage. When there is little structure and organisation this can be a one way of forcing test leads to produce something for others to see.
I myself use the test plan to organise my thoughts early in the project. In the end I usually forget to update it and discard it. I do not think anyone but myself has read it at this stage, but the more testers and test leads who are involved they might be interested and be part of it.
A lot of the information in the Test Plan would better suit a Test Strategy as you say. I think that you might be a bit coloured by the way you work, thus trying to be as flexible as possible. But in some cases you are not expected to be flexible and the more inflexible you are the more control you have? This should naturally be questioned.
A project manager once wanted me to tell them how much test effort was needed and if we were going to make the dead-lines. When I had a very detailed test plan I could point him to that and say that everything is planned and we have the situation under control. With experience we know that this is never the case, on the test side we very seldome have control and cannot plan for it.
But using the Test Plan to structure our thoughts, list important information or where to find it, discussing available resources, dates and so on. This would not fit into a weekly tasks, backlog or whatever we call it.
I agree that a Test Plan should never be written unless you as a tead lead has need of it as a tool, or if you are required to write it in order to meet a requirement for a gate meeting or the project itself. For instance companies like Ericsson use the documentation as a way of pushing the project forward.
I think you might be missing the old saying:
“the plan is nothing, planning is everything”
It also seems that you aren’t attacking the Test Plan, but rather the over-documented test plan, with too many unnecessary and narrowing details.
In a couple of projects I experimented with a Test Plan that wasn’t allowed to be larger than one page. It worked quite OK, but might not be sufficient in an audit situation.
I was attacking the Test Plan; which is in most cases done as an over-documented test plan. Why is it done this way? Because this is the “best practise” when it comes to the testing community.
I might have been unclear in my writing, but my meaning was that “planning is everything” and should therefore be a continous effort. And there are perhaps better documents/artefacts that could deal with the planning outcome.
Would you say that your one-page test plan is more similar to a strategy on how to test in the project; or to a traditional test plan?
My one-page test plan was a strategy, and I think that all test plans should be strategies.