Rapid Test Preparation Martin Jansson
At many bigger companies where you have several teams working with the same test scope there is often a need to communicate how you intend to test something and what areas you intend to cover. It is common that you have a test plan and perhaps a test strategy, they usually do not go into detail on each feature or area that you intend to test. Instead you create a test specification containing test cases with repro-steps and lots of information so that anyone can execute the test case. The test specification need to be reviewed before you commence with testing. Many times you assign one person to be responsible for creating such a spec. It is not uncommon that you set someone to write on this test specification for several weeks. If you have never seen this situation, then good for you!
The rest of you who also see this as a fairly common situation… here is a different way of working with test specifications (or rather not working with them).
Let us assume that you are a member of a test team that is on one test site out of many in a project. The head project manager asks you, as a test lead, to give an estimation on how much time you need to system test feature X. At the same time the head project manager asks another test lead on how to test feature X, but at a lower test level such as integration or function. Instead of giving an estimation directly, the test leads asks to prepare and plan a bit then come back shortly.
Instead of doing it the regular way, as in both test leads giving an estimate to the project manager without communicating or writing down their thoughts, one of the test leads start writing on a test document, namely a Test Proposal (TP). All ideas, risks, work packages, unresolved issues, todos, estimations, contact persons, etc are listed in this TP. The test lead can show the rest of the team the thoughts that have guided him so far and can assist with the rough estimation. Shortly they can return to the project manager with an estimate and explain roughly the plan outlined in the TP. It is not uncommon that some of these features are not able to be added to the overall project, so the test lead does not spend any more time at this stage. If possible the two test leads have cooperated and written in the same TP, but worked on different perspectives and areas. A lot of information would have been the same for both test leads.
A few weeks later feature X appears on the radar to be included in the release after all. The test leads or those responsible for the feature can now continue preparing. A group of two or three testers, depending on size of feature, form up to further prepare the feature. If the feature is available to be looked at in the release, the group starts do a minor test in order to understand how it worked and getting a feeling what to expect. Then they sit down and brainstorm on the feature further. More content is added to the TP and eventually it is ready to be shown to business analysts and developers responsible for the feature as well as to the other test teams working on the same feature. The TP is quite short because of the one-liner test ideas, thus effective to be communicated and reviewed. After a few rounds between the different parties the list of test ideas, open issues, work packages and estimations etc begin to feel ready (as far as ready can be in testing). The TP is therefore approved and the test teams are considered to be prepared to start testing. The total time spent is probably a few days at the most.
Comparing this method with how test specifications and estimations are done, you will by using this method not loose as much information since you collect your thoughts in the TP. It is otherwise common that the person doing the estimation does not give any information on what was included.
Reviewing and communicating a test specification containing all your test scripts with all your repro-steps is incredibly hard. It is not effective to give feedback on huge documentation, no one usually have that much time to spare. Instead the TP only contains the guiding thoughts and is easy to review. A test idea is usually a one-liner with the essence of the test intended, which is perfect for communication.
The key to this is collaboration and cooperation in the team as well as with other stakeholders. You focus on learning a new feature and the domain around it while at the same time hone on your test design. The power of the TP lies in it being focused on the essentials of the planned testing where you do not go into too much detail instead have the rough scetches and therefore making it easy to communicate. This method works well in combination with SBTM and other styles of exploratory testing.
Hi Martin,
Firstly thank you for taking the time out to share this. I think it’s excellent, very lean and effective. The idea that you build it up over time prevents large wasted effort as like you said it’s down to *if* the feature is accepted. I’ve found many a time I’ve wasted time writing test plans for proposed features, only to find these are later dropped from scope to never return. We’ve tackled this by making our test plans very lean, however I like your idea better, as it would make for very smooth reading & blend well into our teams lean documentation structure. It also sounds like it provides for more extensive requirement review due to the test idea format, being very short, sharp & informative, thus meaning your mindset will be more tuned to generate additional ideas.
If possible I’d love to see an example, I already have in my head what this would look like but would love to see what you & your team has done.
Again, thanks for taking the time out to share this.
Cheers,
Darren.
I like the sound of this.
There are elements of the TP idea that I use in certain test areas – the idea being on catching the one-liner thoughts, ideas and having them lightweight enough to review and understand and even drop easily (ie we doesn’t ‘cost’ much to not use or progress ideas).
I see a value in this TP idea where it ties in different test phases around a common thread/document/database – showing the groups thinking and provenance of their ideas. It reminds me a little bit of the test framing idea (in a distributed sense).
I need to do some more thinking about this.
Interesting!
Thanks for the comments.
I focused on writing down the things that I got frequent questions on in the beginning of the preparation. I then saw that I had ideas on testing from the first second someone mentioned the feature. I collected these ideas as possible tests, some where bad but some survived to the end. Some of the test ideas became missions for SBTM other became Checks for regression.
I noticed an engagement in the group that I had not seen before compared to when writing the regular 100 page test spec. Performing the task of preparing for a test felt like a little mind exercise for a small group who then could return to testing after a while.
One of the most important questions you have to ask yourself when planning and preparing for testing is “Does this give any value?” or “Do I spend my time on the right thing?”. It is common that you get some kind of allocation of time and you have to split that time between preparation and actual testing.
This concept is not really new, I’ve been doing it for the last 7 years, but I just got a new name to it (named by a college) and added a few more things that is important in larger organisations.
BR,
Martin
I like that you have looked at this from the perspective of a large company.
Interesting read. If you have any actual examples that you can share that would be great.
Best regards,
Johan
I will make an example of a Test Proposal for an area on Session Tester. The current ones I have now are owned by my client.
I will post a blog article about an example TP here later on.
I like this a lot!
The name is really good, since it indicates that this is something you can comment on, and something that is evolving as we learn more.
When I get the chance to do test documentation the way I like, I tend to do a lightweight test plan (1 page), together with a Test Specification with one-liners, that I update with results and new ideas as they emerge.
Your suggestion seems richer, and if people read and comment, it is perfect.
Regarding one-liner test ideas, I have noticed that the most interesting ideas are those outside the requirements.
So instead of listing many of the obvious, you could write “verify that requirements are met”, and then give better visibility to all the interesting things outside explicit requirements.
Test Proposal should be part of Exploratory Testing 2.0
I agree with Rikard on that it indeed is a very good name that invites to feedback and suggests that it isn’t final.
I was thinking of also including a short test mission in the Test Proposal; like 1-5 rows of information objectives. A test mission that guides on what is extra important and that would meet the stakeholders quality goals.
What are your thoughts on that?
I think that is a good idea Henrik.
If I added what I had included to an example document here, perhaps we can edit it and add more content so that we can play with this idea further.
[…] is a follow-up on a previous post about Rapid Test Preparation. Some of the commenter’s asked for an example; I’ve tried to go half-way at least. Added […]