Introducing exploratory testing in a scripted test environment Martin Jansson
In many organisations it is hard to change how you are working. You might be bound to certain CM tools, how things are expected to be planned, documentation systems, management expectations, project management expectations and so on. In many of these traditional environments you might also use the regular test plans, test matrices, test specifications, test cases with expected result and test record.
If you are doing scripted testing it might sometimes be hard to handle the cases that fall a bit outside the scope, that mess up the planned activities. Some might say that it is better to stick to the plan and leave the vague and hard to reproduce thing for later.
To totally change the way you work might affect many things, so you might want to start out with small changes. If you are in such an environment and consider how it would be possible to try out exploratory testing as an approach to testing I have a few suggestions.
One way is using your current test cases as a guide to where you intend to test, then you use exploratory testing on those areas. The test cases will in this case just map lots of areas that need to be looked at and what you actually cover and what you expect is up to you. This means you will certainly skip a lot of the content of the test case. If this is accepted that is perfect, if not see how you can get away with it.
Another way is executing the test cases and each time you find something outside the test case or something fishy along the way, you create a work page/task for this issue. You then assign someone or a small group to dig deeper into this area using the exploratory test approach. All issues that are outside the regular plan are handled as an exploratory test.
You will report progress and result as usual, noone will know that you tried a new method in secret.
Each time you use exploratory testing you do it as a defined session, you can google how this is done. There are a multitude of good articles out there on how this can be done.
Hi Martin
[ You will report progress and result as usual, noone will know that you tried a new method in secret. ]
I think this is not the right way, what to do in this case if you find an critical defect? There isn`t a scripted test case to link with, the way you find it is that strange you`ll never find it if you`re using formal design techniques.
In my opinion tell them always what you`re doing, there is in each project some extra time (friday afternoon) or waiting on a certain batch job.
My vision is: Don`t be sneaky be open this is your added value as a test expert
looking forward for your comment…
@Andreas,
Thanks for your thoughts on this.
If you find critical defects you handle them as you always do scripted testing or not.
I agree with you totally that you should be open and not be sneaky about such things, in general. But when you are stopped by politics or any of the issues I listed above, this is one way of doing it.
Many big organisations are stuck with the thought they have control over quality and this can create a crippled test organisation. You are stopped from being creative, stopped from doing changes quickly and it is hard to be flexible. The need for propaganda and a well thought plan on how to make the changes ripple through the organisation is evident.
Naturally this does not apply for everyone, but this is a recipe for those who feel familiar with the situation.
I am so happy that I don’t work in such an environment.
But if you are, I think you should be honest.
An hour or two to investigate something that looks fishy would probably be allowed everywhere, and hopefully you get some good results that you can show, and say that we could find more of this kind of important information if you were allowed to test freely from time to time, using what you have learned from the product so far.
If results aren’t persuading, I would either try again, or look for a job at a place where building a good product is a part of the recipe.
If you are a work place where there are perhaps thousands people involved in making a product suite, you will step on toes, there will be conflicts, you will meet people with different agendas or different ideas on what your area is about etc. Things get more complex.
At one time in our life I am sure we will experience something like it. I guess we act differently, either we escape and do not want to work in such an environment or we see it as a chance to make improvements. My recipe above is one of those suggested improvements, even if it can be considered shady.
Do you always say the full truth when talking about quality or do you wrap it up somehow? I think it is too easy to say that you are honest when talking about quality, considering that you are subjective and talking about something abstract.
I was wrong. It is much better to start with session based testing. It is easier to use as the first step, testers and their managers seem to see the benefit of it easier.