Exploratory Testing is not a controlled process Rikard Edgren
Exploratory Testing is not as widely used as it could be, because management doesn’t want it.
Stated reasons for this could be unaccountable, unstructured, sloppy, non-scientific etc, reasons that can be refuted by communication.
But I think the real reason is something Exploratory Testing can’t have: a controlled process.
Management/Companies want to have a plan with dates and costs; they want a test manager to be able to say how many percent of the testing is completed; they incorrectly think that software development is a lot like manufacturing.
Exploratory Testing can’t have this, because the testing will change as new information is uncovered.
Testing might be quick, might take a long, long time, might not be close to complete, might look outside the requirements, and discover so important information that the whole project needs to be re-thinked.
These are not good things for some managers, they want control and precision.
They want to see everything go smoothly and at the promised date we deliver what we said we would deliver.
The focus is on the control, not the result.
That is why managers can prefer to outsource testing to a company that runs the same script over and over until they all pass, than to a company (or ideally, in-house testers) that will change their testing strategies along the way, and leave the project with a bunch of fixed and unfixed bugs.
Exploratory testing is about learning, discovering new things and changing our mind.
And for software testing, which can’t be complete, this is a very good thing.
It enables better products, in a way that can be managed, but not controlled.
This is why it is so hard to sell Exploratory Testing:
First you must sell trust, which is priceless.
I think selling communication is important…
If I ask a PM what they have learnt between an 87% and an 88% pass rate – about which says more about the product, areas for attention and limitations then they begin to realise that they’re picture is insufficient – this is a way to introduce other ways about assessing the “story of the product”.
Also, asking a PM what areas of the product should be investigated more (eg more tests that are not currently defined) is a way for them to realise that the current picture of the product is maybe not the best/most appropriate picture…
These are some ways I introduce the idea of improving communication and the way we describe the product – a bit further down the road then ET comes into the picture – but by then it’s not such a scary concept…
Very well said Rikard!
Last week I was interviewed by Khurram Bhatti and Ghazi Nauman as a part of their Master Thesis “Effectiveness of Exploratory Testing”. In the interview there were a couple of questions about how I presented the idea about Exploratory Testing to my clients. When thinking of it, often my answers were about me being context-driven and promoting the appropriate testing for the current project and context. And since the test strategy was designed to meet the objectives of the project, there was never any discussions about me promoting an exploratory style as being most effective in most of the cases since it made sense for the stakeholders. I guess that it would have made a difference if I had only said that “I will solve this by doing Exploratory Testing”.
So the Trust was kind of built in from the start since the strategy was aimed to solve the test project in such a good way as possible and there were explanations to the selected techniques for appropriate areas (often techniques in combination with an Exploratory Testing approach).
This has been a really good lesson for me since I didn’t really care about the test strategy earlier!
(Another look at this problem can be found here: http://thetesteye.com/blog/2008/03/persuading-about-exploratory-testing-the-provocative-analogy-way/ )
Rikard,
I think what you really meant is to say is that you need to ‘earn trust’ first before the task of ‘selling’ the technique can be achieved. Trust must be earned first via a persons, or groups, actions. You need to prove first that Exploratory Testing is an efficient and effective technique of Software Testing. You have to show its worth.
Selling is something different in my book. It is about gaining acceptance and buy-in from the other party. Successful selling comes from relationship building and the earned trust that comes with it.
You are correct in the idea/statement of having to ‘sell’ testing and its techniques. The target group (management) you are communicating with deals with more concrete ideas such as schedules and costs, which is a part of control. They want to know the who, what, when and where answers to their questions. Provide that and you are going to earn their trust. Then you can start to talk in their language (time and money) about how Exploratory Testing can benefit them. You now have a ‘selling point’ for what you want to do.
Thanks Jim for the English lesson; I was happy with the sell/priceless conflict and didn’t see that the words weren’t the right ones.
I guess all 3 comments mean (and I agree) that you should focus on what the needs are, and how to solve the problems, and eventually boil down to details about techniques/approaches after you have shown that you have understood what it’s all about.
Starting with methods with capital letters is not the right way.
–
So here’s the follow-up thinking:
I’m a fan of using many radically different approaches in testing, but I don’t see it advocated often (Kaner is the big exception.)
Could a reason be that it looks a bit strange to managers if you (after earning the trust) say: “we’re gonna do Exploratory and Requirement-based Testing, but of course also some scenario testing and focued testing around Installabilty, Performance, Security; that will work well if the developers do a fair bit of unit testing, and the review process uses people with different background…”
It is easier to understand a statement like:
“With Risk-Based Testing we will solve your problems.”
…and then the different approaches can be used anyway.
I think both managers and testers want control to some extent, but we can rarely have it. When you sell Exploratory testing as a new approach it is quite common that after a while the customer would say, yes we do that but we call it “hum bug” or something. The test lead and test manager would not consider themselves to have control of the test information based on these exploratory tests. So, if you introduce Exploratory testing which is accountable such as Session-based testing you would infact give back some of that lost control.
Do you actually want to sell ET as one package? I see it as yet another tool to use in the way we work in order to make the test team more flexible among other things.
As a sales person you must have their trust to be able initiate any dialog.
“Do you actually want to sell ET as one package?”
If ET is defined as an overall approach, focused on learning and “freedom and responsibility of the individual tester”, then I would recommend and try to sell it at most places.
If ET is defined as an activity, e.g. “perform your next test based on the result of the previous”, I see it as one of many things a tester should do.
Guess what I’m trying to say is that the more control, the meeker result.
In that case, what do you mean by control and what is a controlled process? What do you mean by “Focus is on control, not the result?”
With a controlled process I mean “you know what is being done, you know when it will be done, and you can check if it goes as planned.”
Instead of this I think the focus should be on trusting the people to do things that provide value, and evaluating the results, together with the “workers”.
In software development, a lot of unexpected things happen. This should be managed, not avoided.