Exploratory Testing is not a test technique Henrik Emilsson

Well, to many people this is nothing new. But still, there are a lot of testers, and indeed test leads, that still think that Exploratory Testing is a technique that can be used in testing. To some extent, it has to do with that both Cem Kaner and James Bach have used this term amongst other techniques (e.g., in the BBST course material). But they have changed and updated presentations as much as possible over the last period of time.

According to Cem Kaner nowadays, the definition of exploratory testing is “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.

And this is important. You can come a long way on reaching the style of Exploratory Testing just by treating testers as intelligent people; which is one of the most important factors in the definition above. In contrast to Exploratory Testing you have Scripted Testing that, in my opinion, treats testers as dumb people or even dumb machines. I think that this approach is devastating for our profession (even though I can somehow see the need for Scripted Testing in some places).

A technique is a recipe for solving a problem, whereas a style (or approach) is a way of thinking around a theme that stretches far beyond solving a particular problem.
So when we talk about selling in Exploratory Testing to managers or project stakeholders it is not a technique that we are selling; it is rather an acceptance of a mindset where testers are treated as professional and intellectual human beings that are able to perform Sapient Testing, and particularly in an Exploratory way. It is not about stakeholders investing in a technique, it is about them showing that they have as much trust in testers as they have in other intelligent co-workers of the project.

Rikard Edgren April 17th, 2010

“treating testers as intelligent people” is the core of it all.

At the same time, the terminology is rich, and I see merit in both the usage of ET as style/approach and activity/technique.
I asked James Lyndsay about this and he said:
“I think we do a lot of exploring, and a lot of testing, and sometimes we do exploratory testing.”

We should also (with all due respect) be aware that ET was coined by Bach and Kaner, it wasn’t invented. It started as a description of skilled testing being done, and has evolved to a rich concept, almost essential when becoming a really skilled tester.
So we can’t really say that people are wrong when viewing exploratory testing as an activity/technique.

Nonetheless, I see value in both definitions, but think the broader style/approach definition captures more of what is important about software testing.
In the end it might win, but that would be easier if there was a different name for the “use the last test as input to the next”-activity.

Devon Smith April 17th, 2010

The QA’s mindset is so important, the knowledge they bring of the software, the issues they are expecting and their intuition about where bugs are hiding. Scripts can guide you to bugs, but the tester’s sixth sense about trouble spots is invaluable.

Making sure to hire the right people, give them space and responsibility to really get involved and take hold of the project and then take advantage of the different backgrounds and experience we all have that allow us to look at software in different ways. Love the term Sapient Testing.


Martin Jansson April 18th, 2010

The core issue, as I see it, is how testing is perceived. Is the intelligence in the person testing or in the test script written by a person? I think scripted test approach is based on the assumptions of taylorism while exploratory test approach is seen as a social science (http://www.kaner.com/pdfs/KanerSocialScienceSTEP.pdf).

If you perceive testing as a social science you will probably try to sell ET. If you do not then perhaps you think it is better to sell scripted test approach.

Dare we say that ISTQB does not see testing as a social science? Or isn’t it that simple?

Rikard Edgren April 21st, 2010

Another source of the terminology confusion is probably the 2002 book Lessons Learned in Software Testing, where ET is named as a technique.
Also, the short definition that is still in use:
“simultaneous learning, test design and test execution.”
is very easy to interpret as an activity.
For me, ET as activity/technique is something I do now and then.
ET as a style/approach is something I have, all the time.

Henrik Emilsson April 27th, 2010

Thanks for the comments, they broaden the topic in several directions!
The appreciation of the mindset; the importance of earning trust; the perceiving of testing; and proper terminology are all important factors when trying to sell ET. And this makes it clearer that ET is more of an approach than a technique.

The terminology confusion is interesting.
ET as technique could be seen upon as Ad Hoc testing, but that term has been disparaged and dragged through the mud. So maybe there is a need for a new word for the technique ET. Or perhaps for the approach ET.
Any suggestions?

One suggestion to the terminology issue might be to just use ET as an approach and then adding Exploratory as a prefix to existing techniques. E.g., Exploratory Specification-based Testing; Exploratory Risk-based Testing. This says something about the selected approach used in combination with the techniques. Analogously you use the term Scripted as a prefix for stating the approach when you want to have the benefits of Scripted Testing.
Using the prefix enables you to guide a tester what matters when it comes to the approach you want to be used for the testing area; it says something about what the tester should focus on; and it says something about the trust you have in the tester; and it also says something about the objective of the testing.

Also, what I meant in my post was that often we try to sell ET, the approach, as a technique. But you wouldn’t need to try so hard to sell any other technique used to solve testing problems. It is often no controversy using the Specification Based or Risk Based Testing techniques in order to meet your objectives.
But when we try to sell ET we probably mean the approach which makes it harder to sell, this since it means that management (think that they) might loose control and need to decide upon an approach that might interfere with their ambition or ability to manage actions, see http://thetesteye.com/blog/2010/04/exploratory-testing-is-not-a-controlled-process/.
A decent comparison is to think of selling in Agile Development (approach, several methods) vs. Test-driven Development (technique).

Rikard Edgren April 28th, 2010

I have some suggestions for “new” names:
activity: ad hoc testing, exploratory ad hoc testing, unscripted testing, freestyle testing, creative testing.
approach: good testing, thorough testing, holistic testing, grounded testing, intelligent testing, usage-centered testing.

Henrik Emilsson April 28th, 2010

technique: oracle-driven testing, jazzy testing, inspiration-based testing, sapient testing, evolutional testing
approach: sapient testing, people-centric testing, evolutional testing, trust-given testing