Exploratory Testing vs. Scripted Testing – rich terminology Rikard Edgren

Exploratory Testing in its purest form is an approach that focus on learning, evolution and freedom.
Cem Kaner’s definition is to the point: “Exploratory software testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”

ET in real life is a collection of ways of testing, sometimes used as example implementations of the approach, sometimes used for testing that don’t follow a script, and sometimes as a synonym to ad hoc testing (because that word has become increasingly under-rated.)

Scripted testing in its purest form is an approach that focus on precision and control. It is yet to be defined by proponents, but a benevolent try could be:
“With a scripted testing approach the testing effort is controlled by designing and reviewing all test scripts in advance.
In this way the right tests are executed, they are well documented, progress towards 100% execution can be controlled, and it is easy to repeat tests when necessary.
The scripted approach is not dependent on many tester heroes, and can take advantage of many types of resources for test execution, since the intelligence in the test scripts are created by test design experts.

Scripted testing in real life mostly means designing test scripts early, and executing them later, and these scripts have quite detailed steps, and a clear expected result.
The terminology is rich, complex and sometimes confusing, since they at least can mean approach, style, method, activity and technique; and these are in reality so connected and intertwined that distinctions aren’t necessary or helpful.

 

So are the distinctions important?
I think they can be, especially if the words are used without details, e.g. in statements like “Exploratory Testing is the opposite of Scripted Testing” or “combining exploratory and scripted testing”.
Both statements can be true, because the first talks about the approach, and the other about methods.
By understanding the different meanings of the words it is possible to get a more nuanced debate, and to see other combinations, e.g. test scripts with an exploratory approach, or scripted approaches with elements of ad hoc testing.

The intention of the used method shows which approach you are using (inferred from Cem Kaner’s Value of Checklists… p.94):
If test scripts are used to control the testing, it is a scripted testing approach.
If test scripts are used as a baseline for further testing, it is an exploratory testing approach.

 

It would be nice to have a solution to this semantic mess, but I don’t think it is feasible to always attach approach or method to Exploratory Testing and Scripted Testing (or to distinguish between upper case Exploratory and lower case exploratory.)
It is extremely difficult to give life to new words, but I do have some hope in the clarifications by testing vs. checking, and less hope for a renaissance of ad hoc testing.
A start would be if more people are aware of the different meanings, and are more precise when necessary, and eventually the problem will dissolve, in 25 years from now.

2 Comments
Justin Hunter November 17th, 2009

Rikard,

Nice post. It is a topic I’ve been thinking about a lot lately.

FYI – There’s a discussion about the general topic here. I just added a link to your post to it:

http://testing.stackexchange.com/questions/237/lessons-learned-from-combining-exploratory-testing-and-scripted-testing

Rikard Edgren May 23rd, 2011

It took me a year and a half to make up my mind about which of these two to use (activity or approach)
I think it is both. It’s like the light, which is both a wave and a particle.
This of course also goes for scripted testing, it is both an activity, and an approach.

I think this is good; you get a broader terminology, that isn’t wrong to use in any of the ways (although misunderstandings can happen), and there will be more powerful follow-up concepts, e.g. exploratory test design, that can be both on-the-fly-testdesign, and investigation of multiple information sources.