Yes, testers provide value early on, and teams can create more value with hybrid skills for everyone; e.g. more testing skills for developers would be great.
If all involved people know at least a little about the crucial areas, you have a diversity that can create good things, faster.

Verification is a part of testing, but we shouldn’t continue acting like it’s the only and most important thing.
We should get rid of tools, processes and mindset that make us only answer yes or no (even though yes/no can be a part of our information.

I guess your right that the classic tester is a low-skilled yay/nay-sayer.
We should change this.
Let testing be difficult.

However, providing better quality information implies having an understanding (however big or small) of what quality looks and feels like. That implies that the skill sets and responsibilities and skills understood as a tester’s domain has to be more than just thinking about how to break something. It moves from breaking, discovering problems to discovering, probing, pondering about better solutions.

When you move from verification to refinement it’s no longer a classical tester that you require. That move pulls me towards perhaps suggesting a cross over/hybrid tester/BA or tester/UX or tester/IA (or any other combination). I suggest the hybrid simply because a comment is usually more useful when it comes from someone that knows what they are talking about. But even then, I am hesitant to say that because the two mindsets are almost opposing in their intent and application.

Note, I am taking about comments that a user would give of “this feels awkward to use” or “the screen looks ugly”. If you want useful, constructive commentary, the language needs to provide more definitive explanation of what could smooth out a particular rough edge.

So, I have two conclusions:

The first is that I believe that the simply binary answer of yes/no is fundamental to the practice of verification and indeed, thus, it’s a part of testing. I don’t believe it should be excluded or removed. I say that because at the core of verification is the constant questioning process, taken on by the tester, to remove ambiguity and ambivalence in what is presented to them as “this is the product”.

The second is that there is space for a wider net of probing and investigation but it does require a wider range of skills and experiences beyond what a normal tester would have to identify potential changes, describe them adequately and document them. I would rather lean towards having the role already involved in the creation of whatever quality aspect you are looking at to review, test and make the judgement.

This is not a problem anyone really has, Henrik. The answers to these questions fall naturally out of an understanding of scripting and exploration. But the questions themselves are simply unimportant.

What *is* important is to be able to identify exploratory aspects of your testing (because we may wish to script them) and to identify scripted aspects (because we may wish to explore instead) and then to be able to evaluate the need for scripting and exploring respectively. That’s what matters. Not finding the exact place of your testing in some quantified version of the continuum.

“But I think that someone who has an exploratory testing approach is doing more or less exploratory testing; and someone who has a scripted testing approach is doing more or less scripted testing.”

These sentences aren’t saying anything. Since “exploratory testing” and the “exploratory testing approach” are exactly the same thing.

I don’t understand your other continuums. They do not seem much related either to scripted testing or exploratory testing, but in any case I can’t tell what point you are trying to make.

All good testing– scripted or exploratory– relies on skill. That’s not a differentiator between ST and ET.

“Control” means different things depending on the level at which you apply it. You could say that ET is required to assert control, or you could say ST is, depending on what you mean by that concept.

The same goes for trust.

Understanding the continuum is crucial to understanding ET. In order to understand it, it helps to work with descriptive examples. Contact me on Skype and I’ll work through some of them with you.

– James

You answered the question from a different perspective than I’d intended – but that was due more to the way I’d structured the question

One of the points I was getting at was that the continuum is useful to visualize /if/ you study testing – and probably study it from a certain (rang of) perspective(s). When you and I talk about the continuum we have a certain understanding of what’s involved (or not) – there’s a critical mass of previous information/learning (think what you might teach your students in term/year 2 that requires something from term/year 1).

I’m not saying this from an elitist approach – more an accessibility perspective – and then I started wondering what limitations there are with using this model – trying to explore it’s applicability and usability… Then I started thinking of a limitation due to non-study of testing.

A manager may influence the testing in ways due to the organizational latent (best) practices – a type of unknown unknowns approach (don’t realize other ways, and their relative importance and use) – that’s more a reflection on the organization and the influences it has been subject to. I’m thinking more (these days) how to tackle some of those problems – hence my line of thought.

Thanks, will continue thinking about this.

Having said that, this visualization of the different approaches is useful for anyone who wants to understand that approaches are influenced by several factors; it is useful in order to understand what drivers that are affecting the selected approach. It is also useful for anyone of us who wants to understand more about software testing.

This kind of visualization could help explain if you ever find a testing project where everything is completely scripted. I have been working in at least one such project.
If you say that very little testing is completely scripted, which I think is true, this is useful in order to understand from where the approach comes from. If you find yourself in a project where you perform some scripted testing and some exploratory testing it is important to understand the management drivers if you want to know what they think is important.
Let’s say that you in your project do a mix of 50% scripted testing and 50% exploratory testing. In this case it is indeed a large difference if your test manager have a scripted approach or an exploratory approach.
If the approach is scripted, the exploratory testing is a support to the scripted testing. If the approach is exploratory, the scripted testing is a support to the exploratory testing.

One question that comes to mind: When might it be useful to describe the continuum or even find ones place on it? Is it more useful for some than others?

In some of your examples I could read them as both symptoms of test management style or of tester style/approach or of latent best practice (due to cognitive fluency). I could also read them as symptoms of the different frames that organisations and testers apply to the test project.

If a tester builds their brand they’ll establish more trust and and gain more autonomy from control. Some people/organisations may happen to have one natural default style – due to factors in their work environment, connected with who they have worked with (tested alongside), experiences, openness to new approaches, their own attitude to uncertainty and inquisitiveness – or framing and cognitive fluency problems – these could be all apart from what they think about testing (whether in terms of study or reflection.)

In some industries (e.g. mobile handsets for the handling of the air interface) there is a “scripted requirement” – GCF conformance testing. These could be viewed as “checks” – although when I was involved with this type of testing (2003-4) I loved finding faults in the tests as well as the products. So in this sense I considered I was doing “scripted testing” rather than “scripted execution” – thoughts of this (and the continuum) triggered a short discussion with Michael Bolton on the interplay of scripted and exploratory approaches, here.

To a manager/exec/stakeholder – what do they want to know? They want information to help make a go/no-go decision.

Potential for controversy (devil’s advocate time):
Why is the continuum a useful description and in what context? Your examples polarise the scripted vs exploratory approach – I could suggest that I read a bias in your description against a scripted approach But very little testing is completely scripted, so how is the continuum useful, to whom, and under what circumstances?

Recently, I’ve started to think more along the lines of not polarising views – especially for managers and testers that do not study testing (it’s not a useful approach IMHO to exclude people because they can’t, don’t want to, or are not able to study testing right now – maybe they just haven’t been reached in the right way yet -> I take that as a personal challenge to try and find better examples). Btw, I find the check vs test distinction useful as /one/ way to illustrate elements of “good testing”.

There is a lot more to think about here – on many different levels – and I’ve only touched a small portion… Thought-provoking. Thanks!

When thinking about your presentation at SWET3, I remember humility in your content, but also that your “mind-changing” activities probably were a lot about inducing humility, both in managers and testers.

You’re right that it’s sometimes seen as a weakness or a bad thing – but it is the complete opposite IMHO – so much learning is associated humility and being humble.

I think continuous learning and development is aided by humility.

I raised the point about humility at PATS, and had meant to write something more about it – on the backlog. This is another reminder that I should do it – because I think it’s very important. Thanks!

@Michael, If we use your metaphor I would expect a report to give:
- I checked what kind of tree it was and it was an oak.
- I checked a random set of 100 leaves. 3 were made of rubber and nailed to the tree, the rest were actual leaves from an oak.
- I tested the tree as a whole and it seems to be healty, no dead limbs or any sign of rot.
- During my testing I noted that there was a hole in the tree that had been home to a squirrel, most probably.
- I checked the width and it was 2 meters wide which usually means it is 100 years old (made up result).
- During my testing I noticed that someone had tried to paste a limb from an apple tree on to it, but that experiment had failed. It did not seem to have affect the health of the tree.
- As for automation we can use a tree scan to determine the composition of the tree as well as the shape. This will naturally not give us a full report, but it will give us something.

We do want to avoid meaningless checks and instead turn them into valuable checks that give an answer to a question we would like answered.

@Rikard, I think we use Test Driven Checking and Check Driven Testing as strategies in our testing. It all depends on context and with the known pitfalls that come with it.

When verifying a bug fix I always check the repro steps first, and then test around the area,
UNLESS the bug fix is part of a more major re-write, in which case I’d do the testing first, and then the checking of the bug later.

I don’t think this is just personal style, I think it has to do with the main purpose of the code change.
If it’s a “small” thing, checking is suitable as the first activity (and if I see a problem, I can stop and communicate it.)
If it’s “larger”, testing is my best bet, so I can learn and understand how to test and check it best (bug verification is a side-activity, less important.)

Thus, the order of your testing should be arranged so you can find important problems, fast.