Exploratory Testing – the learning part Henrik Emilsson
Let me begin by a quote from T.S. Eliot:
We shall not cease from exploration, and the end of all our exploring will be to arrive where we started and know the place for the first time.
One of the most important things with Exploratory Testing is that it allows for you to learn something during the exploration. It is urging you to think for yourself; draw conclusions; misunderstand; learn from mistakes; see the full picture; noticing details.
Learning about the product and the context is a key element in discovering problems that might bug users and stakeholders.
I have a problem with remembering and learning stuff if I (or someone else) have written something down and I’m using the text as the answer each time. E.g. a set of instructions on how to perform a special action; a login password; etc. But if I have done the thing myself, I have no difficulties in performing the special action. And perhaps most importantly, I have an understanding of every needed step and why each step is needed.
When I am asked to review something, it is important that I explore the area in order to learn something from it. To me, this is (almost) the only way to see what is missing. I can do a good job by finding problems in the things that are already in there, but in order to see what is missing I have to learn about the area and what it really means.
It is the same thing with Scripted Testing vs. Exploratory Testing. If tests are scripted, there is no need to reflect upon what you do. The script is there so you don’t have to think. An exploration, on the other hand, requires thought work. It requires that your brain is in an alert state and that you learn something from what you perceive. ***
I also think that the learning part of exploratory testing is what matters when comparing two testers with similar resumes. Ask the candidates if they have had an exploratory approach in the reference projects, and ask them what they have learned.
I bet that you will spot those who have learned something and thereby improved their testing skills, including how to think about the context and its affection. I also believe that they have improved their problem solving skills and boosted their creativity along the way.
Do you have any learning experiences that you can share with us?
Have you experienced when you didn’t learn something? What did that do to you?
*** Note: Of course it depends on where you are on the Scripted vs. Exploratory continuum.
Good to see T.S Eliot quoted in a test blog 🙂
We have a heritage (or legacy) of automation and for years we have automated ‘everything’. This has given us huge benefits in terms of continuous integration and regression testing in the field, but comes at a cost. The draw backs are that we have done little exploratory testing (and have therefore missed a lot of defects) and scripting can be incredibly boring for testers.
We are in the process of limiting our automation – we’ve moved from ‘you must automate everything’ to a policy of ‘you have an obligation to create a regression capability’. This hopefully will give the time space and motivation to do more exploratory testing.
I am hoping this will provide a bit more motivation for our test teams – giving them the ‘autonomy, Mastery and purpose’ that Dan Pink talks about in his book ‘Drive’.
Take a look at Dan’s pitch on TED (http://www.ted.com/talks/dan_pink_on_motivation.html) and think of it in terms of motivating testers.
Cheers
Jon
Thanks Jon for sharing your experience!
Sounds like a good approach; and it is always nice to see when people are treated as intelligent people.
Hope that you will get a better result and a more motivated team!
Even if you just have started, how are people responding to your new approach so far?
Excellent talk from Dan Pink, I will spread this!
It’s true that people learn in different ways (I have about the same learning preference as you: if I am engaged in creating the abstractions, it sticks.)
But what about people that learn by reading or following examples, should they rather be doing scripted testing?
Also, if learning is important, how do you organize testing when a group of people have a joint responsibility. All testing everything, one expert per sub-part, training sessions??
@Rikard.
Regarding “people that learn by reading or following examples”:
It is fine if those people learn this way. But why would they be better suited for scripted testing?
I am not suggesting that; unless the primary objective is that people should learn the test scripts.
I say that “If tests are scripted, there is no need to reflect upon what you do. The script is there so you don’t have to think”. Which means that the intention of the script is that anyone could do it and shouldn’t need to think; e.g., a machine could do it. And machines don’t think very much (yet!).
I have been successful in running scripted tests, but I have been even more successful when I have explored using the scripts as a basis. In the former case, the success was that the tests were executed successfully. And in the latter case the success was not only that the tests were executed, also new tests derived from exploration were executed, I came out with more knowledge about the product and context, new future test ideas were identified, I did find more important bugs (the potato).
Regarding “organizing testing”:
It depends, but I guess that all of your examples might be appropriate in certain situations. I like to organize testing such as we should utilize a person’s expertise end let people grow their expertise, but on the same time increase and spread the competence of the whole group. But there are always other factors that play a role in a project, so sometimes Learning have to stand back for Quick Results. I think that Learning is a luxury that can be afforded and strive to include this as much as possible.
One thing for sure is that if I want someone to learn about a specific area, I would be foolish if I gave that person the task to execute some scripted tests for the area and hoping that the person should pick up something more than what the test scripts contain. But if I instead gave that person the opportunity to learn about an area, and even test scripts to be used as a basis for exploration, the person might learn very much.
Good elaborations, thanks.
Learning is not everything, but it is very important.
Scripted testing being better for some people is another track; a thought I haven’t thought through yet.