thoughts from the test eye

I want to open for a discussion on pair testing or at least widen the concept. By saying pair, we say two people, but why should we limit us to that? Depending what your objective is a different number might be more applicable?

Several of the referred authors in this article have elaborated around pair testing as something that could be wider and be used in different ways, many of the things I have expressed below has also been brought up by them. Still, I’ve added some parts and changed the scenery a bit.

Looking back at pair testing

“Pair testing is a way of approaching a test design process by having two people test the
same thing at the same time and place, continuously exchanging ideas. Even without any
special method or process, the dynamics of pairing enables the generation of more and
different ideas than either tester is likely to produce on his own. It’s an effective
complement to individual testing.” a quote from a draft article by James Bach and Sam Guckenheimer [1] on Pair testing.

This draft goes through the basic concept of pair testing. They discuss how it can be used in different forms and where it might be useful and in which situation.

“Two testers and (typically) one machine.
• Typically (as in XP)
– Pairs work together voluntarily. One person might pair with several others during a day.
– A given testing task is the responsibility of one person, who recruits one or more partners (one at a time) to help out.
• We’ve seen stable pairs who’ve worked together for years.
• One tester strokes the keys (but the keyboard may pass back and forth in a session) while the other suggests ideas or tests, pays attention and takes notes, listens, asks questions, grabs reference material, etc.” a quote from an article on Exploratory pair testing by Cem Kaner and James Bach [2].

Here James Bach and Cem Kaner elaborate around exploratory pair testing, how they suggest the pairing should be done, benefits of doing it and some drawbacks and risks with it. They also identify that it can be done in different combinations and that it is suitable in different situations.

“Working alone is working without a safety net” a quote by Brian Marick in an article [3] on Pair testing.

He summarized what seems to be a discussion in a workshop, where several parties were present. Several examples are listed where pair testing would be good, but he also identifies areas when it is not suitable. Furthermore he points out the importance of it not being more than two, thus naming it pair testing.

Jonathan Kohl takes on a different approach in an article about Pair testing [4]. He gives some examples where he has worked with developers in pair testing sessions with great result. He provides some hands-on tips on how to go about this and things to consider. He is focused on working in pairs, but he also adds the concept of using this technique as a way to teach testing and to grow the relationship between different parties in the organisation.

“As in pair-programming, two people simultaneously test an application at the same computer. It can involve: testing an application from a customer’s perspective and more technical kinds of tests”, a quote from a presentation [5] about Pair Testing by Jonathan Kohl.

In this presentation Jonathan elaborates deeper around the usage of the Pair testing technique. He touches a range of areas and brings up some other aspects than previous writers.

One thing that I notice from the various writers is that you use pair testing or as a technique to accomplish different objectives. Each writer includes different things to the concept of pair testing. Based on that, I saw that a pair testing session could consist of different dimensions. One of these dimensions was the amount of testers involved in a session, which in turn made me question the name of pair testing. Instead I call it group testing, since I think it can be two or more based on the objective. Beside considering that there can be different amount of testers I saw that there are other dimensions worth exploring.

Dimensions of group testing

There are many dimensions that could be investigated and experimented with. Here are the initial dimensions I’ve identified and experience with:

• Amount of participants – how many are part of the group testing session?
• Role as a tester – which role do you play as a tester? Is it explorer/tester, documenter/observer, checker, coach/trainer/teacher, reviewer, manager or something else?
• The scene – Are you running a specific user scenario or perhaps combining a set of user types? What scene have you painted that you are acting in? What personas are valid in this scene? What storytelling aspects have you considered when setting up this scene?
• Mission of group test – What is your objective[s]? What do you wish to have achieved when the group test session is finished? Do you wish to find information quickly, teach, coach or train a member of the group, review testing skill, report information quickly, help trouble shooting or something else?
• Note taking style – How are you taking notes? Do you take note in one single document that all of you write in simultaneously, is one of you taking notes only or are you all writing your own material?
• Partner combo – Which other stakeholders are part of this group test session? Do you pair up with developers, project managers, line managers, CEO, sales and marketing, system managers and business analysts … who?
• Lateral thinking style – Should part of the participants take on different lateral approaches such as thinking the opposite or wearing different thinking hats?
• Personality mix – Consider how you mix personalities such as introverts, extroverts, slow thinkers and quick ones. Which combinations work better than others in your context?
• Debriefing style – Do hold a debrief after each session or is it sufficient with the ongoing discussion you have?
• Accountability focus – Do you have one driver who is accountable or do you have many driving at the same time?
• Focus areas – Does each participant focus one specific characteristic (usability, security etc), a mix of focus areas between persons or possibly everyone focus on one characteristic?
• Test Environment setup – Do you sit behind one system or test environment, one system or test environment per person or possibly a mix of systems per person?
• Session duration – Do you do a longer session or short ones? Do change driver during or after each session, either way consider how this is setup  depending on the duration.
• …

Conclusion

There are many dimensions to the group testing technique that need to be experimented with. My point to this is that I fail to see the point in limiting the technique to a specific setup, but instead let its content/dimensions change depending on context. There are a few major pros that you probably get by using this technique, no matter which dimension you alter. Some of these are:

• an increase in creativity
• an increase in cooperation in the team/organisation between different roles
• an ongoing training of the participants in expressing themselves both in written and verbal form
• …

Based on my own experience, I think this is an excellent way of testing with great result. Therefore I intend to dig further in this area to see what comes up.

References

[1] Pair testing (aka Extreme Testing) by James Bach and Sam Guckenheimer –  http://www.exampler.com/testing-com/test-patterns/patterns/XT-Pattern-jb.pdf
[2] Exploratory testing in pairs by James Bach and Cem Kaner – http://www.kaner.com/pdfs/exptest.pdf
[3] Pair testing by Brian Marick – http://www.exampler.com/testing-com/test-patterns/patterns/pair-testing.pdf
[4] Pair testing – How I brought a developer into the testlab by Jonathan Kohl – http://www.kohl.ca/articles/pairtesting.html
[5] Presentation on Pair testing by Jonathan Kohl – http://www.kohl.ca/presentations/pair_testing.pdf