Another certification, another scam? Martin Jansson 17 Comments

In a recent blog post [1] on Informator-blog, Magnus C Ohlson articulates the idea of pilots having the flight hours but not the actual flight certificate. He insinuates that the artifacts from requirements and testing would be better if people were certified, if I understand him correctly. Furthermore he explains that testers need education, knowledge and experience which I agree with fully. But he promotes that to ensure that someone has the right competence is by having a certificate such as ISTQB or REQB. According to him, you would then know if he or she is a skilled tester.

Here are a few personal experiences that relate to certification, as I see it:
When I did the military I got the opportunity to train for driving a trailer. We did a two-day theoretical exam, and then they let us out on the roads. After studying text books succeeding with a little exam they determined we were ready to start driving. I had no problem passing the exam, but I was a terrible driver. I had only a few months prior to this been able to take the driving license for driving a car. They let out a large amount of trucks, trailers and other vehicles into the small town of Boden. I managed to get around town without getting killed, but all in all the lot of us managed to demolish several traffic lights, some road signs, some trucks and a few light poles. The cost of this adventure was huge. I guess we learned a lot, but the idea that we were ready based on reading a textbook was a bit strange. Months later, after a few hundred miles of driving we started to get good at it, but the 2-3 day theory in the beginning was almost meaningless then. I see the test and requirement certification as having the same symptom, we do not get much value of something so intensive and theory focused.

In the early 2000 a doctor told me I had diaphragmatic hernia. At the same place, a surgeon went through with me the procedure how to get this fixed. She told me that I need to change the way I lived my life, but that it was almost pointless to try according to her. If I were to go through with a surgery, they would have needed to open me up, lift my chest and move around several organs. The whole procedure was very dangerous and the risk for death was high. As part of the surgeon’s analysis, she performed a gastroscopy. This procedure took between 10 and 20 minutes. It was hard for me to determine, after a while I understood what torture was all about. Some years later I met a doctor who hinted that there were other experts in the field that could help me out. I ended up with probably one of the best doctors in the field, who told me a whole different story. He recognized the name of the original surgeon who had given me the first statement. According to him, the original surgeon was not able to perform a gastroscopy in school as well. This new doctor performed a gastroscopy as well, which took 1½ to 2 minutes which is the time it should take, according to him. He then said that an operation was possible with minimum risk to my life. The last 10-20 years they had been using keyhole surgery, the methods the previous surgeon talked about are not used any more. Even if the first surgeon got her license, she was apparently not interested in keeping herself up-to-date with the changes in her craft. Comparing this with testing and requirement certificates, even if you are certified does not mean that you keep up with changes to the craft and that it is applicable.

A friend of mine was troubled about being forced to become certified. He was basically forced to certify himself as a project manager. The curriculum of the certification was solely based on waterfall methods. My friend has been working with agile projects for a while and has left the world of waterfall behind him. He does not see the point in keeping old facts up to date but instead try to learn new things. The person who wrote the curriculum for the certification did not know anything about agile. Comparing this with testing and requirement certification, we will have single point or points of failure in keeping up to speed with changes and improvements in the craft. If the creators of a syllabus are not top notch in the craft, then everyone who needs to certify themselves must lower themselves to the creators level. Even if they are top notch, they cannot compete with the wisdom of crowds, where the crowd is the joint knowledge of the test community.

Another friend of mine took an intensive course to drive a car. She travelled to a small town in northern Sweden to take a two-week course. She took it easily and got back home to Gothenburg. But the driving in the small town was a bit too easy with no highways, very few cars and an environment that did not match what was in her home town. She got the license and got home. When entering the traffic in Gothenburg she became too scared and dared not drive anymore. If we compare this with testing and requirement certification, an intensive course that results in a certificate might only mean that you have paid money to get something that is not valid in your actual context, in your project or hometown.

In a previous article called Testers Greatest Nemesis [2], I wrote about the intent that Dorothy Graham had when they initiated ISTQB. According to her blog posts the original intent was lost over the years. In Sweden there is a movement that has started certification of requirement experts called REQB. I see consultancies are offering the training and certification of it. I do hope this is just not a scam to make money, where recruiters will start filtering out those with 10+ years of experience of requirement handling to those with the name REQB in their CV.


I do not believe the complexity of organizations, projects and business is displayed in a generic multiple choice questionnaire that someone with no knowledge could accidently pass. I hope the movement in REQB learns from previous mistakes that have been seen by introduction of ISTQB. Dorothy and several who comments on her blog has identified a few things to consider.

A certificate tells one story about a person, but it is very fragile. I prefer talking to references, looking at their renown, public appearance, blogs, articles and papers would perhaps give a more vivid story.


[1] Varför certifiera sig inom krav och test –
[2] Testers Greatest Nemesis –

37 Sources now in Swedish the test eye 4 Comments

We have now finalized the Swedish translation of 37 Sources for Test Ideas.

Neither the translation nor the original is perfect, but we think they can be useful to many.

If you are interested in translating 37 Sources, or Software Quality Characteristics (Swedish version), let us know.

Henrik, Martin & Rikard

Regarding comments the test eye No Comments

Due to all spam comments not being stopped by our automatic checkers, we are unfortunately forced to moderate all comments.

We will do this as quick as possible, but there might be some delay before your comment appears.

Hope that you understand this, from our point of view, drastic decision.

Henrik, Martin & Rikard

I am an Exploratory Tester Rikard Edgren 5 Comments

I am curious about how the system works

I look at details and the whole, and at many places

I use many sources to learn what is important

I am creative and see many testing possibilities

I test in many different ways, and adjust to the situation

I am good at finding important problems fast

I am aware of models enabling and limiting my testing

I can evaluate a system from many dimensions

I can communicate my thinking and findings

I am an exploratory tester

Complete Testing Risk Reduction with Bohr-Steinlager-Stumpf Quantization Rikard Edgren 1 Comment

Suppose you have a risk assessment fully agreed by stakeholders, and their relatives.
Create a stable sampling across all adjacent perspectives, and let the corresponding tests be executed on best representatives platforms, theoretically covering 99.5% of potential future usage.
Now, let any opposites conglomerate, and hold their best fit tempo, until the sounds dissolve into the night.
The information is tunneled into various status report templates according to confidently standardized information objectives.

No need for the Black Viper test technique anymore!

Intertwined SFDPOT & CRUCSPIC STMP Rikard Edgren 3 Comments

I hope many of you are using SFDPOT (James Bach) and CRUCSPIC STMP ( in order to investigate what to test. SFDPOT describes elements of the product, and CRUCSPIC STMP describes sought attributes of the system. They are very powerful ways to identify things to test, plus to be able to communicate it effectively.

Both are very rich, and could incorporate each other, e.g. SFDPOT could be a part of Capabilities, and CRUCSPIC STMP could be a part of Operations (often inverted as a risk) There are more connections, as seen here:


I still recommend using both as separate activities, they stimulate thinking in different dimensions (and that’s why testers are needed, right?) That they blend is not a problem, it is the thinking that matters, so just put the stuff where it makes most sense.

If this makes almost no sense to you, interpret as a hint to try both!

Lightweight Charisma Testing Rikard Edgren No Comments

One heavyweight way of testing charisma is to use dozens of potential users on dozens of alternative product solutions/prototypes.
For lightweight charisma testing, it is often fast and fruitful with an awareness of charisma violations.
This method requires an understanding of the unique charisma for your product.

Testers probably won’t be in charge of developing an explicit charisma model, but you can take part, e.g. by recommending the following process:

1. Write down what you believe are the special things about the software (gamemakers might use “playability”, but please be more specific)
2. Get inspired by thetesteye’s generic list of Charisma characteristics (provided below)
3. Document the important characteristics with as much details as possible (to make’em truly mean things in your context.)
4. Share, review, refine and gain approval for your Charisma Guidelines across the teams

When you have this list, it is fun and attention-inspiring to keep these things in the back of your head whatever type of testing you are performing.
Just remember to report appropriately, probably verbal.


Charisma: Does the product have “it”?

– Uniqueness: the product is distinguishable and has something no one else has.
– Satisfaction: how do you feel after using the product?
– Professionalism: does the product have the appropriate flair of professionalism and feel fit for purpose?
– Attractiveness: are all types of aspects of the product appealing to eyes and other senses?
– Curiosity: will users get interested and try out what they can do with the product?
– Entrancement: do users get hooked, have fun, in a flow, and fully engaged when using the product?
– Hype: should the product use the latest and greatest technologies/ideas?
– Expectancy: the product exceeds expectations and meets the needs you didn’t know you had.
– Attitude: do the product and its information have the right attitude and speak to you with the right language and style?
– Directness: are (first) impressions impressive?
Story: are there compelling stories about the product’s inception, construction or usage?

Initial thoughts on group testing Martin Jansson 3 Comments

I want to open for a discussion on pair testing or at least widen the concept. By saying pair, we say two people, but why should we limit us to that? Depending what your objective is a different number might be more applicable?

Several of the referred authors in this article have elaborated around pair testing as something that could be wider and be used in different ways, many of the things I have expressed below has also been brought up by them. Still, I’ve added some parts and changed the scenery a bit.

Looking back at pair testing

“Pair testing is a way of approaching a test design process by having two people test the
same thing at the same time and place, continuously exchanging ideas. Even without any
special method or process, the dynamics of pairing enables the generation of more and
different ideas than either tester is likely to produce on his own. It’s an effective
complement to individual testing.” a quote from a draft article by James Bach and Sam Guckenheimer [1] on Pair testing.

This draft goes through the basic concept of pair testing. They discuss how it can be used in different forms and where it might be useful and in which situation.

“Two testers and (typically) one machine.
• Typically (as in XP)
– Pairs work together voluntarily. One person might pair with several others during a day.
– A given testing task is the responsibility of one person, who recruits one or more partners (one at a time) to help out.
• We’ve seen stable pairs who’ve worked together for years.
• One tester strokes the keys (but the keyboard may pass back and forth in a session) while the other suggests ideas or tests, pays attention and takes notes, listens, asks questions, grabs reference material, etc.” a quote from an article on Exploratory pair testing by Cem Kaner and James Bach [2].

Here James Bach and Cem Kaner elaborate around exploratory pair testing, how they suggest the pairing should be done, benefits of doing it and some drawbacks and risks with it. They also identify that it can be done in different combinations and that it is suitable in different situations.

“Working alone is working without a safety net” a quote by Brian Marick in an article [3] on Pair testing.

He summarized what seems to be a discussion in a workshop, where several parties were present. Several examples are listed where pair testing would be good, but he also identifies areas when it is not suitable. Furthermore he points out the importance of it not being more than two, thus naming it pair testing.

Jonathan Kohl takes on a different approach in an article about Pair testing [4]. He gives some examples where he has worked with developers in pair testing sessions with great result. He provides some hands-on tips on how to go about this and things to consider. He is focused on working in pairs, but he also adds the concept of using this technique as a way to teach testing and to grow the relationship between different parties in the organisation.

“As in pair-programming, two people simultaneously test an application at the same computer. It can involve: testing an application from a customer’s perspective and more technical kinds of tests”, a quote from a presentation [5] about Pair Testing by Jonathan Kohl.

In this presentation Jonathan elaborates deeper around the usage of the Pair testing technique. He touches a range of areas and brings up some other aspects than previous writers.

One thing that I notice from the various writers is that you use pair testing or as a technique to accomplish different objectives. Each writer includes different things to the concept of pair testing. Based on that, I saw that a pair testing session could consist of different dimensions. One of these dimensions was the amount of testers involved in a session, which in turn made me question the name of pair testing. Instead I call it group testing, since I think it can be two or more based on the objective. Beside considering that there can be different amount of testers I saw that there are other dimensions worth exploring.

Dimensions of group testing

There are many dimensions that could be investigated and experimented with. Here are the initial dimensions I’ve identified and experience with:

  • Amount of participants – how many are part of the group testing session?
  • Role as a tester – which role do you play as a tester? Is it explorer/tester, documenter/observer, checker, coach/trainer/teacher, reviewer, manager or something else?
  • The scene – Are you running a specific user scenario or perhaps combining a set of user types? What scene have you painted that you are acting in? What personas are valid in this scene? What storytelling aspects have you considered when setting up this scene?
  • Mission of group test – What is your objective[s]? What do you wish to have achieved when the group test session is finished? Do you wish to find information quickly, teach, coach or train a member of the group, review testing skill, report information quickly, help trouble shooting or something else?
  • Note taking style – How are you taking notes? Do you take note in one single document that all of you write in simultaneously, is one of you taking notes only or are you all writing your own material?
  • Partner combo – Which other stakeholders are part of this group test session? Do you pair up with developers, project managers, line managers, CEO, sales and marketing, system managers and business analysts … who?
  • Lateral thinking style – Should part of the participants take on different lateral approaches such as thinking the opposite or wearing different thinking hats?
  • Personality mix – Consider how you mix personalities such as introverts, extroverts, slow thinkers and quick ones. Which combinations work better than others in your context?
  • Debriefing style – Do hold a debrief after each session or is it sufficient with the ongoing discussion you have?
  • Accountability focus – Do you have one driver who is accountable or do you have many driving at the same time?
  • Focus areas – Does each participant focus one specific characteristic (usability, security etc), a mix of focus areas between persons or possibly everyone focus on one characteristic?
  • Test Environment setup – Do you sit behind one system or test environment, one system or test environment per person or possibly a mix of systems per person?
  • Session duration – Do you do a longer session or short ones? Do change driver during or after each session, either way consider how this is setup  depending on the duration.


There are many dimensions to the group testing technique that need to be experimented with. My point to this is that I fail to see the point in limiting the technique to a specific setup, but instead let its content/dimensions change depending on context. There are a few major pros that you probably get by using this technique, no matter which dimension you alter. Some of these are:

  • an increase in creativity
  • an increase in cooperation in the team/organisation between different roles
  • an ongoing training of the participants in expressing themselves both in written and verbal form

Based on my own experience, I think this is an excellent way of testing with great result. Therefore I intend to dig further in this area to see what comes up.


[1] Pair testing (aka Extreme Testing) by James Bach and Sam Guckenheimer –
[2] Exploratory testing in pairs by James Bach and Cem Kaner –
[3] Pair testing by Brian Marick –
[4] Pair testing – How I brought a developer into the testlab by Jonathan Kohl –
[5] Presentation on Pair testing by Jonathan Kohl –

Scenario Testing, Karlstad 2012 Rikard Edgren No Comments

I have been doing and teaching quite a lot of scenario testing lately.
I have been surprised by the ease and speed, and ability to find important problems, (also giving an embryo to a compelling bug report.)
Maybe it can be useful for you as well, probably as a complementary technique, or as a powerful activity for mature products.

My basis comes from Cem Kaner’s writings, Introduction to Scenario Testing, but mostly BBST Test Design.
The latter is Creative Commons, so I can quote the basics:

“A scenario is a coherent story about how someone uses (or tries to use) the program.
A scenario test uses a scenario as a tool for evaluating a program’s behavior
The elements of the story (adapted from Carroll, 1999)

  • Setting
  • Agents or actors
  • Goals or objectives
  • Motivations and emotions
  • Plot (sequences of actions and events)
  • Actions & events can change the goals”

(slide 301 in BBST, Test Design)

When documenting, I also add a purpose of the scenario test, so it is clear why we’re doing it.

Kaner lists five key characteristics for an ideal scenario test: story, credible, motivating, complex, easy to evaluate:

  • “The test is based on a coherent story about how the program is used, including goals and emotions of people.
  • The story is credible. Stakeholders will believe that something like it probably will happen.
  • The story is motivating. A stakeholder with influence will advocate for fixing a program that failed this test.
  • The story involves complexity: a complex use of the program or a complex environment or a complex set of data.
  • Test results are easy to evaluate. This is important for scenarios because they are complex.”

(slide 302 in BBST, Test Design)

I am not rigid about Easy to evaluate, rather the contrary; I want rich scenario tests and trust our ability to find out the details when encountering problems.

To find out what to base the scenario around, Kaner suggests 17 ways.
These can help, but you get even further by understanding what’s important. To speed that understanding, 37 Sources for Test Ideas might help.
Build the scenario around something important, add complexity (sequences, data, environment) and a credible goal, often spanning outside your product.

I guess you’d like an example, but I won’t give you that, because too many of you will use it as a template and don’t make the effort to find out what matters in your context.
But make sure to not try to cover as much as possible, and don’t be afraid to add corny details (they add life.)

Try it and see if the technique can help you!

My Very First Testing Lesson Rikard Edgren 4 Comments

As everyone else, I fell into testing by chance. I wanted to work as a developer, and saw testing as a stepping stone (this quickly changed, though.)
My first day I tested a Service Pack of a big, localized product. An experienced tester guided me at the start, and I can still remember the conversation.

Rikard: So is this machine the right one to use for this test?
Anders: Yes, you can start testing. Tell me what you are thinking, so it’s easier for me to help you.
Rikard: I double-click the installer and a welcome screen appear. I click Next.
Anders: Wait! Did you read the text?
Rikard: No.
Anders: Well, read the text and tell me what you think about it.
Rikard: OK. I think the welcome screen looks good, it tells me what it’s about to do, and the Swedish is correct.
Anders: Good. You know, some people actually reads this kind of information…
Rikard: Now I follow the script and select a custom install and click Next. I select all components as the test case tells me and click Next, and now it’s time to actually perform the installation.
Anders: Stop! Why do you just follow the script and click Next all the time?
Rikard: Because it tells me to.
Anders: But do you think all users will follow a script and know what to do all the time?
Rikard: I suppose not all of them.
Anders: So then you need to do some variations, click Back and change your mind, act like a normally confused end user.
Rikard: But why isn’t that included in the test steps?
Anders: You can’t specify everything, you can’t just do only what’s required; you are a tester, and supposed to find most of the important bugs.

I did variations, I soon found that the installer would forget the selected components after Back/Next, and that it was easy to not be aware of this.
The bug was reported and fixed, not a big deal perhaps, but a very important lesson for me:

Regardless of what they say, they probably want you to find issues that will annoy end users.