thoughts from the test eye

In a recent discussion with Henrik Andersson on Twitter regarding some consultancies being or claiming to be best at testing. Here is the initial conversation:

Henrik: Dear consultant companies why are you calling yourself consultant when all you talk about are recourses and invoiced hours. Shame on you!

Henrik: Many “consultant” companies claim to be “best at test”. You only train your testers in ISTQB and maybe a TMap book. That can’t be the best!

Henrik: All you companies that claim to be the best. I suggest you to have a show down to actually demonstrate if any of you are any good at all.

Martin: I take you on that challenge!

Henrik: are you one of those companies that claim to be “the best”? I have never heard you say that. But I’m always up for a game 🙂

Martin: No, to claim that is ignorant. But we are good at test related tasks. How do we measure ourselves then?

Henrik: you measure yourself by sharing your knowledge, demonstrate your skills, discussions with peers, trying new stuff, failing.

Martin: Yes, I know. But can you really compare?

Henrik: you can sort out those who sucks at it. but more important is not comparing but gaining respect and recognition from peers

Henrik: for me it is not about having one winner. it is about contributing to do my part in developing our community & learn from it

Martin: If you are good it is harder to distinguish who is the better as tester. I agree with you regarding the community etc.

Martin: I mean, would you compare which information was the most valuable between two who competed? I think it is hard.

Martin: Still, I see some testers who I think are really great while others are just good. What criteria do I have then?

There are many testers who have great renown, thus they talk a lot about testing. I’ve seen a handful of them show their skill. So they fit in with the tester Henrik mentions:

– by sharing your knowledge

– demonstrate your skills

– discussions with peers

– trying new stuff

– failing

A great tester might be an introvert that do not discuss with his/her peers or share his/her knowledge. Is it even required to demonstrate your skills? Are we talking a bout a renown tester here or an unknown tester who is just great in secret? Still, in order to be hired based on your reputation or renown you might need to fulfill the criteria that Henrik lists above. A great tester is only great in certain contexts, finding someone who is great in all contexts would be very rare. Being a generalist and general systems follower would mean that you could adapt to most situations, but would you be “great” in all of them? A specialist in one domain with great testing skills might be better than the generalist? Highly likely.

One criteria that I think is important is that a tester should be different from the next tester. This does not really apply to the tester that goes solo or prefers working alone. If we instead consider having a tester in a group that is really great at some aspect in testing. He/she is not great at everything and will not be able to handle every situation, but as a group they are well equipped to handle many, many contexts.

In my test team I have many great testers, as I see it:

– All have a high focus on what is valuable to the stakeholders.

– One loves everything that is complex and hard to understand, digging in to see what it is made of.

– One is totally unafraid, young and goes paths senior people would not go.

– One is extremely creative and comes up with tests that breaks everything in exciting new ways.

– One has an affinity for making everyone else feel better, thus making everyone work a bit better as a group.

– Some I’ve worked with a long time and probably knows things that I would miss,  thus covering behind me and I behind them.

– Some have the expertise in the current domain.

– Some have similar background and vocabulary with each other.

– Some are new to the group.

– Some are men and some are women.

– Some have excellent programmer skills.

– Some have experience from other roles in product development.

– Some are used to talking to the customer.

– Some are used to leading the group.

– Some are natural leaders.

– Some have worked together for nearly 15 years and still like it.

– Some have known each other for more than 30 years and are still best friends.

If you look at each one as an individual you would see one side, but if you look at the team and what they can acomplish you see greatness. Alone they would not be able to solve any situation in testing, but as a team they have a better chance.

This view is supported by the one that Cem Kaner writes about in his “Recruiting software testers” [1]. For a specific company he identifies an team that he would think he ideal. He lists it like this:

• Senior tester or test manager with experience in business operations or human resources. This person has worn the shoes of the customer for this system. For a vertical application, I think this is essential.
• Senior tester or test manager with strong test planning skills. If this is the test manager, she needs excellent mentoring skills, because she won’t have time to write the test documentation unless she is an individual contributor.
• Test automation hotshot, willing to serve as the group’s tool builder.
• Talented exploratory / intuitive tester, someone who is really good at finding bugs by playing with the product.
• Network administrator. This person has the dual role of helping the other testers set up and deal with the ever-changing configurations that they have to test under, and designing configuration tests to determine whether the product will run on most of the systems in use by the product’s customers.
• Attorney who is willing to wander through the various statutes and regulations looking for rules that the program must cover.

When you assemble a test team in product development, do you then consider the above traits and properties. So, back to the question that Henrik asked initially about consultancies stating they are the best in testing. I am sure they have good testers as individuals, but can they match a team that is handpicked to complement each other? I hardly think so.

When project managers need more resources for the project and especially testers. Do they then ask for “I want a tester” and “he/she must be ISTQB certified”. Being certified in testing is a story of itself, which I think is disturbing.

First, we really need to consider how they come to the conclusion that they need X number of testers. Was that the budget talking or was it someone who did the estimate on how many you “exactly” need? Ok, let’s assume it was budget to simplify the discussion somewhat. How do you then go about assembling the test group? Do you consider things things that I listed above or do you just want a tester, no interest in what they know and how they would fit in with the rest of the group. If they compliment each other is not a priority?

– What if you placed someone who is a (only english-speaking) great tester in another group of other testers who do not know english at all?

– What if you placed a collection of great testers, where each came from a competing consultancies?

– What if you only had introverts as testers in the group?

– What if you only had extroverts as testers in the group?

I often see the requirement in tester ads for domain knowledge rather than knowing anything about testing. I think domain knowledge is important, but you probably need at least one with it in the test group. There are so many other characteristics and backgrounds you want from people in the group.

So, my conclusion is that being a great tester is very context-dependent and you cannot say that you are the best.

References:

[1] Recruting software testers –  http://www.kaner.com/pdfs/JobsRev6.pdf