There are no testers that are the best Martin Jansson

In a recent discussion with Henrik Andersson on Twitter regarding some consultancies being or claiming to be best at testing. Here is the initial conversation:

Henrik: Dear consultant companies why are you calling yourself consultant when all you talk about are recourses and invoiced hours. Shame on you!

Henrik: Many “consultant” companies claim to be “best at test”. You only train your testers in ISTQB and maybe a TMap book. That can’t be the best!

Henrik: All you companies that claim to be the best. I suggest you to have a show down to actually demonstrate if any of you are any good at all.

Martin: I take you on that challenge!

Henrik: are you one of those companies that claim to be “the best”? I have never heard you say that. But I’m always up for a game :)

Martin: No, to claim that is ignorant. But we are good at test related tasks. How do we measure ourselves then?

Henrik: you measure yourself by sharing your knowledge, demonstrate your skills, discussions with peers, trying new stuff, failing.

Martin: Yes, I know. But can you really compare?

Henrik: you can sort out those who sucks at it. but more important is not comparing but gaining respect and recognition from peers

Henrik: for me it is not about having one winner. it is about contributing to do my part in developing our community & learn from it

Martin: If you are good it is harder to distinguish who is the better as tester. I agree with you regarding the community etc.

Martin: I mean, would you compare which information was the most valuable between two who competed? I think it is hard.

Martin: Still, I see some testers who I think are really great while others are just good. What criteria do I have then?

There are many testers who have great renown, thus they talk a lot about testing. I’ve seen a handful of them show their skill. So they fit in with the tester Henrik mentions:

- by sharing your knowledge

- demonstrate your skills

- discussions with peers

- trying new stuff

- failing

A great tester might be an introvert that do not discuss with his/her peers or share his/her knowledge. Is it even required to demonstrate your skills? Are we talking a bout a renown tester here or an unknown tester who is just great in secret? Still, in order to be hired based on your reputation or renown you might need to fulfill the criteria that Henrik lists above. A great tester is only great in certain contexts, finding someone who is great in all contexts would be very rare. Being a generalist and general systems follower would mean that you could adapt to most situations, but would you be “great” in all of them? A specialist in one domain with great testing skills might be better than the generalist? Highly likely.

One criteria that I think is important is that a tester should be different from the next tester. This does not really apply to the tester that goes solo or prefers working alone. If we instead consider having a tester in a group that is really great at some aspect in testing. He/she is not great at everything and will not be able to handle every situation, but as a group they are well equipped to handle many, many contexts.

In my test team I have many great testers, as I see it:

- All have a high focus on what is valuable to the stakeholders.

- One loves everything that is complex and hard to understand, digging in to see what it is made of.

- One is totally unafraid, young and goes paths senior people would not go.

- One is extremely creative and comes up with tests that breaks everything in exciting new ways.

- One has an affinity for making everyone else feel better, thus making everyone work a bit better as a group.

- Some I’ve worked with a long time and probably knows things that I would miss,  thus covering behind me and I behind them.

- Some have the expertise in the current domain.

- Some have similar background and vocabulary with each other.

- Some are new to the group.

- Some are men and some are women.

- Some have excellent programmer skills.

- Some have experience from other roles in product development.

- Some are used to talking to the customer.

- Some are used to leading the group.

- Some are natural leaders.

- Some have worked together for nearly 15 years and still like it.

- Some have known each other for more than 30 years and are still best friends.

If you look at each one as an individual you would see one side, but if you look at the team and what they can acomplish you see greatness. Alone they would not be able to solve any situation in testing, but as a team they have a better chance.

This view is supported by the one that Cem Kaner writes about in his “Recruiting software testers” [1]. For a specific company he identifies an team that he would think he ideal. He lists it like this:

  • Senior tester or test manager with experience in business operations or human resources. This person has worn the shoes of the customer for this system. For a vertical application, I think this is essential.
  • Senior tester or test manager with strong test planning skills. If this is the test manager, she needs excellent mentoring skills, because she won’t have time to write the test documentation unless she is an individual contributor.
  • Test automation hotshot, willing to serve as the group’s tool builder.
  • Talented exploratory / intuitive tester, someone who is really good at finding bugs by playing with the product.
  • Network administrator. This person has the dual role of helping the other testers set up and deal with the ever-changing configurations that they have to test under, and designing configuration tests to determine whether the product will run on most of the systems in use by the product’s customers.
  • Attorney who is willing to wander through the various statutes and regulations looking for rules that the program must cover.

When you assemble a test team in product development, do you then consider the above traits and properties. So, back to the question that Henrik asked initially about consultancies stating they are the best in testing. I am sure they have good testers as individuals, but can they match a team that is handpicked to complement each other? I hardly think so.

When project managers need more resources for the project and especially testers. Do they then ask for “I want a tester” and “he/she must be ISTQB certified”. Being certified in testing is a story of itself, which I think is disturbing.

First, we really need to consider how they come to the conclusion that they need X number of testers. Was that the budget talking or was it someone who did the estimate on how many you “exactly” need? Ok, let’s assume it was budget to simplify the discussion somewhat. How do you then go about assembling the test group? Do you consider things things that I listed above or do you just want a tester, no interest in what they know and how they would fit in with the rest of the group. If they compliment each other is not a priority?

- What if you placed someone who is a (only english-speaking) great tester in another group of other testers who do not know english at all?

- What if you placed a collection of great testers, where each came from a competing consultancies?

- What if you only had introverts as testers in the group?

- What if you only had extroverts as testers in the group?

I often see the requirement in tester ads for domain knowledge rather than knowing anything about testing. I think domain knowledge is important, but you probably need at least one with it in the test group. There are so many other characteristics and backgrounds you want from people in the group.

So, my conclusion is that being a great tester is very context-dependent and you cannot say that you are the best.

References:

[1] Recruting software testers –  http://www.kaner.com/pdfs/JobsRev6.pdf

7 Comments
Simon Morley February 26th, 2011

Martin,

I think you’re touching on group dynamics that applies to more than just test groups. But it’s an interesting problem of how good test teams work.

A much trickier problem might be how to integrate testers that have a strong “waterfall” background with others that have a more “agile” background… They might be good/great in a certain context – what happens when you change the context? Mmm, maybe I need to jot down some thoughts on this….

I was amused by the idea of a “great tester in secret” – how would you know? Something like the Heisenburg uncertainty principle at work there…

Leo Perrotta February 26th, 2011

Hi Martin,
very interesting points!
My view is that diversity joined to the agile methodologies can improve the ‘power’ of workgroup testing and ‘fix’ the project management issues that affect the traditional wow

Andrei February 27th, 2011

The value of a tester is given by his manager and the team. You cannot be an outstanding tester if you”re just passionate about testing.
You need to have the sparkle in your eyes which keeps you alive, which drives you through every professional challenge.
I can only think at an orchestra and its conductor. You can be the best violinist in the world if your team does not perform well, or better, the conductor cannot light that sparkle

Martin Jansson February 28th, 2011

@Simon,
Yes, I agree that this applies to more than just the test team. Still, with the common view on testers that “any resource in the organisation can be a tester” you seldome take group dynamics into consideration. If you think that each tester just runs a script.

This also applies to the point you make about having testers from waterfall work with agile testers. I think agile testers learn to be flexible towards the organisation, the project and to the test team. Testers with strong “waterfall” background will have it more painful by being more rigid. I have transitioned to being a flexible tester and I see many squirm when they change.

@Leo,
Yes, I hope so as well.

@Andrei,
I believe that each tester should have more freedom so that there is a little orchestra with a conductor in each of them. Testers should be intelligent enough to choose their own paths. Still, a masterful test lead and test manager will naturally affect things also. Passion in one tester can be enough to fuel a whole team and its manager, as I see it.

Jesper L Ottosen March 1st, 2011

All kinds soft skills come into play in software testing for me to single out anyone. Some situations you have to be flexible, others stern. Some situations require results orientation and some situations require attention to all details. I can see the skills of all personality in play in software testing – as software testing is a skill of many skills.
from http://www.eurostarconferences.com/blog-posts/2010/9/21/software-testing-is-a-skill-of-many-skills—jesper-ottosen.aspx

Rikard Edgren March 2nd, 2011

I agree that no testers can be the best, because it is the wrong question.
Software development is a team effort, it’s about making great software. It is not a competition for individuals.

Martin Jansson March 3rd, 2011

@Jesper,
I think Software testing is a group of skills, not a skill of many as your title says. Still, I agree with a lot of what you say in your blog post and Andreas made a lot of good points.

@Rikard,
The whole topic started with consultancies stating they were the best. So it was more a challenge than an actual question. This was the response to that challenge. But as you can see with my answer it is a team effort in many contexts, but not all.

Also, in some organisations there is a competition and that making great software is not what it is all about. Sadly, for many employees at many companies the main drive is to get the paycheck at the end of the month.