Testers greatest nemesis Martin Jansson

Background

When I first got in contact with software testers, I worked as PM and developer for a language tool. Our CEO had said that he had hired two testers, easily since you can just pick them from any street corner. Sadly they had no clue what to do and did not find any bugs, they just found out how the OS worked or things that were built-in. After some time we were able to get a new group of testers and now things really changed. Some of them were aspiring to be developers, but settled to be testers for a short time. At that time they had no knowledge about how testing should be done according to the so called rules, but they did a good job and found bugs in our software.

Some year later I began at a product development company. During my years there we had change of manager almost every year for the test department. Each one brought their own perspective on testers. Most of them accepted any personnel from any department when there was lack of testers. During that time we got to experience a lot of different backgrounds, skills and interests from the extra personnel. We also experienced many employees who were moved or even demoted “down” to the test department. Many stayed in testing where they excelled and eventually liked it. During all those years management saw us as the complaining guys from the test department, perhaps a too common view? What we really did was express risks, bugs or any information we thought endangered the company or products under test. I am sure their perception of us was misplaced, but naturally we were somewhat to blame for how we communicated and how we acted when communicating.

Some years later I joined a smaller company with mostly researchers and scientists. Most of them were used to working alone in development projects so they did all things themselves. They did not see the need for testing as a discipline on its own. Eventually when we (testers) got something to test we showed them it was a big difference with what we found compared to them.

The thing that is constant is the confusing perception on a tester is and what we should do.

How are testers perceived?

If you look at testers from a salary perspective we very often have lower salaries than developers and project managers, but we have higher than documentation specialists and support personnel (at least in Sweden). For many salary also drives your career choices, so you naturally want to get out of the testing department. In Sweden consultants can charge higher for test leads than testers at many major customers. This does not motivate consultancies to grow great testers.

If you look at testers through career perspective you often see that tester is a pit stop in pursuit to become a developer. Or perhaps more rarely you see people have been demoted from other positions. Someone needs to take the role of tester, let’s take the person we need the least for other tasks. I also see personnel that are promoted from support to testing (as they express it). If you become test lead you might be on the way to become project manager. Managers know that many with higher ambition will just pass through the test department, while others less motivated will stay behind. Still, there will always be a group of testers who love testing and want to excel in it, but some companies do not have them yet.

In the scripted test approach you most often want a domain expert to write test cases and let someone else (or sometimes the same person) execute the tests. In this situation the tester can be “anybody”, he/she just need to execute the tests. When a manager is seeking new resources to become testers he will accept anybody to become a tester, than you have the potential of getting anyone, even demoted personnel, from other parts of the organisation. This is the most common view on testers, as I see it.

Certification

During my whole career I have not heard that many talk about the need or requirement of certification at places where I worked or at clients. In one case a tester approach me, when he was about to enter my test group. He said he was ISTQB certified and that his employer required all testers to be certified. I told I was not, but I had more than 10 years of test experience and close to 20 years of product development experience. Was that ok? I asked him of his testing skills and what he could do to contribute to my team. He got scared and did not want to join the team. I regret that I scared him off like that. Someone must have introduced the idea that to be a good tester you need to be certified. Or was it perhaps set up as a minimum requirement when handling allocating personnel to teams? Perhaps the original intention was certified tester or experiences enough to cover it? There is seldom context behind decisions like that. My belief is that some consultancy got them to buy-in on the idea, then sold them lots of courses and certification packages.

After reading Dorothy Grahams blog posts ([1], [2] and [3]) about the intention of certification, I wonder why no one spoke up about where things were heading. Their intent might have been to make the perception on testers better, but I think it instead has hurt our craft. At each conference and at most meetings there is often someone who speaks up with lots of argument against certification. I rarely see anyone take up the discussion to meet their arguments or perhaps I do not listen? James Bach has made a lot of good arguments [4].

There are many so called test experts out there who say that certification such as ISEB or ISTQB is needed to be a tester. Some companies even require it of their testers and therefore the recruiters require people seeking jobs to have it. I think it is all a charade. Having testers who take courses in testing, who read books, blogs and articles, who want to learn and who want to excel as testers are what is needed. Passionate testers who want to become great! If they are certified that is ok, perhaps they got some ideas from it and they might have had a great teacher who stimulated them into becoming passionate themselves.

ISTQB uses multiple-choice questions on their exams, but they are quite limited. Cem Kaner has written an excellent post about Writing Multiple Choice Test Questions [5] where he makes some strong arguments. If ISTQB was altered along those lines it would make it harder to pass and naturally harder to create, but it would still not solve the main issue with content being out of date and totally wrong in many areas, as I see it. Jonathan Rees brings up other strong arguments about multiple-choice questions in his article “Frederick Taylor In The Classroom: Standardized Testing And Scientific Management” [6].

Attitude

Just because we have to work up streams does not mean we can keep on having a lousy attitude. I’ve often seen us picture ourselves as victims because of our situation, lack of personnel, time etc. If we are too few to test and if we got too little time, we can only offer to do our best. We can also explain what we could do if we were more and if we had more time. The prior combined with that we often speak in anger when we talk about quality. This only fuels the perception that we are a bunch of idiots, angry ones.

When we get deliverables from developers we are sometimes angry because of the bad quality or the lousy state of a certain build. Do we consider why it is like that, what shortcuts they needed to take or if someone forced the delivery of a new build? Do we really need to focus our blame on the developers? Consider their ever increasing technical debt that they might not get proper priority to adjust.

In most areas of expertise you have lots of education, at various levels of the school system, to back you up. This has just started to get going with testing. At least it is not only a chapter in a book that you skip. There are lots of books, articles, blogs and other sources of information to gain other peoples experience on testing. Why is it ok to think you do not need to learn more about your craft? Why do so many testers with lots of years in the testing craft still state that they have not studied anything to get better at testing? Having that attitude damages the perception on testers by keeping you ignorant of what you claim to be expert at. With the increasing use of agile teams where a tester has a natural part, you are supposed to know at least something about your craft.

What do we do to affect that perception?

If we are continuously providing valuable information to our stakeholders the perception will be altered. This means that you need to know what they find valuable and what could threathen that value. You also need to consider how you communicate, thus in what form, if you are going to use metrics or not, how much subjectivity or objectivity you should use and how you act when communicating. Less drama-queen and more professionalism.

We are working up streams here, so everything that you do that is bad will have a great impact on the perception on testers. Where ever you go you will bring your attitude and ambition. When interacting with non-testers consider what you are saying and how it might appear to them. Consider if you are in the correct crowd to utter your disapproval, if you need to go somewhere else or if you can just go to your manager.

We need to communicate to managers that it is demeaning and de-motivating to be seen as idiots or just anybody. We need to show that having skilled, passionate and motivated testers will give a lot better result. What else can you do to motivate yourselves to get those attributes? Those who have been demoted or are de-motivated, show them how creative and exciting the testing profession can be. Bring in other external passionate testers to give them some new ideas. If nothing of this work, perhaps they need to find what they really want to do and go there.

Before accepting new testers to the team, we need to make sure they are right for the job. Do not accept demoted personnel without explain the consequences. When you as test lead discuss having extra personnel join your team, clarify that you want to test them before accepting them into the group and that some in the team need to be able to veto acceptance.

We need to tell developers that we understand that they must take shortcuts, thus increasing the technical debt, but we can help [7]. Work closer with the developers. Stop building walls between you. The more the developers trust and respect you, the more information you will have before you commence your work as a tester which will lead to a better work done. Remember a good bug is a fixed bug.

Consider how the test organisation is built, how it markets itself and what you communicate to management. See Scott Barbers excellent blog about “What being a Context-Driven Tester means to me” [8] that can be used as a starting point for you and your test organisation. Also consider where you are going with testing [9] to understand where you come from, what your next goal is and perhaps what is pushing you in a certain direction. Are you going in the right direction?

Conclusion

I think the perception on testers is our greatest nemesis, we have to fight it every day.  Certification in testing does not help us, as I see it, but it is not our main target for concern just one of the bullies. There are many things that make us get a bad reputation and are therefore perceived badly. Start changing your own ways and affect those around you to become great, passionate testers who deliver valuable information effectively.

References

[1] Certification is evil? - http://dorothygraham.blogspot.com/2011/02/part-1-certification-is-evil.html

[2] A bit of history about ISTQB certification - http://dorothygraham.blogspot.com/2011/02/part-2-bit-of-history-about-istqb.html

[3] Certification does not assess tester skill - http://dorothygraham.blogspot.com/2011/02/part-3-certification-schemes-do-not.html

[4] Search for ISTQB at James blog - http://www.satisfice.com/blog/index.php?s=istqb or http://www.satisfice.com/blog/index.php?s=certification

[5] Writing Multiple Choice Test Questions - http://kaner.com/?p=34

[6] Frederick Taylor In The Classroom: Standardized Testing And Scientific Management - http://radicalpedagogy.icaap.org/content/issue3_2/rees.html

[7] Developers, let the testers assist with the technical debt - http://thetesteye.com/blog/2011/01/developers-let-the-testers-assist-with-the-technical-debt/

[8] What being a Context-Driven Tester means to me - http://www.testingreflections.com/node/view/8657

[9] Where are you going with testing - http://thetesteye.com/blog/2010/04/where-are-you-going-with-testing/

19 Comments
Jagannath Tammeleht May 2nd, 2011

Awesome;

Several things I thought about and eagerly discussed during my short but intense career in this post.

It is true that testers may have paid less than, for example programmers. But I think the testers have a good salary, or at least those I know, but the salary is relative. The salary of an industry can always be raised and I am sure it will do. But my opinion is that the salary is individual and it’s your responsibility, you want higher pay, you may see to get a higher salary.

With regard to tester see themselves as victims, I can not but agree with you.

Certifications, ojoj There are many different opinions. Of course people will create certifications which there is an opportunity to earn some money, fun for them.

Certifications are not intended as a final test of your knowledge. Your knowledge and expertise is something that is constantly evolving, it is important to understand just that.

“True knowledge exist only in knowing That you know nothing” Socrates

Good posts and there is much to discuss in this post, me like

Rikard Edgren May 3rd, 2011

I think you are spot on with perception of testing as our nemesis.
And I agree that more skilled testers providing value will make a change.
But it will take a lot of time, perhaps 25 years or more.

To speed up I think we could benefit from success stories; public, well-known examples of really good testing that made the critical difference.

Martin Jansson May 3rd, 2011

@Jagannath:
Many companies, mostly the larger ones, has some kind of base salary and for testers I believe it is not as high as other roles. Consultants and smaller companies can probably affect their salaries as you say.

Regarding “Of course people will create certifications which there is an opportunity to earn some money, fun for them.”, we can tell our customers not to fall for the lure and perhaps not work for companies that earn money on that kind of certification. I do not want to sell snakeoil to my clients, making money on something that has no value.

Regarding “Your knowledge and expertise is something that is constantly evolving, it is important to understand just that.”, I agree with you. Still, when we meet recruiters that think a specific certification is better than many years of experience we can tell it is most often wrong.

@Rikard:
Success stories are really great. How do we gain success stories and what do we do about them? We need to work on all levels to speed up things, as you say.

Johan Hoberg May 3rd, 2011

Regarding respect and salary of testers. When I read the Google test blog, and when I have heard James Whittaker speak, I get the feeling that they treat their testers (at least their SETs) just like they treat their developers.

http://googletesting.blogspot.com/2011/05/how-google-tests-software-part-six.html

What do you think/feel regarding this? Do you think their base salary is really lower at Google?

Best regards,

Johan

Martin Jansson May 4th, 2011

@Johan:
At Google, don’t you have to be a developer to become a tester? So tester = developer, thus similar salary and respect? Naturally there are companies where my theories do not apply.

Jagannath Tammeleht May 4th, 2011

@Martin

It is true. Many companies do not pay a fair wage. But then the question is, are all testers worth more money? not, in my opinion. I think it is a combination of several factors. Companies who do not understand that good tester costs money, and that the level and status of this business need to be raised further.

People who believe that certifications are better than experience crazy, I agree with that. But while you mention years of experience, a new problem arises. Published and experience is not necessarily something that are connected together. To many just focus on the years not on personality, ability to learn and develop. As a newcomer to the industry, it is something that I encountered to often, and for sure, in many cases quite right, but not always.

My idea to take the industry to the next step is to start more courses at university and colleges. Not that studies are better than practical experience, or because you can become a good tester because you have a bachelor’s degree in quality assurance. No, but because you are attracting people at an early stage in their careers and not people who are sent down to test the basement. I also believe that it would rise the status of the industry among the public, and thats god and will eventually lead to higher salaries.

/Jagge

Jagannath Tammeleht May 4th, 2011

Years and experience is not necessarily something that are connected together.

You should always verify before you send anything:)

Martin Jansson May 4th, 2011

@Jagannath:
I think we affect the tester worth by degrading and de-motivating them to a position they did no task for. Testers who want to be in their current spot will naturally be worth more to the company, but it also states something about what you think of the test proffession as in the reason for the whole article.

15 years experience in testing can mean a lot or little. I’ve worked with people who has been testers for more than 10 yeras, but they have never read any book, never read any article/blog or done any study to excel as a tester. They have tried to get better as domain experts, which I believe is a common situation.

Many testers out there are only tester by name, they see themselves as something else and do not see the point of getting better at actual testing. Why would you, if you just exeute a script?

Jagannath Tammeleht May 4th, 2011

@Martin

I hear you, and can not but agree with you

Johan Hoberg May 4th, 2011

Do you think that a university degree makes a difference in salary and respect?

Maybe more developers than testers have some kind of master’s degree or bachelor’s degree within a relevant area (software engineering, programming, or something similar) and thus get a higher base salary (and respect)?

I don’t know, just speculating for the sake of the discussion :)

Martin Jansson May 4th, 2011

I think there is an immense variety of setups for salary. In the cases where it is dependant on role, such as tester, I think testers in many cases have lower than developers.

In other cases when they look at degree, years of experience etc we will see other salaries for sure.

Stefan Thelenius May 4th, 2011

I am not sure if testers in Sweden have that much lower salaries than developers in general.

If you take a look at http://lonebarometern.idg.se/loner

Systemdeveloper 35190 SEK, 876 reported salaries
Tester 33522, 148
Test lead 38327, 140

So, if these figures are to trust in general I would say testers are not that far behind…

I think both testers and developers will be paid decent salaries in the long run if the add value to the company they work for.

“Execute script-only”-type of “testers” (checkers?) would I not pay much for as well as I would not for “code-monkey”-type of developers.

A tester with exploratory skills and developers with long-term design ability should earn good salaries and I think they have that in most cases…

Stefan Thelenius May 4th, 2011

Also found this at idg which makes it more even in the “tester vs. developer salary battle”:

Programer 32370, 256

Johan Hoberg May 4th, 2011

What I was trying to say was: Do you think developers in general have a higher relevant education, and that this contributes to the gap in salary and respect (if one exists)? And that if you take this into consideration there is in fact no gap? Is this a possibility?

As I said, I do not know, I am just thinking out loud.

Best regards,

Johan

Jagannath Tammeleht May 4th, 2011

Hmm i dont know if developers have more relevant education then testers. My idea in university education is that could be one in many ways to attract individuals to test busniess. I dont think it will change everything, it may change nothing at all but i would still support the idea. But its good reflection Johan, I like when people higelights problems.

I think u have right Stefan, if you have the right mindset around test and try to develop your skills the salary will increase.

I like your thought about domain skills, i also often meet testers that have worked for many years in same place, they have great domain skills but there test skills havent developt in same pace. But is this a problem? Or is it crossroad? Domain knowledge or test expertis, or is the middle way the best way? Like mellan mjölk :)

This is only my thoughts and I’m open to influences

/Jagge

Johan Hoberg May 4th, 2011

I completely agree with your thoughts on university education for testers. It is better today than 5-6 years ago, but there is still a long way to go. A lot of research is being conducted, and at least in Lund they are putting alot of effort into our field.

At least we have one course in Lund, Sweden:
http://cs.lth.se/ets200

But I agree with you that we need more to attract new talent and advance our field. Industry and academia in close collaboration I think is crucial for the future.

Best regards,

Johan

Martin Jansson May 5th, 2011

One article I forgot to include was http://www.kaner.com/pdfs/TheOngoingRevolution.pdf and http://www.kaner.com/pdfs/testingRevolution.pdf that are flag ships in this discussion. As Cem Kaner points out he went back to Univerisity to bring change. I think that is a good idea for us as well.

We should involve ourselves in the Universities to bring in more courses and add more value to the current ones. I know Henrik Emilsson is involved in courses in Karlstad and does a good job there.

Several of us are involved in Yrkes Högskolan. Me and Rikard are involved in Gotheburg and Henrik E in Karlstad. I think Henrik Andersson is involved in Lund or Malmö.

In september I will have 4 students that will work with me.

Who will be the first to get the tester into Hollywood? We need a movie about us perhaps?

Rikard Edgren May 5th, 2011

I think it will be difficult to create University courses that many can agree that they contain what is needed.
But we should still do it, as Henrik Andersson pointed out diversity is needed in software testing; we want more University courses, and other knowledge in our field.

The mentioned course in Karlstad is on test design, runs on springs at 25% speed, and can be done at a distance: http://www.kau.se/utbildning/kurser/DVGB09/testdesign-for-programvara

Karlstad will also this autumn start a 2-year YH-education for software testers: http://www.studentum.se/Kvalificerad_programvarutestare_Yh_143600.htm

Simon Morley May 6th, 2011

HaHa! I started writing this comment before I read the last paragraph under certification :( Anyway, here it is:-

I have a theory about why certification is popular with non-testers (managers & execs)…

Certification is a form of standardization – it’s compartmentalizing testing into something manageable, predictable and tangible. It taylorism in testing…

Many management practices of the twentieth century follow Taylorism, so it’s hardly surprising that finding something that fits into that model becomes popular.

It’s an observation of testing that “fits” with their ideas about management – “oh, wait – if you can manage testing just like any other aspect of Taylorism then that’s got to be a good thing right?” They’re not looking at the problem of “testing” vs “good testing” (or understanding that there is a problem) because in their experience they have a Taylorism approach….

Unfortunately, a Taylorist approach to management (maybe good for some production lines) doesn’t really fit with “knowledge work” (which is testing)….

Ref: http://en.wikipedia.org/wiki/Scientific_management

On the idea of perception it reminds me of something I wrote about building your brand – consistency is key to perception…