Sometimes when I read status reports or hear project managers talk about testing, I hear that “testing is blocked”. What do they mean by that? When I delve deeper in what they are talking about I sometimes see that the progress on workpackages for the testing team or testers in a team have been combined with the result of a test case such as pass/fail/blocked. But that is hardly the truth? As a tester we do a lot of things, not just running test cases (if we even do that form). If we say that one test might be blocked in the sense that it does not provide value in testing further there. That does not really mean we are blocked? Is a blocked test when you cannot get the answer to a certain question you have about a system? Does it mean that a specific test could not be done, but perhaps a similar one be done that would provide as much value?

If everything around you is blocked and there is nothing you can do as a tester, then you can perhaps say that “testing is blocked”. But what about preparing for coming tests? What about automating some of the things that you have found? Did we then mean that we were blocked in the sense that we did not come up with any ideas on things we could do? Well, perhaps you can look at various test idea triggers that can get you going?

Progress in performed test related workpackages and the result of testing are two different things. Combining them does not really give a good picture of what is happening. So, before stating that “testing is blocked” consider if you actually mean that you have no progress or that you are unable to test in a certain area. Based on what you inform and how you communicate, stakeholders will make decisions based on that.

I usually tell my stakeholders that they should not worry about me or my test team being blocked in progress, there is usually something valuable to do. Still, if I say that some tests cannot be performed or that they are blocked, where the result of those tests are important they need to assist with removing those obstacles.

During a daily stand-up meeting in a test team I was part of, nearly all testers reported that they were blocked and that they had no progress in their test cases. We had a long list scripted tests that needed to be executed according to the test project manager. I investigated what they actually meant when they said that. Interestingly, most of them had a test cases that wanted them to go in a certain direction through the system. What was blocking them was bugs. They found lots of bugs, but did not feel they had time to report them because they really needed to gain progress in those intended test cases. At that time we had to work overtime if we did not have enough progress in our test cases. I asked them to instead of trying to run the test cases they use the test case as a test mission or charter to guide them where to go. When they saw fire or smoke, they followed that trail and documented (session notes) what they saw and what happened, but also log all bugs they found. From then on not one reported being blocked in the daily stand-up meetings. The flow of reported bugs increased and the progresss was fine (in the sense that test cases were executed as far as project management thought). Stating that you are blocked might in fact be how you work that does not enable freedom and creativity.

If we were to measure something as a manager of testing, would we then see a decline in testers reporting being blocked when using methods such as SBTM instead of scripted tests that report pass/fail/blocked?

I see humility as a very good virtue.
It is something I have failed miserably at, partly because it is easy to think something is bad just because there are many problems.
I think it’s a common fallacy for many ambitious testers – you are last in line, maybe with lower status, you want to get heard and become too persuasive…

Humility tend to give better collaboration, and thereby better results. A humble tester more often changes her mind, which is necessary.
The complexity of reality demands a humble tester.

Here is my list of humbling experiences; things I think have made me a better tester.

* write requirements
* programming something bigger than an exercise
* manage projects
* customer report of an important bug you should have caught
* experiencing too little time to test the very important stuff
* failing to explain why your testing is good
* reading something you wrote some time ago, and realizing it’s very shallow
* understanding your lack of humility
* realizing you were wrong

Which ones do I have left?

I spent this weekend in nice, dark, foggy Ringsjöstrand for the third Swedish Workshop on Exploratory Testing. Johan Jonasson, Ola Hyltén, Anders Claesson, Oscar Cosmo, Petter Mattsson, Rikard Edgren, Henrik Andersson, Robert Bergqvist, Maria Kedemo, Sigge Birgisson, Simon Morley.

The format is LAWST-style, which means a presentation is followed by a facilitated discussion, that goes on as long as it brings value.
The theme for this event was Teaching Testing, and the abstracts can be downloaded here.

I did the first presentation using my current assignment as teacher together with Henrik Emilsson on a 2-year higher vocational study for testers.
We have a lot of exercises to make the knowledge stick, and we primarily use open source applications to make it more real.
I shared how we find suitable applications and tie them to something we are teaching, and I showed some examples of applications we have used.
The discussion that followed lasted a couple of hours and spanned many areas: course literature, complexity of software, unreal situations, authorities, ISTQB, tools, how to teach so they learn etc.
I learn a lot by teaching, and also by discussing the teaching.

The second experience report came from Johan Jonasson, who is involved as instructor in BBST online courses.
The videos are available for free, but the best thing with the education is the realistic problem-based assignments and the peer reviews.
Discussions followed for a couple of hours and included cultural and online difficulties/opportunities, how to give feedback in writing, and a summary of the slides material:
“a condensed library of test information that will broaden your views.”

After a break, we had six eight-minute Lightning Talks, where you can’t go deep, but a lot of ideas are brought up:
Anders Claesson shared his model for making sure students really learn what is thought.
Sigge Birgisson talked about his efforts to create a quality vision/model/goal for deliverables.
Rikard Edgren wondered why no one (except Oscar Cosmo) test charisma when we all know it is important. (Emilsson came up with the name Charisma.)
Henrik Andersson showed that objectivity is an illusion, so we should focus on the inherent subjectivity in test selection, execution, interpretation.
Robert Bergqvist wondered if Exploratory Testing has been more accepted, or if it just is a word in fashion.
Simon Morley has a great idea on a Groopman-inspired book “How Testers Think”, although he denies he will write it.
Then we had dinner, bubble pool, a lot of conversations on many subjects (“lean is waste for testing”), perhaps a small beer, and a good night’s sleep.

The second day only had room for one topic, which was Simon Morley on “Mindset Changes: Changing the direction of the oil tanker”, about how to teach the “right” view on testing to both managers and testers.
Many suffer from test case counting, and they have to be cured in different ways.
Simon says to testers he don’t want to hear any numbers, and thereby force them to talk about what they have seen, and what they haven’t done.
He does presentations to managers about problems with number focus, and after that he can use dashboard-like reporting.
The most important dimension might be trust.
Consensus in the room was that gut feelings are better for making decisions, but that we don’t have the proper words to talk about this.
A lot of examples and ideas were shared, but we didn’t reach any solutions before the time was up…
“if communication isn’t good, it doesn’t matter what you do”

Eleven happy check-outs with a richer network and knowledge; SWET has charisma.

I spent two days in Malmö attending a developer conference with a fantastic test track (put together by Sigge Birgisson.)

I did a presentation on Curing Our Binary Disease (slides, abstract), which was much better received than I hoped for (I thought it was a binary love/hate talk)
Good questions and talk about being inside the potato, how do you know where you are?

Pradeep and Rikard in front of Øredev projector

Pradeep Soundararajan talked convincingly about caring for the users, e.g. by using Twitter as a source for test ideas.
He also said the context-driven community has moved from Pass/Fail to “Is there a problem here?” (still a binary question though…)

Shmuel Gershon shared an experience report of a 100% exploratory testing project, with qualitative reporting and everything (“it’s called quality report, not quantity report”.)

Gojko Adzic talked about testers needing to adapt to how development is done now, and put most of their time ensuring others write good automated tests.
“There’s so much mistrust in the processes.”

Henrik Andersson wants, with right, a diversified testing team, and therefore only hires context-driven testers
Just joking, he said people can read the same excellent books, as long as they think critically and learn other things.

Selena Delesie explained how to focus the testing effort on customer needs; “testers are information radiators”.

Zeger van Hese made the presentation that I enjoyed the most. A thoughtful walkthrough similarities between the fine arts and testing.
“The Hungry Eye – thoughtfully looking at software”

Janet Gregory highlighted five enlightened areas since Agile Testing was released: feature acceptance, collaborative automation, large organizations, distributed teams, continuous learning.

Outside the venue I had many interesting conversations, there were testing games, food and drinks, Black Viper and even imitations, and all in all a very nice experience.
Thumbs up for a testing session with Pradeep and Shmuel that showed (the need for) different note taking styles.

The presentations will be made available online at www.oredev.org/

I know of some testers who are pointed out by others to be Bug Magnets; people recognized for their ability to somehow draw bugs to them. Bug Magnets can be found in many workplaces and I bet that you know of someone that falls under this description. I have been appointed a Bug Magnet by some and it have made me thinking on what this phenomena boils down to.
Is it luck? Is it faith? Is it an ability that some are born with and some aren’t? Can you learn this ability? Can you improve it?
I have wondered about this for some time.

However, this summer I had a revelation when I watched an episode of the marvelous TV series “Homicide: Life on the Streets”:

Homicide: Life on the streets
Season 1 Episode 9 “Night of the dead living”
Det. Frank Pembleton and Det. Tim Bayliss

Bayliss [sits pensively. To Frank]: What are you looking at?

Pembleton is playing cat’s cradle.

Bayliss: You have something that you wanna say to me?

Pembleton: Adena Watson. So many unanswered questions.

Bayliss: And you’re saying that I’m not asking them.

Pembleton: I’m saying that you’re not answering them. [He peers at Tim through the cat's cradle.]

Bayliss [sighs]: What questions aren’t I answering, huh?

Pembleton [gets up and flips through a notebook, draws a picture]: Okay, these sixteen row houses on the north side of Kirk Avenue. Adena’s body was found outside the kitchen door in the red yard at 718 Kirk. Now the killer could have dropped her anywhere. Why not the common alley? Why not the yards at either end of the block? These three row houses are empty. One, two, three. The killer would have stood much less of a chance of being seen if he’d dumped her body in any one of these yards. Why would he risk bringing a little girl’s body inside a closed fence of an occupied house?
Maybe he wanted her body to be found immediately. Maybe he wanted to cast suspicion on the people in 718. Maybe he had some … perverse sense of remorse, some impulse to leave her body inside an enclosed yard to protect her from stray dogs.

Bayliss: These are *exactly* [taps the notebook] the questions that I have been trying to answer.

Pembleton: Well, you can try, but you never will.

Bayliss: Why?

Pembleton: You don’t think like a criminal. You don’t have a criminal’s mind.

Frank walks away. Tim grimaces in disbelief.

Bang! Suddenly it struck me. “You don’t think like a criminal. You don’t have a criminal’s mind”.
In order for a police to think of possible outcomes of a crime they have to be able to think as criminals, and put themselves in a criminal’s way of thinking.
If you don’t do this, you are trying to understand and explain the crime based on your logic.
Similarly, in order to “attract” bugs you have to wanna see problems; you have to identify problems that might bug several kinds of stakeholders; you have to put all your knowledge about the project in to consideration; and you have to be able to see the problems that matter. You are not trying to understand and explain the system by using your own logic, instead you are using several input sources to do this: logic, subjective thoughts, people’s skill levels, complexity of system, technology, etc.

So let me present my take on a definition of “Bug Magnet”:

Being a Bug Magnet = Able to foresee possible problems (or able to spot opportunities for things to go wrong), in context.

The most important thing here is the last two words “in context”. That is, even if you have all the bug taxonomies and oracles in the world to support you, you have to be able to understand what matters in this project.
Knowledge about “all common problems in .NET applications” can help you sometimes. Knowledge about “all common problems in .NET applications that developer X often produce”, is however much more useful.
Understanding what bugs our users is more helpful than knowing about what bugs users in general.
Knowing about problems with focus in Windows applications is one thing, finding these problems during testing is to be able to spot opportunities when they are presented to you in your context (see http://thetesteye.com/blog/2010/11/windows-focus/ ).

Now, back to the questions in the beginning of the post.

• Is it luck?
  – No, even if luck sometimes help. For others it is more of welcoming serendipity.
• Is it faith?
  – I don’t believe in faith.
• Is it an ability that some are born with and some aren’t?
  – Maybe. Some people might have a more developed talent, but I think that most people have this talent. Some are born with variations of narcissistic personality disorders and might have difficulties with this.
• Can you learn this ability?
  – I believe so.
• Can you improve it?
  – Yes. However, you might need to consider one or several dimensions to improve: Empathy, reasoning, attention for detail and seeing the whole, recognizing patterns of your own and other peoples mistakes, subjectivity,  general systems theory, context-driven testing, and more.
  Notice that these dimensions are not technical but rather comes from social sciences.

This is a follow-up from Working with the testing debt – part 1 [1] and part 2 [2]. The reason for the clarification is that you so easily come up with a tip without a context or example.

Tip 3: Grow into a jelled team (read Peopleware [3] by Timothy Lister and Tom deMarco for inspiration). Peopleware identifies, among other things, things that you should NOT do in order for a group to grow into a jelled team. As a team it is so much easier to gain momentum in testing and especially so if you are jelled. Do you need to reconsider how you work as a team?

In one project I was test lead and tester. I had two fairly inexperienced testers assigned to me. I used a somewhat exploratory test approach to the planned tests, but they were far from organised and not well communicated. The tests consisted of a long list of one-liner test cases/test ideas. They were quite vague, thus enabling the tester to think for him-/herself. I had added some very specific test cases that were meaningless to run several times, but they were mixed in with the more vague ones. The testers thought it was meaningless to run the test cases several times, but I wanted them to do it anyway. Still, my intention was that they only use them as a guide. I did not communicate well and the testers did not feel like they were part of the test planning (which they weren’t) and thought I had set it up wrong. The mood in the team was foul. The testers had a hard time working with me for several years after that. Looking back I think they spent quite a lot of time that did not give any new information from what we had.

In another project I was assigned as tester together with my regular test group. We worked as consultants at the time. We did not know the project manager, test lead or any of the other testers that well. We were assigned test cases in different areas of the product suite. The test lead walked around and handed out test cases. There was no collaboration in the team. Sometimes when we talked over a coffee we noticed that we had done double work. Some tests were to be run in quite advanced configurations that took 90% of the time to setup. After a while we realised we spent 70-80% of the time setting up configurations, 10-15% of the time reporting bugs and the rest actually testing. By collaborating more we could have saved time to better switch tests around in the configuration that were needed between us. The team composition was 70% new consultants and 30% perm. employees.

And in yet another project the test team had worked together for quite some time. The whole team was involved in creating test missions/charters. Everyone tested and assisted in changing the test plan. When there was a major area that needs to gain some focused testing we assembled a group who generated test ideas, did some test design and did collaborative notetaking on the specific area. During the day the group debriefed several times and changed the test scope based on the feedback. Everyone was involved and felt great.

At one company we had assembled all testers into one big team (see [4] and [5] for setup) where we assisted all projects, the support organisation, the marketing organisation as well as taking on special missions from managers at different levels. When we started we had some unresolved conflicts, but as the team grew and as time went by the conflicts diminished. We worked hard together serving the organisation as best we could. As the team set forth we had the Black Team (inspired from Peopleware) in mind. We challenged the organisation to give us more to test, we could take on anything. We might have taken a bit too much pride in our work, but still I think that was needed after years of being looked upon as the nagging, complaining test team.

At the same company as above we sat in groups of four. When we got a new build we were armed to the teeth with test ideas, assigned areas for investigation and energy to shot down (not literarely) anything coming our way. During testing someone could shriek out in delight as a critical bug was found. The whole group would sometimes plung in and attack the same area from different angles to root out more issues, then continue on the track that they left earlier. We longed from the untouched (by testers mind) new features. We overwelmed the organisation with bugs, never having to consider that we might be the bottle neck at any time.

We all act differently depending on the context. With experience you hopefully learn from many of your mistakes. After you have been in a jelled team, you wish to be in the same situation again. You now know how good things could be and when you are not in that flow, you try to determine how it could be so. The book Peopleware is one of those things that might get you ideas on what you or people around you should stop doing  in order for your team to become jelled. Do not be afraid to tell management things that they that will stop you from growing. In some cases they might not know that they are doing it in a way that corrupts.

How does this affect the testing debt?

The benefits are obvious, a sentence to rise a cautioning finger about, but perhaps somewhat valid in this context. Part of the dilema is how you perceive the team that works well together. Do you see it is as a Clique (roughly described as an arrogant group) or as a Jelled Team? For my description I emphasises on a team that see itself as a jelled test team. When I say to increase the testing debt I mean that your backpack is getting heavier. Each time you start a new task, new project or new assignment you take the content of your backpack into consideration. Being a jelled team enables you to move more smoothly and become more flexible, thus able to become quicker and hopefully deliver better result.

We should also consider that there are differences between a team that is jelled consisting of various roles (such as a cross-functional team) and a team with only testers. Both constallations have different issues and both will have an ever growing testing debt. Some tips would apply to one constallation and not the other, naturally.

Kay Johansen and Anthony Perkins identified a few interesting ideas in their paper Establishing an Agile Testing Team: Our Four Favorite “Mistakes” [6] that I think is relevant to building a jelled testing team. We also have a paper by Elizabeth Hendriksson on Agile Testing [7] that identifies things you should not do as a tester if you intend to be agile. They are most relevant when considering what role testing should have in a cross-functional team or what your attitude is as a test team. Brian Marrick has made a compilation of articles [8] on Agile Testing, somewhat old ones but still valid as a way to consider when trying to become more effective and build a jelled test team. Marrick has also created the article Classic Testing Mistakes [9] that is valid still and should be considered when trying to identify why your team is not jelled yet. I’ve also written two blog posts about growing test teams: Uncertain team composition [10] and Progress [11]. Both tries to focus on things you should not do in order to get a jelled test team.

Some specific things that I see a jelled test team do, that decrease the testing debt, is:

• Build on each others test ideas. With this I mean that we are triggered on each others test ideas to create new ones until we have no further ideas.
• Easy of changing direction. I often promote the concept that the test team or the team in general need to be flexible, thus the naming of The Flexible Test Team. If the team works well and trust each other it will be easier to accept change.
• Collaborative notetaking. In a team where you want to work together and where you enjoy cooperation it is the teams effort that counts, not the individual. This is also true for taking session notes. This is no best practice (as none exists), but a practise that is good in some situations. That the team have the ability to do this is a bonus.

With an unjelled team, a group of people that do not collaborate as well as they could, thus increasing the testing debt, I instead see:

• Stacking of test ideas. This means that each individual identify test ideas and they are summed up, with duplicate ideas removed. An unjelled team is more inclined to do this than a regular jelled team.
• Double work and duplicate testing. This might be the result of bad leadership and test leading, but based on my experience I think it is more common than in a jelled team.

What things do you consider for a jelled test team or a jelled team that has mixed roles but with testers in it? What have I missed as you see it?

References

[1] Working with the testing debt – part 1 - http://thetesteye.com/blog/2011/05/working-with-the-testing-debt-part-1/

[2] Working with the testing debt – part 2 - http://thetesteye.com/blog/2011/06/working-with-the-testing-debt-part-2/

[3] Peopleware - http://www.amazon.com/Peopleware-Productive-Projects-Teams-Second/dp/0932633439/ref=sr_1_1?ie=UTF8&qid=1310020112&sr=8-1

[4] Flexible testing team - http://thetesteye.com/blog/2008/05/the-flexible-testing-team/

[5] Resource planning in a flexible test team- http://thetesteye.com/blog/2008/05/resource-planning-in-a-flexible-test-team/

[6] Establishing an Agile Testing Team: Our Four Favorite “Mistakes” by Kay Johansen, Anthony Perkins - http://agile2004.agilealliance.org/participate/examples/EstablisingAnAgileTeam.pdf

[7] Agile Testing by Elizabeth Hendriksson - http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf

[8] Agile Testing by Brian Marrick –  http://www.exampler.com/testing-com/agile/

[9] Classic Testing Mistakes by Brian Marrick - http://www.exampler.com/testing-com/writings/classic/mistakes.htm

[10] Growing test teams: Uncertain team composition - http://thetesteye.com/blog/2010/04/growing-test-teams-uncertain-team-composition/

[11] Growing test teams: Progress - http://thetesteye.com/blog/2009/10/growing-test-teams-progress/

This is a follow-up from Working with the testing debt – part 1 [1]. The reason for the clarification is that you so easily come up with a tip without a context or example.

Tip 2: Focus on what adds value to developers, business analysers and other stakeholders. If you do not know what they find valuable, perhaps it is time that you found out! Read Jonathan Kohl’s articles [2] on what he thinks adds value for testing.

In one project I worked with a group of experienced developers. I had not worked with them before close hand, but they had received some of my bug reports and knew me by reputation (whatever that was) in the company. When I got their first delivery to me to test I started right away. Immediately I got the feedback that I was not testing the right stuff and there were a bit chilly in their demeanor towards me. I investigated a bit what had happened and found out that they were not really interested in the minor bugs at that moment and that I should focus on the major issues. I explained to them that I report everything I find, I was not expecting everything I found to be fixed though. What was fixed was up to those who prioritized the bugs. Before I started testing I asked them what they wanted me to focus on first. After that they were a lot happier, both knowing I worked on things valuable to them and that they understood that I reported everything that I found.

During another project we were two weeks from the release of the product. We were in the middle of a transition from traditional scripted testing to an hybrid with both scripted and exploratory testing. Rather, we had test scripts that we used as a guideline when we explored, but we reported Pass/Fail on those. At that time Project Management was strict in wanting number of test cases run as well as the Pass/Fail ratio. Earlier test leads had not communicated well why these figures held no value. When we had run all planned test cases project management communicated to their managers that we were done. But we were not, we continued with working on our planned charters and ran sessions. We interviewed the support organisation, business analysts, product management and experts in the test organisation. Eventually we got a long list of risks and areas that we should investigate. We also got a long list of rumours that we intended to confirm or kill. Basically, we were far from done and we still had time before the release as we saw it. We had also received areas that people in the organisation found valuable to get information about. Still, we failed because we had not communicated enough to project management what we were doing. We managed to go through most of the areas and identified lots of new issues as well as killing many old rumours. We failed to bring value to some, but not all.

How does this affect the testing debt?

If you continue to work on things that have no value to you or any of your stakeholders, you must take a stand and change things. Do not accept the situation as it is. If you and everyone around you think you and your test team are not doing anything of value, it will just add to your testing debt.

As I state above Jonathan Kohl gives a good set of questions [2] for you to ask yourself to get back on the path. Also consider what Cem Kaner writes about in Ongoing revolution if software testing [3], because it is still ongoing and it not over.

References

[1] Working with the testing debt – part 1 - http://thetesteye.com/blog/2011/05/working-with-the-testing-debt-part-1/

[2] How do I create value with my Testing? - http://www.kohl.ca/blog/archives/2010_08.html

[3] Ongoing Revolution of Software Testing - http://www.kaner.com/pdfs/TheOngoingRevolution.pdf

This began as an answer to Rikard’s post http://thetesteye.com/blog/2011/06/a-word-of-caution/ where the discussion on “traditional testing” came up.

I often hear comparisons with our “industry” and the Automotive industry.
In that context, you could say that “traditional testing” corresponds to the methods and practices that are applied in line production of large car companies. And the unorthodox testing can be compared with those specialized and often smaller custom car builder companies out there.

The major issue with this kind of comparison is that large car companies makes thousands of the exact same type of car. Instead this means that the “traditional testing” approach is an attempt to apply line production methods when building custom cars. Applying “traditional testing” as if every project and product were the same is both wrong and dangerous.

And this comparison is not that far-fetched… It seems to me like “traditional testing” is promoted as practices that suits many (if not all?) projects and should be followed in order to enable success. Well, good luck!

Further, many practices that we use today comes from the automotive industry (at least in their latest form).
If we fail to see why they implemented them in the automotive industry and just take them as good practices for being effective in line production, we are doing the opposite of what the automotive industry did. They did some investigation on how they could improve their work. Some of that included seeing the human beings as intelligent and social creatures; utilize the diversity of a group of people; etc. And their productivity and efficiency could be measured by measuring number of cars and their quality. So by treating humans as humans they became more efficient (number of non-defective cars) and improved their work methods (analyzing quality of work).
When Lean development is implemented in psychiatric nursing or software development today, the focus is very often on the quantity measuring part which then misses the whole point. Measuring patients or software as uniform units is very wrong and dangerous.
It’s not that Lean or Kanban is to blame, it’s the implementation; and perhaps mostly the implementors.
The role models for Lean implementations in many healthcare institutions in Sweden have been some successful nursing teams that have increased their efficiency and quality of their work by using Lean development as a method. What those successful teams really did were to take command of their own work and found a method that supported their initiative and commitment. (Anyone had similar experience? Me!)
The problem is that when this is implemented by management at the whole hospital or nationwide, the focus is shifted from “Quality of work” to “Quantity of work” because that is the obvious driver for management and really the only incentive for they to implement it. They only need to say that this is a “best practice” and then it’s OK…
I do need to emphasize that you don’t automatically get high quality work by implementing “best practises”!

This is happening all the time; and the last flavor of the month is Agile.
If you read the Agile Manifesto and then think that you must play planning poker or have standup meetings you are obviously not understanding the Agile Manifesto.
I’m with Matt Heusser on his interpretation here: http://www.softwaretestpro.com/Item/5036/My-%27take%27-on-Agile—-or-the-Manifesto-elaborated/Testing-Software-Test-and-QA-Teams-Strategy-Agile-Development

Background

When I first got in contact with software testers, I worked as PM and developer for a language tool. Our CEO had said that he had hired two testers, easily since you can just pick them from any street corner. Sadly they had no clue what to do and did not find any bugs, they just found out how the OS worked or things that were built-in. After some time we were able to get a new group of testers and now things really changed. Some of them were aspiring to be developers, but settled to be testers for a short time. At that time they had no knowledge about how testing should be done according to the so called rules, but they did a good job and found bugs in our software.

Some year later I began at a product development company. During my years there we had change of manager almost every year for the test department. Each one brought their own perspective on testers. Most of them accepted any personnel from any department when there was lack of testers. During that time we got to experience a lot of different backgrounds, skills and interests from the extra personnel. We also experienced many employees who were moved or even demoted “down” to the test department. Many stayed in testing where they excelled and eventually liked it. During all those years management saw us as the complaining guys from the test department, perhaps a too common view? What we really did was express risks, bugs or any information we thought endangered the company or products under test. I am sure their perception of us was misplaced, but naturally we were somewhat to blame for how we communicated and how we acted when communicating.

Some years later I joined a smaller company with mostly researchers and scientists. Most of them were used to working alone in development projects so they did all things themselves. They did not see the need for testing as a discipline on its own. Eventually when we (testers) got something to test we showed them it was a big difference with what we found compared to them.

The thing that is constant is the confusing perception on a tester is and what we should do.

How are testers perceived?

If you look at testers from a salary perspective we very often have lower salaries than developers and project managers, but we have higher than documentation specialists and support personnel (at least in Sweden). For many salary also drives your career choices, so you naturally want to get out of the testing department. In Sweden consultants can charge higher for test leads than testers at many major customers. This does not motivate consultancies to grow great testers.

If you look at testers through career perspective you often see that tester is a pit stop in pursuit to become a developer. Or perhaps more rarely you see people have been demoted from other positions. Someone needs to take the role of tester, let’s take the person we need the least for other tasks. I also see personnel that are promoted from support to testing (as they express it). If you become test lead you might be on the way to become project manager. Managers know that many with higher ambition will just pass through the test department, while others less motivated will stay behind. Still, there will always be a group of testers who love testing and want to excel in it, but some companies do not have them yet.

In the scripted test approach you most often want a domain expert to write test cases and let someone else (or sometimes the same person) execute the tests. In this situation the tester can be “anybody”, he/she just need to execute the tests. When a manager is seeking new resources to become testers he will accept anybody to become a tester, than you have the potential of getting anyone, even demoted personnel, from other parts of the organisation. This is the most common view on testers, as I see it.

Certification

During my whole career I have not heard that many talk about the need or requirement of certification at places where I worked or at clients. In one case a tester approach me, when he was about to enter my test group. He said he was ISTQB certified and that his employer required all testers to be certified. I told I was not, but I had more than 10 years of test experience and close to 20 years of product development experience. Was that ok? I asked him of his testing skills and what he could do to contribute to my team. He got scared and did not want to join the team. I regret that I scared him off like that. Someone must have introduced the idea that to be a good tester you need to be certified. Or was it perhaps set up as a minimum requirement when handling allocating personnel to teams? Perhaps the original intention was certified tester or experiences enough to cover it? There is seldom context behind decisions like that. My belief is that some consultancy got them to buy-in on the idea, then sold them lots of courses and certification packages.

After reading Dorothy Grahams blog posts ([1], [2] and [3]) about the intention of certification, I wonder why no one spoke up about where things were heading. Their intent might have been to make the perception on testers better, but I think it instead has hurt our craft. At each conference and at most meetings there is often someone who speaks up with lots of argument against certification. I rarely see anyone take up the discussion to meet their arguments or perhaps I do not listen? James Bach has made a lot of good arguments [4].

There are many so called test experts out there who say that certification such as ISEB or ISTQB is needed to be a tester. Some companies even require it of their testers and therefore the recruiters require people seeking jobs to have it. I think it is all a charade. Having testers who take courses in testing, who read books, blogs and articles, who want to learn and who want to excel as testers are what is needed. Passionate testers who want to become great! If they are certified that is ok, perhaps they got some ideas from it and they might have had a great teacher who stimulated them into becoming passionate themselves.

ISTQB uses multiple-choice questions on their exams, but they are quite limited. Cem Kaner has written an excellent post about Writing Multiple Choice Test Questions [5] where he makes some strong arguments. If ISTQB was altered along those lines it would make it harder to pass and naturally harder to create, but it would still not solve the main issue with content being out of date and totally wrong in many areas, as I see it. Jonathan Rees brings up other strong arguments about multiple-choice questions in his article “Frederick Taylor In The Classroom: Standardized Testing And Scientific Management” [6].

Attitude

Just because we have to work up streams does not mean we can keep on having a lousy attitude. I’ve often seen us picture ourselves as victims because of our situation, lack of personnel, time etc. If we are too few to test and if we got too little time, we can only offer to do our best. We can also explain what we could do if we were more and if we had more time. The prior combined with that we often speak in anger when we talk about quality. This only fuels the perception that we are a bunch of idiots, angry ones.

When we get deliverables from developers we are sometimes angry because of the bad quality or the lousy state of a certain build. Do we consider why it is like that, what shortcuts they needed to take or if someone forced the delivery of a new build? Do we really need to focus our blame on the developers? Consider their ever increasing technical debt that they might not get proper priority to adjust.

In most areas of expertise you have lots of education, at various levels of the school system, to back you up. This has just started to get going with testing. At least it is not only a chapter in a book that you skip. There are lots of books, articles, blogs and other sources of information to gain other peoples experience on testing. Why is it ok to think you do not need to learn more about your craft? Why do so many testers with lots of years in the testing craft still state that they have not studied anything to get better at testing? Having that attitude damages the perception on testers by keeping you ignorant of what you claim to be expert at. With the increasing use of agile teams where a tester has a natural part, you are supposed to know at least something about your craft.

What do we do to affect that perception?

If we are continuously providing valuable information to our stakeholders the perception will be altered. This means that you need to know what they find valuable and what could threathen that value. You also need to consider how you communicate, thus in what form, if you are going to use metrics or not, how much subjectivity or objectivity you should use and how you act when communicating. Less drama-queen and more professionalism.

We are working up streams here, so everything that you do that is bad will have a great impact on the perception on testers. Where ever you go you will bring your attitude and ambition. When interacting with non-testers consider what you are saying and how it might appear to them. Consider if you are in the correct crowd to utter your disapproval, if you need to go somewhere else or if you can just go to your manager.

We need to communicate to managers that it is demeaning and de-motivating to be seen as idiots or just anybody. We need to show that having skilled, passionate and motivated testers will give a lot better result. What else can you do to motivate yourselves to get those attributes? Those who have been demoted or are de-motivated, show them how creative and exciting the testing profession can be. Bring in other external passionate testers to give them some new ideas. If nothing of this work, perhaps they need to find what they really want to do and go there.

Before accepting new testers to the team, we need to make sure they are right for the job. Do not accept demoted personnel without explain the consequences. When you as test lead discuss having extra personnel join your team, clarify that you want to test them before accepting them into the group and that some in the team need to be able to veto acceptance.

We need to tell developers that we understand that they must take shortcuts, thus increasing the technical debt, but we can help [7]. Work closer with the developers. Stop building walls between you. The more the developers trust and respect you, the more information you will have before you commence your work as a tester which will lead to a better work done. Remember a good bug is a fixed bug.

Consider how the test organisation is built, how it markets itself and what you communicate to management. See Scott Barbers excellent blog about “What being a Context-Driven Tester means to me” [8] that can be used as a starting point for you and your test organisation. Also consider where you are going with testing [9] to understand where you come from, what your next goal is and perhaps what is pushing you in a certain direction. Are you going in the right direction?

Conclusion

I think the perception on testers is our greatest nemesis, we have to fight it every day.  Certification in testing does not help us, as I see it, but it is not our main target for concern just one of the bullies. There are many things that make us get a bad reputation and are therefore perceived badly. Start changing your own ways and affect those around you to become great, passionate testers who deliver valuable information effectively.

References

[1] Certification is evil? - http://dorothygraham.blogspot.com/2011/02/part-1-certification-is-evil.html

[2] A bit of history about ISTQB certification - http://dorothygraham.blogspot.com/2011/02/part-2-bit-of-history-about-istqb.html

[3] Certification does not assess tester skill - http://dorothygraham.blogspot.com/2011/02/part-3-certification-schemes-do-not.html

[4] Search for ISTQB at James blog - http://www.satisfice.com/blog/index.php?s=istqb or http://www.satisfice.com/blog/index.php?s=certification

[5] Writing Multiple Choice Test Questions - http://kaner.com/?p=34

[6] Frederick Taylor In The Classroom: Standardized Testing And Scientific Management - http://radicalpedagogy.icaap.org/content/issue3_2/rees.html

[7] Developers, let the testers assist with the technical debt - http://thetesteye.com/blog/2011/01/developers-let-the-testers-assist-with-the-technical-debt/

[8] What being a Context-Driven Tester means to me - http://www.testingreflections.com/node/view/8657

[9] Where are you going with testing - http://thetesteye.com/blog/2010/04/where-are-you-going-with-testing/

We, testers, are we all alike? Looking back over the years I’ve been involved in testing, I would say I have met a whole bunch of different testers. Some shared my passion and fascination for testing, others were not testers they just happened to have the job title. However, I have always appreciated our craft for having such diversity of people. We have testers that have strong domain knowledge, programming skills and business knowledge. All of those are easy to connect the value to have in a test team, they find important information about the technical implementation of the product.

Then we have the other group, the ones who have capabilities in philosophy, cognition, history, economics, art, music and myself a drop out from statistics at university. We are not here because we love code or the coolest technologies, we are here because we love the way people and products interact and misunderstand each other. We are fascinated by the dance between a human and a machine. We find patterns in this dance, we find unintentional behaviors, and the unexpected surprises us. We look beyond constraints of the application, we go where no developer intended to go. By doing this we notice other kinds of things that might be a threat or a problem for the success of the product. Why are we doing this you may ask? It is because the technical solution does not impress us, we do not take particular interest or care in what specific way the code is written. What drives us is the question: will this solve a problem for a user and in which ways can it fail to do so? What we bring to the table is the social and behavioral science aspects of the product combined with the human element. At technology facing companies I sometimes see the above overlooked or under-valued.

But what worries me more is that we, testers, are walking away from my precious diversity. Strong forces favor uniformity among testers and the most obvious one is ISTQB. After going through their program of various certifications everyone knows the same, speaks the same language and uses the same templates. There is no problem replacing one ISTQB Advanced Level certified test manager with another one with same level of certification. The same goes with TMap, as long as you can read the book you know the answer to all testing problems, it’s easy!

But this is nothing new, I and many with me have been fighting both ISTQB and TMap for years. But here comes the kicker, have you heard of this cool new thing called “agile”? It’s a really sweet thing, close to a drug in our craft. It came and it has conquered. Nowadays there are almost no companies who dare say that they are not “agile”, instead they make the most ridiculous variations of what they call “agile”. Don’t get me wrong here, this new movement is quite appealing to me and I appreciate much of what it stands for.

Now getting back on track there is one thing that worries me about this agile thing. That is the view on us testers. Now to be an “agile tester” you basically need to be a developer with preferably some basic knowledge of testing, but it is much cooler if you know Selenium, TDD, ATDD and other strange abbreviations. If a tester according to the agile folks is basically a developer, why do you call them a tester, or wait you don’t! Everyone is a team member, I’m sorry.
Now we are walking from “must have a certification” to “must have programing skills”. Guys, don’t you see what is happening. We are just replacing one uniformity with another, beware of this!

We are running a great risk that all our testers are finding the same information and worse, are blind on the same spots. In testing we do not know what information we will get and we do not upfront know what questions to ask the product (don’t be mislead to believe anything else). We have to approach the testing from various different angles and have several different ears and eyes observing it. If one thing, we should love diversity among testers and we should encourage it.

This post came to my mind when I once again listened to Malcolm Gladwells brilliant TED Talk.

And no we do not all want black coffee!