This is a follow-up from Working with the testing debt – part 1 [1] and part 2 [2]. The reason for the clarification is that you so easily come up with a tip without a context or example.

Tip 3: Grow into a jelled team (read Peopleware [3] by Timothy Lister and Tom deMarco for inspiration). Peopleware identifies, among other things, things that you should NOT do in order for a group to grow into a jelled team. As a team it is so much easier to gain momentum in testing and especially so if you are jelled. Do you need to reconsider how you work as a team?

In one project I was test lead and tester. I had two fairly inexperienced testers assigned to me. I used a somewhat exploratory test approach to the planned tests, but they were far from organised and not well communicated. The tests consisted of a long list of one-liner test cases/test ideas. They were quite vague, thus enabling the tester to think for him-/herself. I had added some very specific test cases that were meaningless to run several times, but they were mixed in with the more vague ones. The testers thought it was meaningless to run the test cases several times, but I wanted them to do it anyway. Still, my intention was that they only use them as a guide. I did not communicate well and the testers did not feel like they were part of the test planning (which they weren’t) and thought I had set it up wrong. The mood in the team was foul. The testers had a hard time working with me for several years after that. Looking back I think they spent quite a lot of time that did not give any new information from what we had.

In another project I was assigned as tester together with my regular test group. We worked as consultants at the time. We did not know the project manager, test lead or any of the other testers that well. We were assigned test cases in different areas of the product suite. The test lead walked around and handed out test cases. There was no collaboration in the team. Sometimes when we talked over a coffee we noticed that we had done double work. Some tests were to be run in quite advanced configurations that took 90% of the time to setup. After a while we realised we spent 70-80% of the time setting up configurations, 10-15% of the time reporting bugs and the rest actually testing. By collaborating more we could have saved time to better switch tests around in the configuration that were needed between us. The team composition was 70% new consultants and 30% perm. employees.

And in yet another project the test team had worked together for quite some time. The whole team was involved in creating test missions/charters. Everyone tested and assisted in changing the test plan. When there was a major area that needs to gain some focused testing we assembled a group who generated test ideas, did some test design and did collaborative notetaking on the specific area. During the day the group debriefed several times and changed the test scope based on the feedback. Everyone was involved and felt great.

At one company we had assembled all testers into one big team (see [4] and [5] for setup) where we assisted all projects, the support organisation, the marketing organisation as well as taking on special missions from managers at different levels. When we started we had some unresolved conflicts, but as the team grew and as time went by the conflicts diminished. We worked hard together serving the organisation as best we could. As the team set forth we had the Black Team (inspired from Peopleware) in mind. We challenged the organisation to give us more to test, we could take on anything. We might have taken a bit too much pride in our work, but still I think that was needed after years of being looked upon as the nagging, complaining test team.

At the same company as above we sat in groups of four. When we got a new build we were armed to the teeth with test ideas, assigned areas for investigation and energy to shot down (not literarely) anything coming our way. During testing someone could shriek out in delight as a critical bug was found. The whole group would sometimes plung in and attack the same area from different angles to root out more issues, then continue on the track that they left earlier. We longed from the untouched (by testers mind) new features. We overwelmed the organisation with bugs, never having to consider that we might be the bottle neck at any time.

We all act differently depending on the context. With experience you hopefully learn from many of your mistakes. After you have been in a jelled team, you wish to be in the same situation again. You now know how good things could be and when you are not in that flow, you try to determine how it could be so. The book Peopleware is one of those things that might get you ideas on what you or people around you should stop doing  in order for your team to become jelled. Do not be afraid to tell management things that they that will stop you from growing. In some cases they might not know that they are doing it in a way that corrupts.

How does this affect the testing debt?

The benefits are obvious, a sentence to rise a cautioning finger about, but perhaps somewhat valid in this context. Part of the dilema is how you perceive the team that works well together. Do you see it is as a Clique (roughly described as an arrogant group) or as a Jelled Team? For my description I emphasises on a team that see itself as a jelled test team. When I say to increase the testing debt I mean that your backpack is getting heavier. Each time you start a new task, new project or new assignment you take the content of your backpack into consideration. Being a jelled team enables you to move more smoothly and become more flexible, thus able to become quicker and hopefully deliver better result.

We should also consider that there are differences between a team that is jelled consisting of various roles (such as a cross-functional team) and a team with only testers. Both constallations have different issues and both will have an ever growing testing debt. Some tips would apply to one constallation and not the other, naturally.

Kay Johansen and Anthony Perkins identified a few interesting ideas in their paper Establishing an Agile Testing Team: Our Four Favorite “Mistakes” [6] that I think is relevant to building a jelled testing team. We also have a paper by Elizabeth Hendriksson on Agile Testing [7] that identifies things you should not do as a tester if you intend to be agile. They are most relevant when considering what role testing should have in a cross-functional team or what your attitude is as a test team. Brian Marrick has made a compilation of articles [8] on Agile Testing, somewhat old ones but still valid as a way to consider when trying to become more effective and build a jelled test team. Marrick has also created the article Classic Testing Mistakes [9] that is valid still and should be considered when trying to identify why your team is not jelled yet. I’ve also written two blog posts about growing test teams: Uncertain team composition [10] and Progress [11]. Both tries to focus on things you should not do in order to get a jelled test team.

Some specific things that I see a jelled test team do, that decrease the testing debt, is:

• Build on each others test ideas. With this I mean that we are triggered on each others test ideas to create new ones until we have no further ideas.
• Easy of changing direction. I often promote the concept that the test team or the team in general need to be flexible, thus the naming of The Flexible Test Team. If the team works well and trust each other it will be easier to accept change.
• Collaborative notetaking. In a team where you want to work together and where you enjoy cooperation it is the teams effort that counts, not the individual. This is also true for taking session notes. This is no best practice (as none exists), but a practise that is good in some situations. That the team have the ability to do this is a bonus.

With an unjelled team, a group of people that do not collaborate as well as they could, thus increasing the testing debt, I instead see:

• Stacking of test ideas. This means that each individual identify test ideas and they are summed up, with duplicate ideas removed. An unjelled team is more inclined to do this than a regular jelled team.
• Double work and duplicate testing. This might be the result of bad leadership and test leading, but based on my experience I think it is more common than in a jelled team.

What things do you consider for a jelled test team or a jelled team that has mixed roles but with testers in it? What have I missed as you see it?

References

[1] Working with the testing debt – part 1 - http://thetesteye.com/blog/2011/05/working-with-the-testing-debt-part-1/

[2] Working with the testing debt – part 2 - http://thetesteye.com/blog/2011/06/working-with-the-testing-debt-part-2/

[3] Peopleware - http://www.amazon.com/Peopleware-Productive-Projects-Teams-Second/dp/0932633439/ref=sr_1_1?ie=UTF8&qid=1310020112&sr=8-1

[4] Flexible testing team - http://thetesteye.com/blog/2008/05/the-flexible-testing-team/

[5] Resource planning in a flexible test team- http://thetesteye.com/blog/2008/05/resource-planning-in-a-flexible-test-team/

[6] Establishing an Agile Testing Team: Our Four Favorite “Mistakes” by Kay Johansen, Anthony Perkins - http://agile2004.agilealliance.org/participate/examples/EstablisingAnAgileTeam.pdf

[7] Agile Testing by Elizabeth Hendriksson - http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf

[8] Agile Testing by Brian Marrick –  http://www.exampler.com/testing-com/agile/

[9] Classic Testing Mistakes by Brian Marrick - http://www.exampler.com/testing-com/writings/classic/mistakes.htm

[10] Growing test teams: Uncertain team composition - http://thetesteye.com/blog/2010/04/growing-test-teams-uncertain-team-composition/

[11] Growing test teams: Progress - http://thetesteye.com/blog/2009/10/growing-test-teams-progress/

We, testers, are we all alike? Looking back over the years I’ve been involved in testing, I would say I have met a whole bunch of different testers. Some shared my passion and fascination for testing, others were not testers they just happened to have the job title. However, I have always appreciated our craft for having such diversity of people. We have testers that have strong domain knowledge, programming skills and business knowledge. All of those are easy to connect the value to have in a test team, they find important information about the technical implementation of the product.

Then we have the other group, the ones who have capabilities in philosophy, cognition, history, economics, art, music and myself a drop out from statistics at university. We are not here because we love code or the coolest technologies, we are here because we love the way people and products interact and misunderstand each other. We are fascinated by the dance between a human and a machine. We find patterns in this dance, we find unintentional behaviors, and the unexpected surprises us. We look beyond constraints of the application, we go where no developer intended to go. By doing this we notice other kinds of things that might be a threat or a problem for the success of the product. Why are we doing this you may ask? It is because the technical solution does not impress us, we do not take particular interest or care in what specific way the code is written. What drives us is the question: will this solve a problem for a user and in which ways can it fail to do so? What we bring to the table is the social and behavioral science aspects of the product combined with the human element. At technology facing companies I sometimes see the above overlooked or under-valued.

But what worries me more is that we, testers, are walking away from my precious diversity. Strong forces favor uniformity among testers and the most obvious one is ISTQB. After going through their program of various certifications everyone knows the same, speaks the same language and uses the same templates. There is no problem replacing one ISTQB Advanced Level certified test manager with another one with same level of certification. The same goes with TMap, as long as you can read the book you know the answer to all testing problems, it’s easy!

But this is nothing new, I and many with me have been fighting both ISTQB and TMap for years. But here comes the kicker, have you heard of this cool new thing called “agile”? It’s a really sweet thing, close to a drug in our craft. It came and it has conquered. Nowadays there are almost no companies who dare say that they are not “agile”, instead they make the most ridiculous variations of what they call “agile”. Don’t get me wrong here, this new movement is quite appealing to me and I appreciate much of what it stands for.

Now getting back on track there is one thing that worries me about this agile thing. That is the view on us testers. Now to be an “agile tester” you basically need to be a developer with preferably some basic knowledge of testing, but it is much cooler if you know Selenium, TDD, ATDD and other strange abbreviations. If a tester according to the agile folks is basically a developer, why do you call them a tester, or wait you don’t! Everyone is a team member, I’m sorry.
Now we are walking from “must have a certification” to “must have programing skills”. Guys, don’t you see what is happening. We are just replacing one uniformity with another, beware of this!

We are running a great risk that all our testers are finding the same information and worse, are blind on the same spots. In testing we do not know what information we will get and we do not upfront know what questions to ask the product (don’t be mislead to believe anything else). We have to approach the testing from various different angles and have several different ears and eyes observing it. If one thing, we should love diversity among testers and we should encourage it.

This post came to my mind when I once again listened to Malcolm Gladwells brilliant TED Talk.

And no we do not all want black coffee!

I saw an interesting interview with Rob Sabourin today http://www.youtube.com/watch?v=HZRXdaN7gkY  (Thanks for the tip, Jon Bach!)
One thing he says in this video is: “… There are a lot of template junkies out there. [Testers are]  filling out templates and not actually testing. That frustrates testers…”

Hey, isn’t this the same thing that happens in strictly scripted testing environments?
When testers are following detailed test scripts they are actually ending up filling out templates. With the same frustration as when filling out templates where you also don’t have to bother use your brain.
The result is the same for scripted testing as with a filled out template. Little mind work has been involved and so the information is very narrow and/or shallow…

It is sometimes said that templates are good because it is prevent someone from doing mistakes and/or making sure that the format is the same all the time. But isn’t a template to a document the same as what a script is to a test? A prescribed recipe on how to complete the task. I.e. it is about management control.

Isn’t it so that you easily can be hit by inattentional blindness as you fill out the template and only worrying about not screwing up the format? As with scripted testing, the focus lies in the format rather in the content. Both in scripted testing and in filling out templates you worry about making sure all things are filled out as expected, not the actual testing or the actual writing of informative text.

And, as with scripted tests, if we could fill out the templates automatically, we wouldn’t need to do them manually.

First of all. I am honored to be invited as a guest writer at the testeye! I will start my contribution by asking for your assistance.

In the next year I have been asked to give a couple of classes for test managers that are fairly new in their role. I have been looking through the content description of a class I am supposed to take over and realize that I have a problem. So much of it seems to be picked from an ISTQB-class and just seem to be cementing the old ways of testing. I refuse to spend half a day talking about eight ways of measuring test which means measuring quality and is based on test case success ratio. And I am not going to dive into standards too much either.

So what am I going to put in a two day class for new test managers? These are some of the more important subjects I want to talk about.

1. Schools of testing: a test manager needs to understand that there are some fundamentally different ways that testing can be done. Since many of the students will be working as consultants they need to be prepared for the many different company approaches they may encounter.
2. Test Strategy: What is the testers’ mission. How do I plan my resources and my tasks. Time estimation. What do I need to know to do a good job. What kind of documentation should we create? When am I done testing?
3. Communication: How do I as test manager communicate with the rest of the project and within my team. This includes giving and receiving feedback, basics of presentation techniques, maybe a bit of Myers-Briggs model
4. Explaining test to non-testers: Why are we not QA, why does it take so much time? Why do developers need to test as well. What is our goal as testers. What responsibilities do we NOT assume.
5. Managing a group of testers: Is it like being a project manager with testing skills? Important management principles: avoiding micro-management, coaching your staff, manager = problem remover, administrative tasks you need to do. Session-based test management
6. Effective bug management: templates, process, content, bug advocacy
7. Career plan for TM: what to read, what to do

So, fellow testers. What would be the three most important things YOU want to see in a test management class? Maybe the three things you wish they had told you before you agreed to your first job as a test manager.

And I have this great idea as well. What if we skilled context-driven testers together created some really good material for a test management class and made that material free to use for everyone? Like the BBST-class.  I really believe in sharing which is why my test design book in Swedish is available for free download at testzonen.se. Maybe it´s time to make the English version downloadable from thetesteye?

Cheers

-Tobbe

I have become allergic to models that are brought from other industries, and put on software testing as a best practice, or something really good.
Software testing is unique, and you might violate important aspects when applying a template that doesn’t match.
It is a big difference between producing 100,000 cars a year, and one piece of new software.

As I believe that software testing is one of the most creative professions, maybe there are other places to look for really good management methodologies?
The Beatles wrote great pop songs, and a lot of them, where they using Kanban?
Did Just-in-Time play a role for the productivity of Thomas Alva Edison?
Was Leonardo da Vinci an early adopter of Kaizen?
Were the New Wave in French Film under Lean management?
Could the String Theory emerge without Six Sigma?

You might argue that these examples are very different from software testing, and that is true.
Therefore we should manage and improve our work in ways that are suitable to what we do, and who we are.
There are no frameworks for passion.

Let’s have a look at an example:
A key element in Lean is “reduce waste”, which seems like a natural and good thing to do.
But this is very problematic in testing, because you don’t know (in advance) what is waste. And what was waste in one situation might be very important in another. It might be reasonable to identify waste in activities that are predictable, but it’s not recommendable in situations with a lot of serendipity.
Of course it can be a good match as well, but I’d prefer saying “I read some about car manufacturing, and realized that parts of our test report aren’t useful to anyone, so we will skip those parts” to “let’s adopt Lean Management to our testing process!”

It’s a good idea to be inspired from other fields, but please don’t take a whole package, and please think carefully if there are things about testing that are necessary to consider, e.g. doctors can not make mistakes, testers should make mistakes.

I’m thinking movie production might be the closest to “normal” software testing projects: it involves a lot of people, a lot of different types of creativity, is complex, and spans over quite some time. What management models are helpful in the film industry?
Does it help if movie requirements involve things like “There must be exactly 3,5 minutes of action scenes including tricycles”?

This is a follow up from previous articles on Growing test teams based on the ideas from Peopleware by Tom DeMarco and Timothy Lister.

Uncertain team composition

If you are newly assigned to be a team leader there is a big chance that you also have a team, but that is not always the case. Just as you want to start organizing the team, steer toward a set of goals, begin promising your stakeholders what you will be able to accomplish… you begin to wonder… why am I sitting alone on this team meeting? Where are they?

The test team will have a harder time growing when …

• you do not know who is in the team
• you do not know how much time in the resource plan each member is allocated to the team
• you do not know how much time in reality each member is allocated to the team
• your team members have other assignments that they spend time on that are not communicated
• your test manager does not see you as a team, but instead a resource pool where all members are to be plucked out at any time
• your team members does not see themselves as part of your team and continue to work on their own agendas
• you have hidden members that should be part of the team, but are safely tucked away in a hidden project

When you are uncertain about your team composition it will block you from going forward. You will be handicaped as a team leader and will most certainly fail, or burn out trying.

Exploratory Testing is not as widely used as it could be, because management doesn’t want it.
Stated reasons for this could be unaccountable, unstructured, sloppy, non-scientific etc, reasons that can be refuted by communication.
But I think the real reason is something Exploratory Testing can’t have: a controlled process.

Management/Companies want to have a plan with dates and costs; they want a test manager to be able to say how many percent of the testing is completed; they incorrectly think that software development is a lot like manufacturing.
Exploratory Testing can’t have this, because the testing will change as new information is uncovered.
Testing might be quick, might take a long, long time, might not be close to complete, might look outside the requirements, and discover so important information that the whole project needs to be re-thinked.
These are not good things for some managers, they want control and precision.
They want to see everything go smoothly and at the promised date we deliver what we said we would deliver.
The focus is on the control, not the result.
That is why managers can prefer to outsource testing to a company that runs the same script over and over until they all pass, than to a company (or ideally, in-house testers) that will change their testing strategies along the way, and leave the project with a bunch of fixed and unfixed bugs.

Exploratory testing is about learning, discovering new things and changing our mind.
And for software testing, which can’t be complete, this is a very good thing.
It enables better products, in a way that can be managed, but not controlled.

This is why it is so hard to sell Exploratory Testing:
First you must sell trust, which is priceless.

Imagine that you run a software consultant shop where you take on projects for customers. The projects cover such areas as new software development; implementations of IT systems; and web site development.

Let’s say that you are about to create a offer for a new project to a customer.

Do you dare to specify the proper amount of hours dedicated to software testing? Or do you feel ashamed of having to test the software before letting the customer lay its hand on it?
Do you just add a couple of hours as a separate post so that it doesn’t look bad if someone asks about “any software testing planned”?
Do you include all the software testing hours needed in the total estimate? Or included in the total per function?

I think that we should treat software testing as any other task that are needed in order to develop functionality so that the hours that are specified per function/requirement/area covers all necessary actions and tasks in order to deliver ready functionality.
If you include such tasks as Design, Interaction Design, Specification, Requirement Analysis, Architecture, Coding, etc, you should also include Software Testing amongst these tasks. And you should be proud of doing Software Testing!

By including software testing in your time estimates, you give yourself a competitive advantage. When your customer selects between several offers and sees that you have included software testing and some of the competitors haven’t, it is a signal to the customer that makes them wonder why the others haven’t got any software testing (or why they haven’t specified any). Your offer might come out as a more expensive one, but since you have specified the difference it becomes obvious that they cannot just compare the price tag.

What are your thoughts on this?

A lot of these ideas come from Peopleware by Tom DeMarco and Timothy Lister. As I see it, they realised it is easier to show things that will stop the growth instead of listing things that will actually create the team. Jelled teams are created when many of the factors have been eliminated that stop us from growing.

What stops growth of a test team? I identify new things almost every day that in one way or another disrupts the team or stops it from growing.

The hunt for test progress!

When we talk about progress it is directly linked to a goal, thus progress towards a certain goal. If the goal is unclear or has been lost, the progress estimation can sometimes shift toward things that was not meant to be.

How do you determine progress then? When are we done testing? If our plan from start is fixed, we might have a defined set of tests that must be run in order to say we are complete. That is, complete with what we thought from the beginning. But the plan changes, no? If that is the case the progress report is ever changing up or down.  Is it perhaps not really that interesting to focus our time on bulletproof progress estimations? We stop testing when time runs out or when someone says stop?

I think the test team have a harder time growing when …

• the ability to show test progress becomes more important than the actual testing done or the information produced from it.
• we think it is important getting more green than red in the pie chart or bar chart.
• we avoid testing areas that might result in bugs because that might disrupt the expected weekly progress.
• it is more important doing a test that show progress than doing a test that might actually find bugs.
• we avoid helping developers fix the bugs found because we need to show test progress.

Too much focus on progress will generate bad energy in the test group and therefore slow us down, as I see it.

From a managerial perspective without knowing too much about testing, your sole experience comes from the scripted test environment…

What does Scripted Testing include?

• Control over what is to be tested, in the sense that you have a clear coverage of test cases on certain areas. Reports where you can see exactly how many test cases are left to do before you are finished.
• Hierarchy among testers, with different roles playing their part in the system. An excellent way to have career moves. Test leads who do the thinking and the others to execute.
• Scalability in personel by easily bringing in new people who can execute the test scripts that the test leads have written. Getting new resources in the middle of a project is easy since you can take almost anyone.
• Test Management Software. If you want your testers to have the best tools, there are tools that cost several millions. They will generate excellent reports, they can automate so that the boring repeatable tests do not have to be rerun manually.
• Education. There are lots of certification to be had and many firms who can teach you the essentials.

What does Exploratory Testing (ET) include?

• Flatness in organisation. The testers make tests as they go along. They do not need the traditional test leads. It is uncertain what kind of career exits you have with this kind of focus.
• Chaos. You have no idea on how you are going to test. You have not planned everything out in detail before you start testing. You cannot report exactly how long time you need and as much time as possible is not good enough.
• No scaleability. You only want testers. Not anyone can be a testers. It is hard to just get anyone to help out since you cannot use just anyone from the organisation, they must know the basic skills to help out.
• No real education. You cannot get certified. None of the major consultants teach this. There are courses, but they are new and few teach it.
• No real Test Management Software. The major software vendors have no tools for this. The software is seldome made by commercial vendors.

What do your managers think about this? What sales pitch do ET have for management? I’ve twisted what ET can include to bring out the worst of it, but I think that an inexperience test manager with his roots in the scripted test environment has at least part of this view. Cem Kaner has a good presentation called A Tutorial in Exploratory Testing. That presentation do sell ET really well. If you are going to try to get your organisation to start using ET and intend to do some propaganda for your test manager Kaners presentation is a good start.