Last week, me and Martin won the competition “The EuroSTAR Test Lab Apprentices”!
Read more at: http://www.eurostarconferences.com/delegates/the-test-lab-apprentice.aspx

See you in the Test Lab in Copenhagen!

Cheers,
Henrik & Martin

It’s a truth that we can’t find all bugs, but is it really a truth that we can’t find all important bugs?
And it’s a cliche when used as answer to the (sincere) “why didn’t you find that bug?” question.
Testers are paid to find important information about what they are testing, and included are the big bugs, so you can fix them before the software is used in production.

Besides some good things about forthcoming test strategies, I hope you also can say: “if you have read the test plan and status reports it shouldn’t be a surprise bugs like this got away.” Well-informed risk taking is part of our business.
However, if there is a test that seems important, and would take a couple of minutes to perform, this argument is not valid. Testers (in a serious project) should try to perform all important tests that can be performed fast.

There are testing lessons to be learned if the answer is:
* we didn’t think about that scenario
* it wasn’t included in the requirements
* someone told us not to test that
* we had no idea that the customers where running that kind of configuration
* we found it, but didn’t report it well enough
* we didn’t put the bug there, and they hid it awfully well
* this type of issue can’t be found with automated tests

And there is the occasional invalid question, e.g. when the bug was reported in a good way, or when there was no testing time alotted for a late change.

Bottom line is: take the question seriously, and try to improve your testing to avoid this and other important issues (next time it will probably be another kind of issue…)

This is a follow up from previous articles on Growing test teams based on the ideas from Peopleware by Tom DeMarco and Timothy Lister.

Uncertain team composition

If you are newly assigned to be a team leader there is a big chance that you also have a team, but that is not always the case. Just as you want to start organizing the team, steer toward a set of goals, begin promising your stakeholders what you will be able to accomplish… you begin to wonder… why am I sitting alone on this team meeting? Where are they?

The test team will have a harder time growing when …

• you do not know who is in the team
• you do not know how much time in the resource plan each member is allocated to the team
• you do not know how much time in reality each member is allocated to the team
• your team members have other assignments that they spend time on that are not communicated
• your test manager does not see you as a team, but instead a resource pool where all members are to be plucked out at any time
• your team members does not see themselves as part of your team and continue to work on their own agendas
• you have hidden members that should be part of the team, but are safely tucked away in a hidden project

When you are uncertain about your team composition it will block you from going forward. You will be handicaped as a team leader and will most certainly fail, or burn out trying.

This is an idea you see and hear now and then. It comes in different shapes, ranging from testers needing to have an independent manager, to testers being best if physically separated from developers, or even outsourced, or crowdsourced.

Cem Kaner writes in The Ongoing Revolution in Software Testing that this notion primarily is a “fear of bias”, and he is right, as always.
The rationale for independent testing boils downs to testing not being influenced by the development. Of course the influence can sometimes be negative, but the positive influence is more common, and more important.

There are three important reasons why I wouldn’t want to work separated from development:
* More and better information (both ways)
* More ownership and motivation
* Working together with different knowledge and focus gives a holistic view, and a better end result.

But since this separation can take so many forms, there is a need for some clarifications.
* If the testing team belong to a separate group hierarchically, but in all positive ways can influence and be influenced by other groups, I see no problem with it.
* If outsourcing includes developers, testers, documenters, and maybe even product management, I see no problem.
* it is OK if parts of the test effort are performed elsewhere, in order to get different views and approaches; Beta testing is an excellent example.
* if the ambition isn’t higher than doing exactly what is specified in detailed requirements, separation might even make that easier.
* Regardless of setup, if there is a mentality of minding your own business only, I don’t think it is a good setup.

I’m not sure if it is the physical or the mental separation that undermines the ability to get involved in each other’s work, which I see as a good thing, not because of lack of trust, but because we wanna make sure that the end result is a great product.

A lot of these ideas come from Peopleware by Tom DeMarco and Timothy Lister. As I see it, they realised it is easier to show things that will stop the growth instead of listing things that will actually create the team. Jelled teams are created when many of the factors have been eliminated that stop us from growing.

What stops growth of a test team? I identify new things almost every day that in one way or another disrupts the team or stops it from growing.

The hunt for test progress!

When we talk about progress it is directly linked to a goal, thus progress towards a certain goal. If the goal is unclear or has been lost, the progress estimation can sometimes shift toward things that was not meant to be.

How do you determine progress then? When are we done testing? If our plan from start is fixed, we might have a defined set of tests that must be run in order to say we are complete. That is, complete with what we thought from the beginning. But the plan changes, no? If that is the case the progress report is ever changing up or down.  Is it perhaps not really that interesting to focus our time on bulletproof progress estimations? We stop testing when time runs out or when someone says stop?

I think the test team have a harder time growing when …

• the ability to show test progress becomes more important than the actual testing done or the information produced from it.
• we think it is important getting more green than red in the pie chart or bar chart.
• we avoid testing areas that might result in bugs because that might disrupt the expected weekly progress.
• it is more important doing a test that show progress than doing a test that might actually find bugs.
• we avoid helping developers fix the bugs found because we need to show test progress.

Too much focus on progress will generate bad energy in the test group and therefore slow us down, as I see it.

This was originally a response to Rikard’s post “Multi-Dimensional Software Testing”, but here I have developed my thoughts a bit.

As I see it, there are more or less obvious stakeholders and stakeholders that might be more or less hidden. A “customer” might be such an obvious stakeholder. It might then just be a matter of recognizing which of these “customers” that matter and take those expectations into consideration when selecting test activities.

The project manager is another example of an obvious stakeholder.

And a product owner would naturally be considered an obvious stakeholder.

But how about the hidden stakeholders; and what are their interests in the project/team?

Maybe the members of the test team would have interests in the team’s work that would affect the testing activities (i.e. their own work)? Someone in the team might be dependent on other peoples work. Some people in the team might lack important skills which would affect the chosen test activities. Are there some people that only can work part time and thereby affect which times meetings are held in the team?

The members of the test team are them self stakeholders to the project and might have important interests. This might have impact on the testability, documentation effort, release schedules, communication, etc. Project members other than the testers are also stakeholders to the project and they might have their specific interests.

Business managers might have much to say about the testing activities; and they might especially be interested in the cost of the testing activities. They might have influence on hardware or software that would be needed and thereby affect the testing activities.

Other test groups that work in parallel might be interested in the outcome; and especially to make sure that your team doesn’t over-deliver and make their team look bad. And they might be interested in what type of cooperation that could take place in order to be more effective. They might also have an interest in what type of reporting to do in order to compile a common test report. Other test groups might also be interested in creating a common vocabulary to use in the test scripts.

The maintenance team that would receive the product after it has been released might be interested in the outcome. And especially they might be interested in the format of the test product i.e. documentation, test scripts, procedures, etc. They might also be interested in good practises that have come up during the project; and if these good practises were designed to work for the maintenance team, they have indeed already affected the testing activities in your team.

Certainly there are more hidden stakeholders than those above to take into consideration when selecting test activities or test strategy to focus on. But my goal is to illustrate the problem with a couple of hidden stakeholders in order to make you think about your situation and possible stakeholders that might matter for you.

Note:
Many of these stakeholders are already taken into consideration on a daily basis, but I believe that they are not considered as stakeholders. Maybe you would discover more important aspects of each stakeholder if you tried to analyse each of them more thoroughly.

E.g., try to list all people or roles that matter to the team and start to analyse what their interests are and how they might affect your team/project.

And you should not be surprised that some of these were hidden to you at first, and when their interests are revealed you might discover that they would have a large impact on your testing activities.