First of all. I am honored to be invited as a guest writer at the testeye! I will start my contribution by asking for your assistance.

In the next year I have been asked to give a couple of classes for test managers that are fairly new in their role. I have been looking through the content description of a class I am supposed to take over and realize that I have a problem. So much of it seems to be picked from an ISTQB-class and just seem to be cementing the old ways of testing. I refuse to spend half a day talking about eight ways of measuring test which means measuring quality and is based on test case success ratio. And I am not going to dive into standards too much either.

So what am I going to put in a two day class for new test managers? These are some of the more important subjects I want to talk about.

1. Schools of testing: a test manager needs to understand that there are some fundamentally different ways that testing can be done. Since many of the students will be working as consultants they need to be prepared for the many different company approaches they may encounter.
2. Test Strategy: What is the testers’ mission. How do I plan my resources and my tasks. Time estimation. What do I need to know to do a good job. What kind of documentation should we create? When am I done testing?
3. Communication: How do I as test manager communicate with the rest of the project and within my team. This includes giving and receiving feedback, basics of presentation techniques, maybe a bit of Myers-Briggs model
4. Explaining test to non-testers: Why are we not QA, why does it take so much time? Why do developers need to test as well. What is our goal as testers. What responsibilities do we NOT assume.
5. Managing a group of testers: Is it like being a project manager with testing skills? Important management principles: avoiding micro-management, coaching your staff, manager = problem remover, administrative tasks you need to do. Session-based test management
6. Effective bug management: templates, process, content, bug advocacy
7. Career plan for TM: what to read, what to do

So, fellow testers. What would be the three most important things YOU want to see in a test management class? Maybe the three things you wish they had told you before you agreed to your first job as a test manager.

And I have this great idea as well. What if we skilled context-driven testers together created some really good material for a test management class and made that material free to use for everyone? Like the BBST-class.  I really believe in sharing which is why my test design book in Swedish is available for free download at testzonen.se. Maybe it´s time to make the English version downloadable from thetesteye?

Cheers

-Tobbe

I have become allergic to models that are brought from other industries, and put on software testing as a best practice, or something really good.
Software testing is unique, and you might violate important aspects when applying a template that doesn’t match.
It is a big difference between producing 100,000 cars a year, and one piece of new software.

As I believe that software testing is one of the most creative professions, maybe there are other places to look for really good management methodologies?
The Beatles wrote great pop songs, and a lot of them, where they using Kanban?
Did Just-in-Time play a role for the productivity of Thomas Alva Edison?
Was Leonardo da Vinci an early adopter of Kaizen?
Were the New Wave in French Film under Lean management?
Could the String Theory emerge without Six Sigma?

You might argue that these examples are very different from software testing, and that is true.
Therefore we should manage and improve our work in ways that are suitable to what we do, and who we are.
There are no frameworks for passion.

Let’s have a look at an example:
A key element in Lean is “reduce waste”, which seems like a natural and good thing to do.
But this is very problematic in testing, because you don’t know (in advance) what is waste. And what was waste in one situation might be very important in another. It might be reasonable to identify waste in activities that are predictable, but it’s not recommendable in situations with a lot of serendipity.
Of course it can be a good match as well, but I’d prefer saying “I read some about car manufacturing, and realized that parts of our test report aren’t useful to anyone, so we will skip those parts” to “let’s adopt Lean Management to our testing process!”

It’s a good idea to be inspired from other fields, but please don’t take a whole package, and please think carefully if there are things about testing that are necessary to consider, e.g. doctors can not make mistakes, testers should make mistakes.

I’m thinking movie production might be the closest to “normal” software testing projects: it involves a lot of people, a lot of different types of creativity, is complex, and spans over quite some time. What management models are helpful in the film industry?
Does it help if movie requirements involve things like “There must be exactly 3,5 minutes of action scenes including tricycles”?

This is a follow up from previous articles on Growing test teams based on the ideas from Peopleware by Tom DeMarco and Timothy Lister.

Uncertain team composition

If you are newly assigned to be a team leader there is a big chance that you also have a team, but that is not always the case. Just as you want to start organizing the team, steer toward a set of goals, begin promising your stakeholders what you will be able to accomplish… you begin to wonder… why am I sitting alone on this team meeting? Where are they?

The test team will have a harder time growing when …

• you do not know who is in the team
• you do not know how much time in the resource plan each member is allocated to the team
• you do not know how much time in reality each member is allocated to the team
• your team members have other assignments that they spend time on that are not communicated
• your test manager does not see you as a team, but instead a resource pool where all members are to be plucked out at any time
• your team members does not see themselves as part of your team and continue to work on their own agendas
• you have hidden members that should be part of the team, but are safely tucked away in a hidden project

When you are uncertain about your team composition it will block you from going forward. You will be handicaped as a team leader and will most certainly fail, or burn out trying.

Exploratory Testing is not as widely used as it could be, because management doesn’t want it.
Stated reasons for this could be unaccountable, unstructured, sloppy, non-scientific etc, reasons that can be refuted by communication.
But I think the real reason is something Exploratory Testing can’t have: a controlled process.

Management/Companies want to have a plan with dates and costs; they want a test manager to be able to say how many percent of the testing is completed; they incorrectly think that software development is a lot like manufacturing.
Exploratory Testing can’t have this, because the testing will change as new information is uncovered.
Testing might be quick, might take a long, long time, might not be close to complete, might look outside the requirements, and discover so important information that the whole project needs to be re-thinked.
These are not good things for some managers, they want control and precision.
They want to see everything go smoothly and at the promised date we deliver what we said we would deliver.
The focus is on the control, not the result.
That is why managers can prefer to outsource testing to a company that runs the same script over and over until they all pass, than to a company (or ideally, in-house testers) that will change their testing strategies along the way, and leave the project with a bunch of fixed and unfixed bugs.

Exploratory testing is about learning, discovering new things and changing our mind.
And for software testing, which can’t be complete, this is a very good thing.
It enables better products, in a way that can be managed, but not controlled.

This is why it is so hard to sell Exploratory Testing:
First you must sell trust, which is priceless.

Imagine that you run a software consultant shop where you take on projects for customers. The projects cover such areas as new software development; implementations of IT systems; and web site development.

Let’s say that you are about to create a offer for a new project to a customer.

Do you dare to specify the proper amount of hours dedicated to software testing? Or do you feel ashamed of having to test the software before letting the customer lay its hand on it?
Do you just add a couple of hours as a separate post so that it doesn’t look bad if someone asks about “any software testing planned”?
Do you include all the software testing hours needed in the total estimate? Or included in the total per function?

I think that we should treat software testing as any other task that are needed in order to develop functionality so that the hours that are specified per function/requirement/area covers all necessary actions and tasks in order to deliver ready functionality.
If you include such tasks as Design, Interaction Design, Specification, Requirement Analysis, Architecture, Coding, etc, you should also include Software Testing amongst these tasks. And you should be proud of doing Software Testing!

By including software testing in your time estimates, you give yourself a competitive advantage. When your customer selects between several offers and sees that you have included software testing and some of the competitors haven’t, it is a signal to the customer that makes them wonder why the others haven’t got any software testing (or why they haven’t specified any). Your offer might come out as a more expensive one, but since you have specified the difference it becomes obvious that they cannot just compare the price tag.

What are your thoughts on this?

A lot of these ideas come from Peopleware by Tom DeMarco and Timothy Lister. As I see it, they realised it is easier to show things that will stop the growth instead of listing things that will actually create the team. Jelled teams are created when many of the factors have been eliminated that stop us from growing.

What stops growth of a test team? I identify new things almost every day that in one way or another disrupts the team or stops it from growing.

The hunt for test progress!

When we talk about progress it is directly linked to a goal, thus progress towards a certain goal. If the goal is unclear or has been lost, the progress estimation can sometimes shift toward things that was not meant to be.

How do you determine progress then? When are we done testing? If our plan from start is fixed, we might have a defined set of tests that must be run in order to say we are complete. That is, complete with what we thought from the beginning. But the plan changes, no? If that is the case the progress report is ever changing up or down.  Is it perhaps not really that interesting to focus our time on bulletproof progress estimations? We stop testing when time runs out or when someone says stop?

I think the test team have a harder time growing when …

• the ability to show test progress becomes more important than the actual testing done or the information produced from it.
• we think it is important getting more green than red in the pie chart or bar chart.
• we avoid testing areas that might result in bugs because that might disrupt the expected weekly progress.
• it is more important doing a test that show progress than doing a test that might actually find bugs.
• we avoid helping developers fix the bugs found because we need to show test progress.

Too much focus on progress will generate bad energy in the test group and therefore slow us down, as I see it.

From a managerial perspective without knowing too much about testing, your sole experience comes from the scripted test environment…

What does Scripted Testing include?

• Control over what is to be tested, in the sense that you have a clear coverage of test cases on certain areas. Reports where you can see exactly how many test cases are left to do before you are finished.
• Hierarchy among testers, with different roles playing their part in the system. An excellent way to have career moves. Test leads who do the thinking and the others to execute.
• Scalability in personel by easily bringing in new people who can execute the test scripts that the test leads have written. Getting new resources in the middle of a project is easy since you can take almost anyone.
• Test Management Software. If you want your testers to have the best tools, there are tools that cost several millions. They will generate excellent reports, they can automate so that the boring repeatable tests do not have to be rerun manually.
• Education. There are lots of certification to be had and many firms who can teach you the essentials.

What does Exploratory Testing (ET) include?

• Flatness in organisation. The testers make tests as they go along. They do not need the traditional test leads. It is uncertain what kind of career exits you have with this kind of focus.
• Chaos. You have no idea on how you are going to test. You have not planned everything out in detail before you start testing. You cannot report exactly how long time you need and as much time as possible is not good enough.
• No scaleability. You only want testers. Not anyone can be a testers. It is hard to just get anyone to help out since you cannot use just anyone from the organisation, they must know the basic skills to help out.
• No real education. You cannot get certified. None of the major consultants teach this. There are courses, but they are new and few teach it.
• No real Test Management Software. The major software vendors have no tools for this. The software is seldome made by commercial vendors.

What do your managers think about this? What sales pitch do ET have for management? I’ve twisted what ET can include to bring out the worst of it, but I think that an inexperience test manager with his roots in the scripted test environment has at least part of this view. Cem Kaner has a good presentation called A Tutorial in Exploratory Testing. That presentation do sell ET really well. If you are going to try to get your organisation to start using ET and intend to do some propaganda for your test manager Kaners presentation is a good start.

What is essence of the discussion on release team/showstopper meetings?

I am assuming that there is a meeting. I’ve been to many different kinds of showstopper meetings and most companies handle them differently. One important item on the agenda for the meeting is usually the bugs that are found late in the project, thus at the end phase where they might be considered showstoppers.

When these bugs are discussed if they are to be fixed or not, I’ve seen cases when the argument is a bit vague. The actual topic is sometimes not brought to the surface, instead it is someone who “feels” that this should be fixed or it should not. What is it that they feel? Articulate those feelings and try to be clear on what it is you see. Why should it be fixed or why shouldn’t it?

In these cases I think it is best to be clear that we talk about cost, at least in most cases, in one sense or another, thus the cost of not fixing it or the cost of fixing it. Some costs will affect support, others will affect marketing and sales and some will affect development itself. I can recommend trying to get everyone attending the meeting to focus their thoughts on what the bugs will cost you. Is a few days slip perhaps worth it? In some cases the date for the release is extremely important, this is most often the case for bigger organisations. Instead it is easier to make some changes after the release decision and make a few patches afterwards. The actual release will not reach the customer the same day anyhow. Either way it is cost you discuss and you most often decide based upon it.

Cem Kaner has written an article on this that I’ve used many times when setting up these meetings. You can find the article here.

Who makes the decision that the product is ready for release?

There are many different cases of this situation, here are a few examples.

In some companies it is the QA department that makes this decision. This means that it is QA that takes the risk for the release. It also means that if something happens after the release it is QA that is to blame for making a “faulty” decision. I’ve been in situation when management said “Why did they not test that?” or “How could they miss that area? It was so obvious.”. If this happens QA will be worried more about what the rest of the company will say and will in some situations say that the product is not ready for release just to avoid possible blame. Does QA have the authority to get longer time to test, are they able to get more bugs fixed, what authority does QA have?

In some cases it is the project itself that makes the decision where the whole group (developers, testers, project manager and so on) decides. This means that the project and most often the project manager him/herself takes the risk for the release. So, the blame will fall on the project and its members. Is the project able to get resources, time etc if need be? In most cases no.

In other cases it is management or a release team consisting of managers from all departments that makes the decision. This means that someone outside of the project makes the decision for the release. It also means the blame will fall on this team, not the project itself. It is also common that this team can affect resources, time etc for the project.

We also have the absurd case where no one makes the decision. The release just happens and no formal decision was made. In this case it is more of a bad process issue that this happens. Management is responsible unless the project manager has the formal responsibility in these situations.

If you are unsure who makes this decision you really need to discuss this with management. Someone must take the risk and responsibility for releasing the product. I recommend that it is management or a release team that makes this decision, since they have the authority to make changes. This naturally leads into the role of QA, are they Quality Assurance or Quality Assistance (as Cem Kaner so nicely put it)? I think that if testers can avoid making this decision it will make their life a lot easier.

The indispensible worker is the one who always saves the day by touching the program by giving some magic input in order for it to work. He will quickly get the work done and make the customer happy. Upper management will see him as the perfect employee, but many co-workers will think differently.

In the indispensible workers back-water you will notice half-finished tasks, programs that need the magic input in order to work, documentation that usually are not the latest version or just not there, hidden tasks that was performed once a week manually to avoid the weekly disaster and so on.

In my experience, if someone has become indispensible, where the company cannot work without them being there, it will often lead to disaster. This can easily be seen if someone is sick, on vacation or if they leave the company, when certain things start to break down or that the team stands still waiting for the magic touch.

As a co-worker you avoid these traps by requiring documentation enough for someone else to perform the task or that you have at least a backup for the critical tasks. The hardest thing is getting upper management to see that it is not often good to focus on the short term solutions where you save a small bit of time in the moment, but where you have to pay for it tenfold in the future.