A Factory of Skilled Testers Rikard Edgren
I do not see myself as a member of any of the Schools of Testing, and I have ethical problems with labelling other people than yourself.
However, I see the schools as a fruitful tool for enhancing your understanding of views on testing.
So please join me in the following thought experiment.
The following is not a perfect match of my personal opinions, it is fiction. I try to be a bit funny and serious at the same time, and I know this is easily misunderstood.
Suppose I run a consultency firm specialized in testing.
We take any kind of testing job (we can also do checking only if requested) and we are very successful; we provide a lot of valuable information to our clients.
The key to our success comes from this recipe:
1. A lot of exploratory testing
ET gives better results, and by explicitly giving testers responsibility and freedom for their activities, they stay motivated and get better over time.
It happens that clients question this approach, but usually it ís sufficient to point at the many Internet resources, and say that they have to be modern, this is the greatest super-good practice right now.
2. Training scheme
We hire people that are fast learners, curious and ambitious. All must take AST courses Foundation, Bug Advocacy, Test Design (the bug reporting is essential, we impress our clients early on, and is the reason we can offer a Money-Back-Guarantee.)
Then everyone must go Rapid Software Testing with Bach or Bolton, which inspires and give breadth to the thinking.
Finally they take Foundation and Advanced ISTQB certification (this is last to avoid problems at previous classes.)
We are no fans of ISTQB, but it is good to know what others know, and we eliminate the risk of losing a deal on a (ridicilous) client requirement.
We also do continuous training on collaboration and feedback, after all, people are the most important part of any context.
3. SBTM
We manage the testing efforts by three sessions a day, this squeezes out maximum from the testers without wearing them out.
We log all the proposed statistics so we can show numbers and progress.
We always add 24% to originally planned sessions, and make sure this cover unexpected things.
Planning and other processes are run context-wise, we use whatever the client is using.
4. Standards
For all jobs, we walk the client through thetesteye’s Quality Characteristics, and a secret company standard for Multiple Information Sources.
The client decides what’s most important in dialogue with us. (We of course also do at least one test per requirement.)
Testers use HICCUPPS(F), CRUCSPIC STMP, CIDTESTD and SFDPOT in their daily work, we know it’s not perfect, but easy to remember.
5. Reasonable Resource Allocation
In a typical project we want to have 1 tester per 3 developers. We tell clients this is our standard where we know we can do a good job. We can do higher or lower ratio, but clearly communicate which expectations this should give.
This makes it easy for clients to measure cost; it has a direct relationship with no. of developers, development time, and start of test effort (the sooner the better, but clients decide.)
Since we have a common background and practices, it is easy to replace our factory workers if needed. But we always keep 1 person from the old staffing, so crucial knowledge can be transferred.
Scaling up or down is not a problem, we have a pool of skilled resources, that grows over time as we hire more talents.
I feel we have a highly manageable process, that repeatedly bring good results. Sure, all projects are unique, but this scheme has proven to be very good, so far.
So what do you think dear readers, would this be a good approach?
Is it Factory, Context-Driven, or something else?
I need to know more about what you really value in order to give an opinion.
If you have to prioritize what is most important in your approach, what do you really care about?
If you are forced to create a slim version of your approach, what would be most important?
The schools of software testing are not different approaches, they are more of meta approaches to approaches.
To avoid confusion about “approach” I can rephrase the last question:
Do you think this way of running a business aligns with the Context-Driven School of Testing, or the Factory/Standard School of Testing, or both, or something else?
These are our hypothetical, most important things, in priority order:
1. Make money
2. Satisfy clients (e.g. if they want a metric they will get it, along with accompanying context)
3. Communicate important information (1. bugs they want to fix, 2. risks, 3. enhancements)
I think it is slimmed, and see no reason to trim further.
Without you answering my second question, I’ll put my money on that you (as team leader?) are context-driven since in all ingredients in your recipe value people and people’s skill in software testing.
If you had assured your “process” by creating templates, promoting standards and implemented best practices that would enable anybody to test in “your way”, you would probably have been a factory guy.
It is how you think of software testing, not how much you have methodized.
To me it looks like Factory/Standard school since the focus is on using the same process everywhere, on all projects. Many of the parts of the recipe describe approaches and techniques that are being suggested and mastered by members of the context-driven crowd, as being appropriate in certain contexts. The fact that the recipe is context-oblivious (or maybe context-imperial?) makes me think that your business does not align with the context-driven school, since the number one and most importand principle of context-driven testing is: “The value of any practice depends on its context.”
@Patrik: I think the most common misunderstanding about context-driven testing is that the most important thing is the process and that context-driven testers should always adopt to the current process because that is to be driven by the context (according to this thought).
But the most important aspect of context-driven testing is people working together. And sometimes you have to adopt to the current process, but that does not mean to accept stupid things.
Actually I don’t see much of a process in Rikard’s recipe (except for the SBTM ingredient perhaps), I see rules of thumb on what to think of when working in a project. All ingredients, more or less, are dependent on having independent, intelligent testers.
Thanks for forcing me to elaborate this made-up scenario.
Henrik, people IS the process.
We have some standard ways to get good information to start with; we have some tools that our skilled testers use (yes, skills are the core for the testing content); and we deliver good information about the product (if customers don’t have report templates, we use low-tech testing dashboard.)
I don’t want other companies to test our way (could reduce profit), but we need one way in order to train people fast, to use a common language, and grow together.
Patrik, we don’t advocate this recipe for all contexts, we use it for all projects WE are involved in. Even when it isn’t perfect, we believe the speed of familiarity and ability to replace workers is beneficial.
We love standards, but only those that are really good. We use a handful, and besides our own mentioned, we are working on a kit for identifying and enabling semi-automated testing.
Hi Rikard,
After reading your nice post a question poped up in my mind. Based on my understanding the 5 steps you have mentioned are quit good to go through for being a good tester. But do you arrange “internal discussion sessions” being another source of learning? If yes, then I would be happy to know how you do that.
Since this is a fantasy story, I have to make up an answer to your question Khurram.
Our top priority is results, and always achieving good results.
We believe this is best accomplished by motivated resources that continually learn and improve.
So internal discussions is an important factor.
1. The most common happens in the daily work at customer site, during session debriefs, lunch and the occasional beer after work.
2. All testers are encouraged to contact each other to help out with tricky problems, we try to have some slack for niche experts that tend to get a lot of questions. We spend 90% of our work time at customers, the remaining is for learning. The learning can be done in any way, as long as they can show they get better.
A typical example is small workshops, organized by our testers, sometimes attended by passionate testers from other companies.
3. Twice a year we organize an internal 1-day conference, with abstracts, papers and competitions.
The best material is either sent to conferences, or kept secret, if it would give away too much to our competitors.
As I said, this is not a true story, so feel free to add learning suggestions for a testing factory fuelled by people.
There are two kinds of people: those who believe in labelling and those who don’t.
In your context the test group has perhaps different view points on testing, but the majority is aligned? Different managers have different views on what testing is and what it should do, just like schools of testing? After your company got bought I guess your context has changed? Different budgets, agendas, priorities etc? Have you adapted your process to these new contexts or have you used the same process all the time? Even when half the test group was changed? Does each project manager work the same as the next and you are able to keep the same test process for all? Do you have the same stakeholders for all projects?
If you satisfy your client, there is a big chance that you make money. Perhaps change it to give the client what they need, not what they necessarily want.
Any smooth process can be seen as a pipeline or factory. It does not really mean that it is a factory.
Well there certainly are a lot of context-droven elements in there. However some of your statements raises questions.
1. ET always, on every project? Regardless of what Mission you are given and what circumstances there are? I am sceptical to that approach. Not very context-driven in my eyes, i recall many times where ET has not been the best fit.
2. Training: also advanced level: is that considered important if how you work and think in large parts contradict what is taught? I am not aware of anyone asking for advanced level since the ones that ask have a hard time to know the difference from foundation level.
3, SBTM always and always three sessions per day? Also for overall planning, testdesign, requirements analysis? I don´t do that today but it is an interesting idea to use SBTM even more… Sessions vary in length depending on size and status of what is tested etc
5 Resource allocation: this is so hard to estimate. It depends on so many factors that are out of my control as a tester. Are the developers experienced, in the area they are workingin this project? is there configuration management, do they use continuous integration, code checks? Is there a project manager, that actually manges the project? Any good requirements to start with and stake holders to discuss with? Is the organisation mature when it comes to buid software? Any range from 1-10 to 2-1 are reasonable numbers that I have found during interviews with testers from all over the.. Ok, mostly Sweden and England. And then the PM asked – Yeah I can see it is hard to tell, but just give me a number and I usually do…
We always do Exploratory Testing, as in focus on learning; freedom and responsibility for the individual tester.
But as in all modern factories, we adapt to the changing context; we change and improve when we can. We are context-aware and context-sensitive, but I see context-driven as much stronger than that.
When I started this fictitious company I wanted to hold on to Context-Driven and Factory School as long as possible (in the spirit of Bohr, who thought you could reach higher if keeping opposites together.)
Now I think it is a Factory paradigm (control and predictive costs), but with super-good practices, developed by context-driven testers.
Even if it is totally made up, I think it is one of the strongest documented Factory School examples.
Attack this rather than the dying view that testing can be performed by anyone, if templates and standards are used.