Another certification, another scam? Martin Jansson
In a recent blog post [1] on Informator-blog, Magnus C Ohlson articulates the idea of pilots having the flight hours but not the actual flight certificate. He insinuates that the artifacts from requirements and testing would be better if people were certified, if I understand him correctly. Furthermore he explains that testers need education, knowledge and experience which I agree with fully. But he promotes that to ensure that someone has the right competence is by having a certificate such as ISTQB or REQB. According to him, you would then know if he or she is a skilled tester.
Here are a few personal experiences that relate to certification, as I see it:
When I did the military I got the opportunity to train for driving a trailer. We did a two-day theoretical exam, and then they let us out on the roads. After studying text books succeeding with a little exam they determined we were ready to start driving. I had no problem passing the exam, but I was a terrible driver. I had only a few months prior to this been able to take the driving license for driving a car. They let out a large amount of trucks, trailers and other vehicles into the small town of Boden. I managed to get around town without getting killed, but all in all the lot of us managed to demolish several traffic lights, some road signs, some trucks and a few light poles. The cost of this adventure was huge. I guess we learned a lot, but the idea that we were ready based on reading a textbook was a bit strange. Months later, after a few hundred miles of driving we started to get good at it, but the 2-3 day theory in the beginning was almost meaningless then. I see the test and requirement certification as having the same symptom, we do not get much value of something so intensive and theory focused.
In the early 2000 a doctor told me I had diaphragmatic hernia. At the same place, a surgeon went through with me the procedure how to get this fixed. She told me that I need to change the way I lived my life, but that it was almost pointless to try according to her. If I were to go through with a surgery, they would have needed to open me up, lift my chest and move around several organs. The whole procedure was very dangerous and the risk for death was high. As part of the surgeon’s analysis, she performed a gastroscopy. This procedure took between 10 and 20 minutes. It was hard for me to determine, after a while I understood what torture was all about. Some years later I met a doctor who hinted that there were other experts in the field that could help me out. I ended up with probably one of the best doctors in the field, who told me a whole different story. He recognized the name of the original surgeon who had given me the first statement. According to him, the original surgeon was not able to perform a gastroscopy in school as well. This new doctor performed a gastroscopy as well, which took 1½ to 2 minutes which is the time it should take, according to him. He then said that an operation was possible with minimum risk to my life. The last 10-20 years they had been using keyhole surgery, the methods the previous surgeon talked about are not used any more. Even if the first surgeon got her license, she was apparently not interested in keeping herself up-to-date with the changes in her craft. Comparing this with testing and requirement certificates, even if you are certified does not mean that you keep up with changes to the craft and that it is applicable.
A friend of mine was troubled about being forced to become certified. He was basically forced to certify himself as a project manager. The curriculum of the certification was solely based on waterfall methods. My friend has been working with agile projects for a while and has left the world of waterfall behind him. He does not see the point in keeping old facts up to date but instead try to learn new things. The person who wrote the curriculum for the certification did not know anything about agile. Comparing this with testing and requirement certification, we will have single point or points of failure in keeping up to speed with changes and improvements in the craft. If the creators of a syllabus are not top notch in the craft, then everyone who needs to certify themselves must lower themselves to the creators level. Even if they are top notch, they cannot compete with the wisdom of crowds, where the crowd is the joint knowledge of the test community.
Another friend of mine took an intensive course to drive a car. She travelled to a small town in northern Sweden to take a two-week course. She took it easily and got back home to Gothenburg. But the driving in the small town was a bit too easy with no highways, very few cars and an environment that did not match what was in her home town. She got the license and got home. When entering the traffic in Gothenburg she became too scared and dared not drive anymore. If we compare this with testing and requirement certification, an intensive course that results in a certificate might only mean that you have paid money to get something that is not valid in your actual context, in your project or hometown.
In a previous article called Testers Greatest Nemesis [2], I wrote about the intent that Dorothy Graham had when they initiated ISTQB. According to her blog posts the original intent was lost over the years. In Sweden there is a movement that has started certification of requirement experts called REQB. I see consultancies are offering the training and certification of it. I do hope this is just not a scam to make money, where recruiters will start filtering out those with 10+ years of experience of requirement handling to those with the name REQB in their CV.
Conclusion
I do not believe the complexity of organizations, projects and business is displayed in a generic multiple choice questionnaire that someone with no knowledge could accidently pass. I hope the movement in REQB learns from previous mistakes that have been seen by introduction of ISTQB. Dorothy and several who comments on her blog has identified a few things to consider.
A certificate tells one story about a person, but it is very fragile. I prefer talking to references, looking at their renown, public appearance, blogs, articles and papers would perhaps give a more vivid story.
References
[1] Varför certifiera sig inom krav och test – http://informatorutbildning.blogspot.se/2012/10/varfor-certifiera-sig-inom-krav-och-test_10.html
[2] Testers Greatest Nemesis – http://thetesteye.com/blog/2011/05/testers-greatest-nemesis/
Often, when I have to read rants against certification, it comes down to this: a certificate cannot replace experience.
Well, duh. That’s a no-brainer.
But what you all seem to forget is this: a certification course can well assure that a BEGINNER in the field gets a good start.
Remember when you did all that self-learning? All the mistakes that you made, some of them painful? Yes, it is possible. But it hurts not only you but others around you that depend on your expertise while you learn. And it takes time.
Starting with a course lets a beginner learn from the mistakes of others, getting over that whole painful period of errors much sooner and avoiding the really big blunders.
You use a certificate to guarantee a bottom line in a candidate. And I use the things learned in a certification course as a net, in which I gather the real experience. Without the net, experience just drifts by, the lessons to be learned meaningless. With the net, I can draw better conclusions and better link the experiences to theories.
In short: a certificate is not the end of learning, it is the beginning.
I believe there are other ways than getting these certs to get a good start in a career. I am not speaking of all certification in the world. I believe you get a cert or diploma when you are done with all three BBST courses. From what I have heard they are not easy to do. The cost of getting them is probably covering minimal expenses for those involved.
I agree mostly with what James Bach and other write about certification. I added a different view to it based on my experiences as well as wondering about the intent of the REQB cert. Is it to make money or is it something else?
dictadicit said:
“In short: a certificate is not the end of learning, it is the beginning.”
If only they were regularly advertised and thought of in that way.
Agree with Jonathan…
There is a much bigger problem than the certifications themselves… the weight that organisations put on them!
Example – Cutting CVs from from a list because they are not certified.
The organisations need to /understand/ exactly what the certification gives a person before making such detrimental decisions.
[…] Another certification, another scam Written by: Martin Jansson […]
Martin, unfortunately you misunderstood me. I do not put an equal sign between a certificate and experience. As dictadicit says, it is a good start. One problem is that universities (until recently) have not provided any good test or requirements courses. So how should people learn about the tools in the toolbox? A course is a good start and you tend to get more focused if there is an exam involved as well. But after that the real challange comes, to use the tools, get acquainted and later on experienced, which is something you do not become attending a course.
I agree that within many professions there is always the possibility to get different ceritificates and still be a bad practioner. That is why you interview people and follow-up on references. I also agree with David that sometimes organisations put to much weight on the certifications. I do now know if people are lazy or what it is but it happends too often.
From my point of view it is all about knowledge, learning and experience. Some people hate certifications for different reasons and would do anything to get rid of them if they could. A certification is not equal to experienced. It says that you know about the tools and processes, and how they should be used. It does not say if you are experienced or not, if you do not have an ISTQB Expert certificate … 😉
Magnus, thanks for your clarification.
Independent of what you wrote, I am still worried about the intent of the REQB and how it will be used by consultancies to make money.
Regarding the certification as a good start.
I have always been an eager learner som when ISEB showed up I took the exam and when the advanced level was added I took that as well. I managed to get through the exams by assuming was was expected to anser on the questions and after that tried to apply my new learnings. I found that really hard.
I actually turned out that a lot of the content – not all of course – was completely wrong in my context. This happened over and over again. Take the example on the test design technique state modellng. A typical exam question is to read a model and determine which transitions are possible and which are not. This is the equivalence of solving puzzles for children. The most important lesson I have on this is that the actual process of creating the model and the constant questioning of the model is what realy brings value. Accepting hat something is true just because there is a model saying so is the wrong mindset of a tester. My job is to try to find situations where thing are wrong and to correct the model.
The focus on heavy documentation and detailed procudure is really bad for you. I have to disregard and to relearn and understand what really good testing is all about – let me tell you – it is not what is said in the ISTQB syllabus.
Getting to the point – I do not think the current certification is a good start – on the contrary it can be a really bad start for a novice and one that can lead them down the wrong track. How come the rest of the world (yes I am exaggerating) is trying to eliminate waste while a whole bunch of pretty experienced testers are trying to create more? Should we go back to the waterfall model that was a mstake from the beginning if you read the whole Winston Royce article.
I am proud of beeing part of a movement that constantly questions what we know about testing instead of cementing old truths that where only hypothesis in the first place and proved to be wrong later in plenty of practical situations.
Magnus,
Do you seriously believe that a certification of the kind you write about is evidence that one know “how they should be used” (regarding tools and processes). What in the certification exam supports you to make that conclusion?
Also just because a ISTQB or REQB certification CAN be used as a starting point does not mean that it by default IS a good starting point to set a person’s career of to, it can equally well be a starting point heading in the wrong direction. In that case it can be devastating for a person future learning’s and development.
As Martin points out, there are many other things to choose from as a starting point and I recommend to be very careful about what starting point you choose especially when it is some other persons career you are playing with.
Dear all,
ISTQB Foundation is not the only starting point, a testers life is full of choices, but the thing is you have to start somewhere. If we start with the state transition modelling I agree it is in the dialog, the questioning of the model, things happens. But to be able to take part in a discussion like that you have to know what a state transition model is before you can take part and contribute. Of course we can involve inexperienced people in the discussion for dfferent reasons but the very experienced person will have difficulties discussing with the inexperienced (Dreyfus and other models show this).
Neither do I like a lot of documentation, but some companies and standards require it. We are all aware of the different testing schools. Therefore, it is your responsibility as a teacher to share your experience with the course attendees and tell the about the world out there and how different companies work in different ways. It is also about how you design exercises to get the most out of a course and actually learn something.
Certification or not certification? As I pointed out before, I do not put an equalsign between certification and experience with a tool (or process). Depening on the type of course you can get more or less familiar with the tools. But we all have to agree by attending a course you do not get experienced enough to select good testing heuristics. As new to the test professions you need a starting point.
You are all very experienced and you all seems to agree that ISTQB (or REQB) is not a good starting point, but what do you actually suggest as a starting point? If ISTQB and the certifications now is such a “terrible” thing, why not improve it instead of shooting at it? 🙂
Instead of echoing what Martin, Jonathan, Henrik and Tobbe have already said (and that I agree with), I’ll respond directly to that last comment.
Magnus wrote: “If ISTQB and the certifications now is such a “terrible” thing, why not improve it instead of shooting at it?”
Why would I help improve somebody else’s money making scheme?
That might seem a facetious remark, but in fact I’m being rather serious. Why not instead support community-driven or non-profit initiatives where whatever profits are made would actually be put invested back into the community instead into some John Doe’s retirement fund?
And the other question: “…but what do you actually suggest as a starting point?”
No surprise to those who know me… The Association for Software Testing is a US based 501(c) organization (non-profit) that rank much higher on my list of worthy organizations to support. Part of their mission statement include: “Support the teaching of software testing by encouraging projects to develop and publish resources that assist classroom presentation, grading, and self-study.”
In my mind, the AST’s Foundations course material for black box software testing, available for free (as in beer) under a creative commons license online, is a much better “starting point” for junior software testers than anything I’ve seen from the ISTQB. It teaches testers the foundations of what it’s like to think like a tester, instead of mainly focusing on a glossary… And when I send testers to the instructor lead AST courses, I know that the money I invest will benefit both my employees and the community and be used to continue the development of good educational material for testers.
That’s my suggestion. Or one of them at least.
“ISTQB Foundation is not the only starting point, a testers life is full of choices, but the thing is you have to start somewhere.”
Okay. That’s fine. How about we start by rejecting bullshit money-grabs?
Let me tell you about my personal history with this. When the ISTQB was being set up, I was approached by a pair of fellows who took me out to dinner and suggested that I become the leader of the Canadian contingent. What struck me about their pitch was the focus on how profitable it would be for me. And it looks like it could have been pretty profitable.
I urge people to do the math: where does the money go? Let’s assume, for the sake of argument, that the certification exam costs $150 (it’s more expensive in some places, cheaper in others). According to the ISTQB, “As of March 2012, ISTQB® has issued over 240.000 certifications in more than 70 countries world-wide, with a current growth rate of approximately 10.000 new certifications being issued per quarter.” That’s $36,000,000. Where, pray tell, has that money gone?
What we have here is effectively a tax on new testers and on companies dumb enough to pay to “certify” testers who are already in their employ. That’s why I’m not interested in improving it instead of shooting at it. Actually, that’s only one reason.
“A certification is not equal to experienced,” says Magnus. But it is claimed, in the ISTQB’s name, that a certification is equal to qualified. A 40-question certification exam is checking. It checks whether the subject has memorized a pat answer to a question that has one, exactly one, and only one, right answer with respect to the exam. Like is not like that. The exam does not evaluate whether the subject has any knowledge of alternative interpretations of the question or the answers. It does not evaluate how the exam-taker will respond in an environment that does not use the ISTQB’s vocabulary across the board (and no organization in the world uses every element of the ISTQB’s glossary). It does not evaluate whether the person can apply the memorized knowledge in any way.
Another ploy that one tends to hear from ISTQB people is how important training and education are. I agree with that. In fact, that’s a service I provide too. But my question is this: what does the exam have to do with the training?
It’s soooo tedious having to continue this discussion five years after giving this talk. http://www.developsense.com/presentations/notyetcertified.pdf. Nothing has changed, except that about 180,000 more people have been parted from their money.
—Michael B.
Magnus, could you please provide an example where you see a real value in a certificate and describe in what ways that value can be proven?
I see a couple of problems here. First of all some are against the certifications due to the money issue and some are against it becuase of the content in the syllabus. In some comments it is a mix of these opinions. So what is the main problem?
Did you skip all your exams at the univeristy? Has someone questioned your education due to the fact that the universities get money for students passing their exams? You can attend an ISTQB course but you do not have to write the exam and you can write the exam without attending a course as well. It is up to you if you would like to spend the money taking the exam or not. I am not the one forcing you to do it one way or the other.
I teach the ISTQB Foundation but I also teach a number of other courses that has nothing to do with ISTQB (or REQB). There are similarities and you are free to attend one of those instead. From my point of view, relevant exercises are important to actually get more familiar with the tools then just reading about them, independently if it is an ISTQB course or not. This is what makes a course outstanding compared average.
Even though ISTQB states that you are qualified (equal to experienced?!?) this is not my opionion even though some people seems to belive so. But I do think that you tend to get a little bit more focused on a course if there is an exam. I do also think that if a course is accredited you know what it shall contain as a minimum and then you can discuss and compare what you have learned with other people attending the same course (but given by some other provider).
At the end of the day it is a free country and you are allowed to make your own choices.
If there are two strong issues with ISTQB certification, does it matter which is the main reason?
For example: Killing another person is bad because: 1. you prevent them from accomplishing anything else, 2. you create a deep loss for their family & friends and 3. you may go to jail. Does it really matter which of those you consider to be the “main” reason? It is still not at good idea to kill another person.
The point is there are multiple reasons that ISTQB certification is harmful to the profession of Software Testing. In addition to the money grab, false marketing, and ineffective exam there is also the attempt to teach the “way to test” without consideration of the situation.
I reach the Rapid Software Testing course which stresses that there is NOT one way to test. You must apply different approaches to different situations. I also teach how to minimize ineffective documentation and maximize tester engagement in the act of actually testing.
You also state that “it is a free world” but if uninformed HR departments pre-screen employees based on a certification that my 12 yr old son could obtain then the freedom of choice is actually being dictated by the false marketing claims of the ISTQB.
You could do the profession a huge service by helping in the fight against the atrocious marketing claims of the ISTQB.
Thanks to everyone who added more value to the original post.
Thanks to Magnus for diving into the discussion. This article was not supposed to be critical against you in person, just my interpretation on what you wrote.
My way of helping the test community has been by blogging, sharing information freely, helping students in testing, extending the work of others such as our 37 sources for test ideas and Software Quality Characteristics among other things. I believe in Open Source Testing, where we publish the artifacts in our testing to further develope research in the test domain.
I try to share as much as I can with everyone else. For me, it is much better than honing the current certificates.
I was discussing ISQTB with an ISQTB trainer recently.
I made the observation that the ISQTB approach does not fit well with an Agile approach (due to over emphasis on documentation and up front planning).
The trainer told me that there is a new certification called CAT (Certified Agile Tester) which covers Agile testing.
The fact that even a trainers/expert in ISQTB admits that it doesn’t cover Agile is interesting enough.
What I find more disturbing is the attempt to over specialize testing roles into so called Manual Testers, Technical Testers, and Agile Testers.
I think that the distinctions between these roles are false and we would be better off training good all round testers.
I am all for test training and development in testing but ISQTB is attempting to own test practice.
Ultimately, some fault lies with testers themselves because many of us are unable to articulate what true testing is really about.
ISQTB has rushed into to fill that vacuum and are proving to be very successful in cementing their position in the industry.