This began as an answer to Rikard’s post http://thetesteye.com/blog/2011/06/a-word-of-caution/ where the discussion on “traditional testing” came up.

I often hear comparisons with our “industry” and the Automotive industry.
In that context, you could say that “traditional testing” corresponds to the methods and practices that are applied in line production of large car companies. And the unorthodox testing can be compared with those specialized and often smaller custom car builder companies out there.

The major issue with this kind of comparison is that large car companies makes thousands of the exact same type of car. Instead this means that the “traditional testing” approach is an attempt to apply line production methods when building custom cars. Applying “traditional testing” as if every project and product were the same is both wrong and dangerous.

And this comparison is not that far-fetched… It seems to me like “traditional testing” is promoted as practices that suits many (if not all?) projects and should be followed in order to enable success. Well, good luck!

Further, many practices that we use today comes from the automotive industry (at least in their latest form).
If we fail to see why they implemented them in the automotive industry and just take them as good practices for being effective in line production, we are doing the opposite of what the automotive industry did. They did some investigation on how they could improve their work. Some of that included seeing the human beings as intelligent and social creatures; utilize the diversity of a group of people; etc. And their productivity and efficiency could be measured by measuring number of cars and their quality. So by treating humans as humans they became more efficient (number of non-defective cars) and improved their work methods (analyzing quality of work).
When Lean development is implemented in psychiatric nursing or software development today, the focus is very often on the quantity measuring part which then misses the whole point. Measuring patients or software as uniform units is very wrong and dangerous.
It’s not that Lean or Kanban is to blame, it’s the implementation; and perhaps mostly the implementors.
The role models for Lean implementations in many healthcare institutions in Sweden have been some successful nursing teams that have increased their efficiency and quality of their work by using Lean development as a method. What those successful teams really did were to take command of their own work and found a method that supported their initiative and commitment. (Anyone had similar experience? Me!)
The problem is that when this is implemented by management at the whole hospital or nationwide, the focus is shifted from “Quality of work” to “Quantity of work” because that is the obvious driver for management and really the only incentive for they to implement it. They only need to say that this is a “best practice” and then it’s OK…
I do need to emphasize that you don’t automatically get high quality work by implementing “best practises”!

This is happening all the time; and the last flavor of the month is Agile.
If you read the Agile Manifesto and then think that you must play planning poker or have standup meetings you are obviously not understanding the Agile Manifesto.
I’m with Matt Heusser on his interpretation here: http://www.softwaretestpro.com/Item/5036/My-%27take%27-on-Agile—-or-the-Manifesto-elaborated/Testing-Software-Test-and-QA-Teams-Strategy-Agile-Development

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise.

Quote taken from Ward Cunningham in “The WyCash Portfolio Management System” [1] from 1992. Martin Fowler further elaborates around the subject “Technical Debt” [2] and identifies the “Technical Debt Quadrant” [3], where he makes a distinction between prudent and reckless debt as well as deliberate and inadvertent debt. These four debts creates the quadrant, as Martin sees it. Uncle Bob points out in his article “A mess is not a debt” [4] that some deliberate shortcuts should not be considered as a technical debt, but instead are just a mess. I think “A mess” could be seen as Broken Windows (another metaphor) that in the end turn into a debt you must pay, thus I agree with the comment that Daniel Broman had. Steve McConnell discuss the intentional and unintentional technical debt in an article [5]. He also elaborates around short-term and long-term debt. One among many important suggestion that he brings up is to maintain a debt list connected with workpackages needed to pay off a specific debt.

If the shortcut you are considering taking is too minor to add to the debt-service defect list/product backlog, then it’s too minor to make a difference; don’t take that shortcut. We only want to take shortcuts that we can track and repair later.

The quote above from Steve McConnel implies an openness to the debt that I like. It also implies we should focus on debt that we make a strategic decision on. The comments to this article also sheds more light to debt metaphor. Andy Lester has a talk in “Get out of Technical Debt Now!” [6], where he identifies many issues that are not strictly technical but instead more in the  social domain. There are many aspects that I agree with, but I would consider them as another kind of debt.

Ted Theodoropoulos has a series of articles where he defines [7] Technical Debt, then goes into how to identify [8] it, how to quantify [9] it, how to plan a remediation [10] and finally how to governance [11] it. Ted goes deeper than the previous authors have gone and is perhaps a bit more structured in his definitions, as I see it. Still, the sum of all views from all authors is imporant. All of these developers use the Debt metaphor to make it easier to communicate to different stakeholders.

By now you should have a good understanding what Technical Debt is and perhaps have some ideas on how we, as testers, can help our fellow developers with the Technical Debt. None of the authors above mention the usage of testing or QA in this. If we know there is a technical debt that is ever increasing, why is it so seldome that we talk about it openly in the project between tester and developer? It is a fact that developers cannot leave the code in as good shape as they always want it and that they must take shortcuts. Some of the authors above have stated to have a list of these shortcuts, open issues and bugs before hand. This would give us testers lots of valuable information that would make our testing even better. If the technical debt is a continuous struggle for the developers, we as testers are there to help!

Jonathan Kohl has touched the subject before in this article about “Testing Debt” [12] that partly relates to the subject. Johanna Rothman has in her two articles about “What Testers Can Do about Technical Debt” [13] and [14]. I will try to build on their ideas.

1. Work closer to the developers, gain their trust. Assist them so that you understand when certain shortcuts are not sloppyness, but deliberate with known risks. Bugs found in these areas might not be a big surprise to them.
2. I would improve communication within the project to a such a degree where we can talk about the technical debt (and possibly testing debt) openly and so that we can start pinning down the debt lists, assuming we have both testing debt and technical debt (developer debt). With testing debt I am referring to my own definition in the article “Turning the tide of bad testing” [15]. Still, there are many aspects of the debts that are social factors that are not so easy to talk about, but does that mean that we should not add it to the list? It might be that it is too hard to mix the issues that are directly related to code with social factors. In that case we might be talking about different kinds of debt for different situations and decisions.
3. I would assume that the developers know which content in the of debt list they will consider for fixing soon and which will be postponed for the far far future. We would be able to talk about what areas should have higher priority so that we can put focus where it belongs.
4. If we know an area is broken (dead horse heuristic) and that it will be fixed in the near future, there are probably other areas that could getter higher attention.
5. Areas which the developer consider are “safe” to postpone into the far future, we as testers could verify their claims of certainty or identify new risks that make them reconsider.
6. We could just see the debt as a risk area in itself. Test it and adding bugs as additional cost of having the debt left behind and as an aid to pinpoint where to get the most of a payback.
7. A tester could be involved in the decision for the strategic debt, perhaps to evaluate the decision and to look for possible issues that will threaten the outcome of increasing the technical debt.
8. Debt that is related to testability should perhaps be reconsidered. As testers, we would not be able to help out as much if this was threatened.
9. Try to talk openly with developers about non-strategic debt (as Ted defines it), thus areas which has poorly written code or something similar. By being open about the situation will enable you as tester to help out more.
10. When development start to payback on the debt you can work closely with the developers to decrease the introduction of new bugs and new areas that increase debt.
11. …

What would you consider adding to this list that I have left out?

References:

[1] The WyCash Portfolio Management System - http://c2.com/doc/oopsla92.html

[2] TechnicalDebt - http://martinfowler.com/bliki/TechnicalDebt.html

[3] Technical Debt Quadrant - http://martinfowler.com/bliki/TechnicalDebtQuadrant.html

[4] A mess is not a technical debt - http://blog.objectmentor.com/articles/2009/09/22/a-mess-is-not-a-technical-debt

[5] Technical Debt - http://blogs.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx

[6] Get out of Technical Debt Now! - http://www.media-landscape.com/yapc/2006-06-26.AndyLester/

[7] Technical Debt – Definition - http://blog.acrowire.com/technical-debt/technical-debt-part-1-definition/

[8] Technical Debt – Identification - http://blog.acrowire.com/technical-debt/technical-debt-part-2-identification/

[9] Technical Debt – Quantifying - http://blog.acrowire.com/technical-debt/technical-debt-part-3-quantifying/

[10] Technical Debt – Remediation - http://blog.acrowire.com/technical-debt/technical-debt-part-4-remediation/

[11] Technical Debt – Governance - http://blog.acrowire.com/technical-debt/technical-debt-part-5-governance/

[12] Testing Debt – http://www.kohl.ca/blog/archives/000148.html

[13] What Testers Can Do about Technical Debt – Part 1 - http://www.stickyminds.com/s.asp?F=S3629_COL_2

[14] What Testers Can Do about Technical Debt – Part 2 - http://www.stickyminds.com/s.asp?F=S3643_COL_2

[15] Turning the tide of bad testing - http://thetesteye.com/blog/2010/11/turning-the-tide-of-bad-testing/

I am passionate about testing. This passion gives me energy,  the fire that makes me what to excel and get better at testing. It is this passion that I share with my co-workers, customers and my employees, hopefully growing their interest in testing and sharing my visions and goals. I use it to paint my views on a better tomorrow in the testing community, as I see it.

Would this passion then make us more biased in our testing than someone dispassionate? I think not. Everyone is biased from time to time.

Passion is synonym with ardent, blazing, burning, dithyrambic, fervent, fervid, fiery, flaming, glowing, heated, hot-blooded, hotheaded, impassioned, perfervid, red-hot, scorching and torrid. Dispassionate can be seen as not showing, and not affected by emotion, bias, or prejudice. Would it even be possible to compare the two in a testing duel?

My CEO has told me that he want competent and honest people in his company. He says, “Imagine having competent and dishonest people, that would be terrible.” I like those traits, but I would also like to add passionate as a trait.

As a customer or employer, would you want a tester who is passionate or a tester who is dispassionate about their beliefs in testing? Isn’t it so that we are just passionate over different things, which naturally is ok?

If you haven’t read the excellent articles by Michael Bolton regarding Testing vs. Checking yet, now is a good time to do it:

http://www.developsense.com/blog/2009/08/testing-vs-checking/
http://www.developsense.com/blog/2009/09/transpection-and-three-elements-of/
http://www.developsense.com/blog/2009/09/pass-vs-fail-vs-is-there-problem-here/

Done?

One thing that struck me with this is that the more testing you do will result in less testing and more checking. I.e., the more you test, the more you know, the more heuristics you will develop; and the more you become a checker for things that you assume or already know. The experience and skill that you have gathered in order to be a good Exploratory Tester actually gives you test ideas that you really just need to check.

It is a paradox in theory, but if you think of it:
Lets say that you are the best Exploratory Tester that exist on planet Earth, and you have experience from all kinds of software development and teams/companies. Then you would have so much knowledge about products, people, software, user interaction, etc, that you eventually would sit on a gold mine of experience and knowledge. And there is nothing more to learn since you already know everything. This would mean that all tests that you do are merely checks to see if they meet the knowledge (assumption) or not.

Michael defines Checking as “Checking is something that we do with the motivation of confirming existing beliefs” and “Checking is a process of confirmation, verification, and validation“.

Well, the more you know, the more you then confirming existing beliefs.

Update 2010-03-29:

Thanks for all interesting and thoughtful comments!
I now realize that the paradox isn’t true, and cannot ever be. It was an intriguing thought that ended up being something not really true. Thanks for your help in clarifying that! Still, this post and comments have at least made me think around the subject! And it has made me wiser.

When is a system small, non-complex or unprioritzed enough not to be tested? If there is a test organisation working on the bigger system that will be released to customer, what happens to the other smaller systems then? Is it so that they are almost always left untested? I usually identify these as applications that are created by one person, they can be used in critical parts of the product development, but are not viewed at as a product, sub-system or application that needs to be tested.

During mid 90′s I worked at a small localization company. At that time I was involved in a project where we were preparing a major translation of swedish material that was going to be translated to 20 languages. In order to succeed with such a translation it was important that we first translated the material into english and then went from there to the other languages. I did a little script to fix the glossaries for the various languages. It was just a small script to prepare for translation to the various languages. At that time we had a few testers but they were either testing commercial applications or testing for our customers, so there were no resources left for my stuff to be tested. Naturally there were loads of bugs in my code. I accidently made an offset in the script for each row, so each translated language used my glossaries with an offset of the translation. The proofreaders noticed the error eventually, but it had cost the company a few millions by then.

During another project we had a problem working with our customers files. A customer required us to use a specific program that used one working folder which contained close 100000 RTF-documents. Each time a translator checked the folder to find a specific document it took him a few hours just to see the file. So, when we wanted to do many changes to many files at the same time we calculated this to take 3 years time in order to implement our changes. So, we did a little perl script to do these changes we want by searching and replacing directly in the RTF-format (very dangerous way). First we did just one little change and when we saw that this worked well, we added more. Eventually we had a quite long list of changes being done to these 100000 files. We probably saved a few hundred years of time and budget, but eventually there were bugs in the script. One of these bugs where that I thought I had fixed issues with too many spaces and replaced them with just one space, but RTF sometimes needed two or more spaces. The result was that whole sections of text disappeared totally from being shown. This was just a small tool.

At most companies there are many such small systems floating around in the organisation. Shall the test organisation take a step toward testing those? If they are so small might it be so that testing also takes little effort? How about automation tests or production tests? Are they prioritized to be tested or do they fall into the same unimportant category?

If we see testing as a service, I think it is important that the test organisation involve themselves in all development, even if it is a few minutes time. Developers also need to involve testers, to give them a chance to do their job. Naturally it will be impossible to do everything, but in some cases it only takes such a short time to find a few bugs that it is worth it.

I’ve recently finished James Bach’s book Secrets of a Buccaneer Scholar. I liked it, but I don’t agree with all of it. As a tester, I feel that it inspires me and gives me new ideas in my way of thinking and how I perceive learning, especially self-education. I fully agree with James on that you should follow your passion. If you are able to assist those around you to that end, it will make you grow even more. In march James will be in Sweden and hold a series of courses, one of them is Self-Education for testers. It think it would be very fun and educational to attend, I will see if I can make time.

One thing that I consider after reading the book is how I can inspire my daughter to learn, test new things and to follow her passion. When she receives a new toy, I want her to explore how it works. For instance, she got a little toy puppy that execute somersaults. It had a lot of mechanic inside it and sounded like it was very fragile, if you actually played with it. The purpose of the toy was probably just to watch it. I dislike such toys. I asked my daughter how she thought it worked. She did a somewhat exploratory, destructive test by enabling the puppy to do the somersault, then directly afterwards hugged it tightly. There was a popping sound in the mechanic as the puppy tried to execute the somersault, while being held secure. After that test it was not possible for the puppy to somersault, rather it performed a half one and landed on its nose. I applauded my daughters destructive sense of testing of the toy. Translating that into testing terms, she prioritized which test to start with and considered what was the biggest chance a user would do then executed that. One test to render the system useless. Wonderful!

I think we have a lot to learn (or perhaps relearn) from our children.

The first build has been delayed. I was able to test a bit on the last support issue, but it was just a quick one and it was a few weeks ago. I’ve nothing to report bugs on. Should I enter some bugs into the code myself just to quelch my thirst? Should I test some other product, perhaps an open source tool? I crave for testing something. I must have my fix! Give me something to test! I’ve spent a lot of time on planning, collecting test data, configurations and so on, but it is testing that does it for me.

I have become addicted to testing.